During snap-sync, we request ranges of values: either a range of accounts or a range of storage values. For any large trie, e.g. the main account trie or a large storage trie, we cannot fetch everything at once.
Short version; we split it up and request in multiple stages. To do so, we use an origin field, to say "Give me all storage key/values where key > 0x20000000000000000". When the server fulfils this, the server provides the first key after origin, let's say 0x2e030000000000000 -- never providing the exact origin. However, the client-side needs to be able to verify that the 0x2e03.. indeed is the first one after 0x2000.., and therefore the attached proof concerns the origin, not the first key.
So, short-short version: the left-hand side of the proof relates to the origin, and is free-standing from the first leaf.
On the other hand, (pun intended), the right-hand side, there's no such 'gap' between "along what path does the proof walk" and the last provided leaf. The proof must prove the last element (unless there are no elements).
Therefore, we can simplify the semantics for trie.VerifyRangeProof by removing an argument. This doesn't make much difference in practice, but makes it so that we can remove some tests. The reason I am raising this is that the upcoming stacktrie-based verifier does not support such fancy features as standalone right-hand borders.
This change addresses an issue in snap sync, specifically when the entire sync process can be halted due to an encountered empty storage range.
Currently, on the snap sync client side, the response to an empty (partial) storage range is discarded as a non-delivery. However, this response can be a valid response, when the particular range requested does not contain any slots.
For instance, consider a large contract where the entire key space is divided into 16 chunks, and there are no available slots in the last chunk [0xf] -> [end]. When the node receives a request for this particular range, the response includes:
The proof with origin [0xf]
A nil storage slot set
If we simply discard this response, the finalization of the last range will be skipped, halting the entire sync process indefinitely. The test case TestSyncWithUnevenStorage can reproduce the scenario described above.
In addition, this change also defines the common variables MaxAddress and MaxHash.
* cmd, core: resolve scheme from a read-write database
* cmd, core, eth: move the scheme check in the ethereum constructor
* cmd/geth: dump should in ro mode
* cmd: reverts
This change
- Removes the owner-notion from a stacktrie; the owner is only ever needed for comitting to the database, but the commit-function, the `writeFn` is provided by the caller, so the caller can just set the owner into the `writeFn` instead of having it passed through the stacktrie.
- Removes the `encoding.BinaryMarshaler`/`encoding.BinaryUnmarshaler` interface from stacktrie. We're not using it, and it is doubtful whether anyone downstream is either.
* eth: enforce announcement metadatas and drop peers violating the protocol
* eth/fetcher: relax eth/68 validation a bit for flakey clients
* tests/fuzzers/txfetcher: pull in suggestion from Marius
* eth/fetcher: add tests for peer dropping
* eth/fetcher: linter linter linter linter linter
This is a minor refactor in preparation of changes to range verifier. This PR contains no intentional functional changes but moves (and renames) the light.NodeSet
This PR will allow a previously underpriced transaction back in after a timeout
of 5 minutes. This will block most transaction spam but allow for transactions to
be re-broadcasted on networks with less transaction flow.
---------
Co-authored-by: Felix Lange <fjl@twurst.com>
This PR makes EIP-4788 work in the engine API and miner. It also fixes some bugs related to
EIP-4844 block processing and mining. Changes in detail:
- Header.BeaconRoot has been renamed to ParentBeaconRoot.
- The engine API now implements forkchoiceUpdatedV3
- newPayloadV3 method has been updated with the parentBeaconBlockRoot parameter
- beacon root is now applied to new blocks in miner
- For EIP-4844, block creation now updates the blobGasUsed field of the header
ReadSkeletonHeader can return nil if the header is missing, so we should
not access fields on it. Note that calling .Hash() on a nil header is fine, so there
is no need to actually check for nil.
Co-authored-by: Martin Holst Swende <martin@swende.se>
This changes the forkID calculation to ignore time-based forks that occurred before the
genesis block. It's supposed to be done this way because the spec says:
> If a chain is configured to start with a non-Frontier ruleset already in its genesis, that is NOT considered a fork.
This PR removes the newly added txpool.Transaction wrapper type, and instead adds a way
of keeping the blob sidecar within types.Transaction. It's better this way because most
code in go-ethereum does not care about blob transactions, and probably never will. This
will start mattering especially on the client side of RPC, where all APIs are based on
types.Transaction. Users need to be able to use the same signing flows they already
have.
However, since blobs are only allowed in some places but not others, we will now need to
add checks to avoid creating invalid blocks. I'm still trying to figure out the best place
to do some of these. The way I have it currently is as follows:
- In block validation (import), txs are verified not to have a blob sidecar.
- In miner, we strip off the sidecar when committing the transaction into the block.
- In TxPool validation, txs must have a sidecar to be added into the blobpool.
- Note there is a special case here: when transactions are re-added because of a chain
reorg, we cannot use the transactions gathered from the old chain blocks as-is,
because they will be missing their blobs. This was previously handled by storing the
blobs into the 'blobpool limbo'. The code has now changed to store the full
transaction in the limbo instead, but it might be confusing for code readers why we're
not simply adding the types.Transaction we already have.
Code changes summary:
- txpool.Transaction removed and all uses replaced by types.Transaction again
- blobpool now stores types.Transaction instead of defining its own blobTx format for storage
- the blobpool limbo now stores types.Transaction instead of storing only the blobs
- checks to validate the presence/absence of the blob sidecar added in certain critical places
The Go authors updated golang/x/ext to change the function signature of the slices sort method.
It's an entire shitshow now because x/ext is not tagged, so everyone's codebase just
picked a new version that some other dep depends on, causing our code to fail building.
This PR updates the dep on our code too and does all the refactorings to follow upstream...
This change removes a chainconfig parameter passed into rawdb.ReadLogs, which is not used nor needed.
It also modifies the filter loop slightly, avoiding a labeled break and instead using a method.
This change does not modify any behaviour.
* core/blobpool: implement txpool for blob txs
* core/txpool: track address reservations to notice any weird bugs
* core/txpool/blobpool: add support for in-memory operation for tests
* core/txpool/blobpool: fix heap updating after SetGasTip if account is evicted
* core/txpool/blobpool: fix eviction order if cheap leading txs are included
* core/txpool/blobpool: add note as to why the eviction fields are not inited in reinject
* go.mod: pull in inmem billy form upstream
* core/txpool/blobpool: fix review commens
* core/txpool/blobpool: make heap and heap test deterministic
* core/txpool/blobpool: luv u linter
* core/txpool: limit blob transactions to 16 per account
* core/txpool/blobpool: fix rebase errors
* core/txpool/blobpool: luv you linter
* go.mod: revert some strange crypto package dep updates
This is a spin-out from the EIP-4844 devnet branch, containing just the Engine API modifications
and nothing else. The newPayloadV3 endpoint won't really work in this version, but we need the
data structures for testing so I'd like to get this in early.
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
This change removes PoW header syncing related code from LES and also deletes
duplicated packages les/catalyst, les/downloader and les/fetcher. These package copies
were created because people wanted to make changes in their eth/ counterparts, but weren't
able to adapt LES code to the API changes.
The struct logger (or opcode tracer) was missing the return data field even
if this was explicitly enabled by user via `"enableReturnData": true` in the config.
This PR fixes this issue.
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
This change makes the StateDB track the state key value diff of a block transition.
We already tracked current account and storage values for the purpose of updating
the state snapshot. With this PR, we now also track the original (pre-transition) values
of accounts and storage slots.
This change adds back the 'geth --dev' mode of operation, using a cl-mocker.
---------
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
Co-authored-by: lightclient <14004106+lightclient@users.noreply.github.com>
The clean trie cache is persisted periodically, therefore Geth can
quickly warmup the cache in next restart.
However it will reduce the robustness of system. The assumption is
held in Geth that if the parent trie node is present, then the entire
sub-trie associated with the parent are all prensent.
Imagine the scenario that Geth rewinds itself to a past block and
restart, but Geth finds the root node of "future state" in clean
cache then regard this state is present in disk, while is not in fact.
Another example is offline pruning tool. Whenever an offline pruning
is performed, the clean cache file has to be removed to aviod hitting
the root node of "deleted states" in clean cache.
All in all, compare with the minor performance gain, system robustness
is something we care more.
The state availability is checked during the creation of a state reader.
- In hash-based database, if the specified root node does not exist on disk disk, then
the state reader won't be created and an error will be returned.
- In path-based database, if the specified state layer is not available, then the
state reader won't be created and an error will be returned.
This change also contains a stricter semantics regarding the `Commit` operation: once it has been performed, the trie is no longer usable, and certain operations will return an error.
This removes the feature where top nodes of the proof can be elided.
It was intended to be used by the LES server, to save bandwidth
when the client had already fetched parts of the state and only needed
some extra nodes to complete the proof. Alas, it never got implemented
in the client.
* all: move main transaction pool into a subpool
* go.mod: remove superfluous updates
* core/txpool: review fixes, handle txs rejected by all subpools
* core/txpool: typos
We had to do this workaround because it wasn't possible to export typed arrays from
JS to []byte. This was added in dop251/goja@2352993, so we can use the better way now.
* core/txpool: abstraction prep work for secondary pools (blob pool)
* core/txpool: leave subpool concepts to a followup pr
* les: fix tests using hard coded errors
* core/txpool: use bitmaps instead of maps for tx type filtering
Drop the notions of uncles, and disables activities while syncing
- Disable activities (e.g. generate pending state) while node is syncing,
- Disable empty block submission (but empty block is still kept for payload building),
- Drop uncle notion since (ethash is already deprecated)
eth: make StorageRangeAt take a block hash or number
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
This change implements async log retrievals via feeding logs in channels, instead of returning slices. This is a first step to implement #15063.
---------
Signed-off-by: jsvisa <delweng@gmail.com>
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Sina Mahmoodi <1591639+s1na@users.noreply.github.com>
ethclient accepts certain negative block number values as specifiers for the "pending",
"safe" and "finalized" block. In case of "pending", the value accepted by ethclient (-1)
did not match rpc.PendingBlockNumber (-2).
This wasn't really a problem, but other values accepted by ethclient did match the
definitions in package rpc, and it's weird to have this one special case where they don't.
To fix it, we decided to change the values of the constants rather than changing ethclient.
The constant values are not otherwise significant. This is a breaking API change, but we
believe not a dangerous one.
---------
Co-authored-by: Felix Lange <fjl@twurst.com>
This PR modifies the interface for the results of `debug_traceBlock` and `debug_traceCall` by adding the `txHash`, allowing users to identify which transaction's trace result corresponds to.
---------
Co-authored-by: Martin Holst Swende <martin@swende.se>
Adds an optional config parameter to eth_call which allows users to override block context fields (same functionality that was added to traceCall in #24871)
---------
Co-authored-by: Martin Holst Swende <martin@swende.se>
Péter Szilágyi
Martin Holst Swende
rjl493456442
hyunchel
lightclient
phenix3443
Marius van der Wijden
bnovil
Sina Mahmoodi
Delweng
Felix Lange
jwasinger
Joe Netti
Shude Li
Felföldi Zsolt
Ryan Schneider
Guillaume Ballet
Dan Laine
hero5512
ucwong
minh-bq
makcandrov
sjlee1125
chirag-bgh