Commit Graph

2529 Commits

Author SHA1 Message Date
Péter Szilágyi 593e303485
core/txpool, eth, miner: pre-filter dynamic fees during pending tx retrieval (#29005)
* core/txpool, eth, miner: pre-filter dynamic fees during pending tx retrieval

* miner: fix typo

* core/txpool: handle init-error in blobpool without panicing

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2024-02-17 13:37:14 +02:00
Sina Mahmoodi 95741b1844
core: move genesis alloc types to core/types (#29003)
We want to use these types in public user-facing APIs, so they shouldn't be in core.

Co-authored-by: Felix Lange <fjl@twurst.com>
2024-02-16 19:05:33 +01:00
Péter Szilágyi 3c30de219f
core/txpool/blobpool: update the blob db with corruption handling (#29001)
Updates billy to a more recent version which is more robust in the face of corrupt data (e.g. after a hard crash)
2024-02-16 16:33:14 +01:00
colin a193bb0c73
core/txpool/legacypool: remove a redundant heap.Init (#28910)
Co-authored-by: Martin HS <martin@swende.se>
Co-authored-by: Felix Lange <fjl@twurst.com>
2024-02-15 19:50:17 +01:00
bk efddedc16c
core/txpool/blobpool: rename variables in comments (#28981)
Co-authored-by: Felix Lange <fjl@twurst.com>
2024-02-15 10:20:10 +01:00
Martin HS 8321fe2fda
tests: fix goroutine leak related to state snapshot generation (#28974)
---------

Co-authored-by: Felix Lange <fjl@twurst.com>
2024-02-14 17:02:56 +01:00
rjl493456442 fe91d476ba
all: remove the dependency from trie to triedb (#28824)
This change removes the dependency from trie package to triedb package.
2024-02-13 14:49:53 +01:00
Ng Wei Han beb2954fa4
core/txpool/legacypool: use uint256.Int instead of big.Int (#28606)
This change makes the legacy transaction pool use of `uint256.Int` instead of `big.Int`. The changes are made primarily only on the internal functions of legacypool. 

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2024-02-13 10:10:11 +01:00
Sina Mahmoodi ac5aa672d3
internal/ethapi: add support for blobs in eth_fillTransaction (#28839)
This change adds support for blob-transaction in certain API-endpoints, e.g. eth_fillTransaction. A follow-up PR will add support for signing such transactions.
2024-02-08 19:53:32 +01:00
lightclient 2732fb10d2
params, core/forkid: add mainnet timestamp for Cancun (#28958)
* params: add cancun timestamp for mainnet

* core/forkid: add test for mainnet cancun forkid

* core/forkid: update todo tests for cancun
2024-02-08 19:36:38 +01:00
zoereco 2ab365f6d8
all: fix docstring names (#28923)
* fix wrong comment

* reviewers input

* Update log/handler_glog.go

---------

Co-authored-by: Martin HS <martin@swende.se>
2024-02-07 21:10:49 +01:00
lightclient 449d3f0d87
core,params: add holesky to default genesis function (#28903) 2024-02-07 17:19:14 +01:00
lightclient 1f50aa7631
cmd,internal/era: implement `export-history` subcommand (#26621)
* all: implement era format, add history importer/export

* internal/era/e2store: refactor e2store to provide ReadAt interface

* internal/era/e2store: export HeaderSize

* internal/era: refactor era to use ReadAt interface

* internal/era: elevate anonymous func to named

* cmd/utils: don't store entire era file in-memory during import / export

* internal/era: better abstraction between era and e2store

* cmd/era: properly close era files

* cmd/era: don't let defers stack

* cmd/geth: add description for import-history

* cmd/utils: better bytes buffer

* internal/era: error if accumulator has more records than max allowed

* internal/era: better doc comment

* internal/era/e2store: rm superfluous reader, rm superfluous testcases, add fuzzer

* internal/era: avoid some repetition

* internal/era: simplify clauses

* internal/era: unexport things

* internal/era,cmd/utils,cmd/era: change to iterator interface for reading era entries

* cmd/utils: better defer handling in history test

* internal/era,cmd: add number method to era iterator to get the current block number

* internal/era/e2store: avoid double allocation during write

* internal/era,cmd/utils: fix lint issues

* internal/era: add ReaderAt func so entry value can be read lazily

Co-authored-by: lightclient <lightclient@protonmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>

* internal/era: improve iterator interface

* internal/era: fix rlp decode of header and correctly read total difficulty

* cmd/era: fix rebase errors

* cmd/era: clearer comments

* cmd,internal: fix comment typos

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2024-02-07 09:18:27 -07:00
lmittmann 199e0c9ff5
core/state, core/vm: minor uint256 related perf improvements (#28944) 2024-02-07 17:01:38 +01:00
rjl493456442 0b5d8d2b58
core: cache transaction indexing tail in memory (#28908) 2024-02-06 10:44:42 +08:00
Dimitris Apostolou 8fd43c8013
all: fix typos in comments (#28881) 2024-02-05 22:16:32 +01:00
zoereco 253447a4f5
core/types: fix typo (#28922) 2024-02-04 13:55:30 +08:00
Péter Szilágyi 47d76c5f95
core/txpool: don't inject lazy resolved transactions into the container (#28917)
* core/txpool: don't inject lazy resolved transactions into the container

* core/txpool: minor typo fixes
2024-02-02 20:39:12 +02:00
Péter Szilágyi 62affdc9c5
core/txpool/blobpool: post-crash cleanup and addition/removal metrics (#28914)
* core/txpool/blobpool: clean up resurrected junk after a crash

* core/txpool/blobpool: track transaction insertions and rejections

* core/txpool/blobpool: linnnnnnnt
2024-02-02 18:26:35 +02:00
rjl493456442 eaac53ec38
core: reset tx lookup cache if necessary (#28865)
This pull request resets the txlookup cache if chain reorg happens, 
preventing them from remaining reachable. It addresses failures in
the hive tests.
2024-01-30 09:34:14 +08:00
protolambda 2e947b7a00
core/types: fix and test handling of faulty nil-returning signer (#28879)
This adds an error if the signer returns a nil value for one of the signature value fields.
2024-01-27 21:16:20 +01:00
rjl493456442 6b0de79935
core: move tx indexer to its own file (#28857)
This change moves all the transaction indexing functions to a separate txindexer.go file and defines a txIndexer structure as a refactoring.
2024-01-23 21:00:50 +01:00
Péter Szilágyi 542c861b4f
core/txpool, eth/catalyst: fix racy simulator due to txpool background reset (#28837)
This PR fixes an issues in the new simulated backend. The root cause is the fact that the transaction pool has an internal reset operation that runs on a background thread.

When a new transaction is added to the pool via the RPC, the transaction is added to a non-executable queue and will be moved to its final location on a background thread. If the machine is overloaded (or simply due to timing issues), it can happen that the simulated backend will try to produce the next block, whilst the pool has not yet marked the newly added transaction executable. This will cause the block to not contain the transaction. This is an issue because we want determinism from the simulator: add a tx, mine a block. It should be in there.

The PR fixes it by adding a Sync function to the txpool, which waits for the current reset operation (if any) to finish, and then runs an entire round of reset on top. The new round is needed because resets are only triggered by new head events, so newly added transactions will not trigger the outer resets that we can wait on. The transaction pool would eventually internally do a reset even on transaction addition, but there's no easy way to wait on that and there's no meaningful reason to bubble that across everything. A clean outer reset will at worse be a small noop goroutine.
2024-01-23 20:59:38 +01:00
trocher 2dc74770a7
core/vm: fix misleading comment (#28860)
fix misleading comment
2024-01-23 15:17:42 +01:00
Marius van der Wijden c89a3da7d9
core/state/snapshot: use AddHash/ContainHash instead of Hasher interface (#28849)
This change switches from using the `Hasher` interface to add/query the bloomfilter to implementing it as methods.
This significantly reduces the allocations for Search and Rebloom.
2024-01-23 15:15:48 +01:00
Martin HS a5a4fa7032
all: use uint256 in state (#28598)
This change makes use of uin256 to represent balance in state. It touches primarily upon statedb, stateobject and state processing, trying to avoid changes in transaction pools, core types, rpc and tracers.
2024-01-23 14:51:58 +01:00
Darioush Jalali 819a4977e8
core: fix genesis setup in benchReadChain (#28856) 2024-01-23 21:46:34 +08:00
rjl493456442 78a3c32ef4
core, core/rawdb, eth/sync: no tx indexing during snap sync (#28703)
This change simplifies the logic for indexing transactions and enhances the UX when transaction is not found by returning more information to users.

Transaction indexing is now considered as a part of the initial sync, and `eth.syncing` will thus be `true` if transaction indexing is not yet finished. API consumers can use the syncing status to determine if the node is ready to serve users.
2024-01-22 21:05:18 +01:00
Marius van der Wijden f55a10b64d
params, core/forkid: enable cancun on sepolia and holesky (#28834)
This change enables Cancun 

- Sepolia at 1706655072 (Jan 31st, 2024)
- Holesky at 1707305664 (Feb 7th, 2024)

Specification: https://github.com/ethereum/execution-specs/pull/860
2024-01-20 16:03:14 +01:00
Felix Lange 0e93da3197
crypto/kzg4844: add helpers for versioned blob hashes (#28827)
The code to compute a versioned hash was duplicated a couple times, and also had a small
issue: if we ever change params.BlobTxHashVersion, it will most likely also cause changes
to the actual hash computation. So it's a bit useless to have this constant in params.
2024-01-19 11:41:17 +01:00
Sina Mahmoodi e5d5e09faa
internal/ethapi: handle blobs in API methods (#28786)
EIP-4844 adds a new transaction type for blobs. Users can submit such transactions via `eth_sendRawTransaction`. In this PR we refrain from adding support to `eth_sendTransaction` and in fact it will fail if the user passes in a blob hash.

However since the chain can handle such transactions it makes sense to allow simulating them. E.g. an L2 operator should be able to simulate submitting a rollup blob and updating the L2 state. Most methods that take in a transaction object should recognize blobs. The change boils down to adding `blobVersionedHashes` and `maxFeePerBlobGas` to `TransactionArgs`. In summary:

- `eth_sendTransaction`: will fail for blob txes
- `eth_signTransaction`: will fail for blob txes

The methods that sign txes does not, as of this PR, add support the for new EIP-4844 transaction types. Resuming the summary:

- `eth_sendRawTransaction`: can send blob txes
- `eth_fillTransaction`: will fill in a blob tx. Note: here we simply fill in normal transaction fields + possibly `maxFeePerBlobGas` when blobs are present. One can imagine a more elaborate set-up where users can submit blobs themselves and we fill in proofs and commitments and such. Left for future PRs if desired.
- `eth_call`: can simulate blob messages
- `eth_estimateGas`: blobs have no effect here. They have a separate unit of gas which is not tunable in the transaction.
2024-01-17 15:06:14 +01:00
Alfie John 9ee6809ff4
core/txpool/blobpool: fix typos 2024-01-15 21:45:14 +02:00
Darioush Jalali 29b73555ae
core/state: unexport GetOrNewStateObject (#28804) 2024-01-14 12:32:23 +01:00
Marius van der Wijden cfff3cbbf1
params, core/forkid: schedule cancun fork on goerli (#28719)
This PR schedules the cancun fork for the goerli testnet as discussed on ACD.
Spec: ethereum/execution-specs#860

We schedule:
  goerli at 1705473120
2024-01-08 20:33:32 +01:00
Darioush Jalali 2365d77968
core/vm: update comments to match eip number (#28743) 2024-01-02 11:39:53 +01:00
cygaar d2e3cb894b
core/state: logic equivalence for GetCodeHash (#28733) 2023-12-26 16:38:11 +08:00
rjl493456442 f469470aff
core/rawdb: improve state scheme checking (#28724)
This pull request improves the condition to check if path state scheme is in use. 

Originally, root node presence was used as the indicator if path scheme is used or not. However due to fact that root node will be deleted during the initial snap sync, this condition is no longer useful.

If PersistentStateID is present, it shows that we've already configured for path scheme.
2023-12-22 07:50:41 +01:00
rjl493456442 cca94792a4
core, cmd, trie: fix the condition of pathdb initialization (#28718)
Original problem was caused by #28595, where we made it so that as soon as we start to sync, the root of the disk layer is deleted. That is not wrong per se, but another part of the code uses the "presence of the root" as an init-check for the pathdb. And, since the init-check now failed, the code tried to re-initialize it which failed since a sync was already ongoing.

The total impact being: after a state-sync has begun, if the node for some reason is is shut down, it will refuse to start up again, with the error message: `Fatal: Failed to register the Ethereum service: waiting for sync.`. 

This change also modifies how `geth removedb` works, so that the user is prompted for two things: `state data` and `ancient chain`. The former includes both the chaindb aswell as any state history stored in ancients. 

---------

Co-authored-by: Martin HS <martin@swende.se>
2023-12-21 20:28:32 +01:00
lightclient 577be37e0e
cmd/devp2p: update eth/snap protocol test suites for PoS (#28340)
Here we update the eth and snap protocol test suites with a new test chain,
created by the hivechain tool. The new test chain uses proof-of-stake. As such,
tests using PoW block propagation in the eth protocol are removed. The test suite
now connects to the node under test using the engine API in order to make it
accept transactions. 

The snap protocol test suite has been rewritten to output test descriptions and
log requests more verbosely.

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-12-20 17:23:48 +01:00
wangyifan cd58897f18
core/rawdb: implement size reporting for live items in freezer_table (#28525)
This is the fix to issue #27483. A new hiddenBytes() is introduced to calculate the byte size of hidden items in the freezer table. When reporting the size of the freezer table, size of the hidden items will be subtracted from the total size.

---------

Co-authored-by: Yifan <Yifan Wang>
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2023-12-18 20:10:54 +01:00
alex edc864f9ba
all: fix typos in comments (#28682)
chore(core,eth):fix a couple of typos
2023-12-18 09:35:12 +01:00
ucwong 81fd1b3cf9
core/txpool : small cleanup refactors (#28654) 2023-12-12 17:23:36 +02:00
Felföldi Zsolt fff843cfaf
beacon/light: add CommitteeChain (#27766)
This change implements CommitteeChain which is a key component of the beacon light client. It is a passive data structure that can validate, hold and update a chain of beacon light sync committees and updates, starting from a checkpoint that proves the starting committee through a beacon block hash, header and corresponding state. Once synced to the current sync period, CommitteeChain can also validate signed beacon headers.
2023-12-08 13:38:00 +01:00
Martin HS 1048e2d6a3
cmd/evm: fix dump after state-test exec (#28650)
The dump after state-test didn't work, the problem was an error, "Already committed", which was silently ignored. 

This change re-initialises the state, so the dumping works again.
2023-12-08 11:06:01 +01:00
rjl493456442 ab0eb46a84
core/state: make stateobject.create selfcontain (#28459) 2023-11-29 16:07:51 +08:00
jwasinger 28e7371701
all: replace log15 with slog (#28187)
This PR replaces Geth's logger package (a fork of [log15](https://github.com/inconshreveable/log15)) with an implementation using slog, a logging library included as part of the Go standard library as of Go1.21.

Main changes are as follows:
* removes any log handlers that were unused in the Geth codebase.
* Json, logfmt, and terminal formatters are now slog handlers.
* Verbosity level constants are changed to match slog constant values.  Internal translation is done to make this opaque to the user and backwards compatible with existing `--verbosity` and `--vmodule` options.
* `--log.backtraceat` and `--log.debug` are removed.

The external-facing API is largely the same as the existing Geth logger.  Logger method signatures remain unchanged.

A small semantic difference is that a `Handler` can only be set once per `Logger` and not changed dynamically.  This just means that a new logger must be instantiated every time the handler of the root logger is changed.

----
For users of the `go-ethereum/log` module. If you were using this module for your own project, you will need to change the initialization. If you previously did 
```golang
log.Root().SetHandler(log.LvlFilterHandler(log.LvlInfo, log.StreamHandler(os.Stderr, log.TerminalFormat(true))))
```
You now instead need to do 
```golang
log.SetDefault(log.NewLogger(log.NewTerminalHandlerWithLevel(os.Stderr, log.LevelInfo, true)))
```
See more about reasoning here: https://github.com/ethereum/go-ethereum/issues/28558#issuecomment-1820606613
2023-11-29 08:33:50 +01:00
Péter Szilágyi 61b844f2b2
eth/gasestimator: allow slight estimation error in favor of less iterations (#28618)
* eth/gasestimator: early exit for plain transfer and error allowance

* core, eth/gasestimator: hard guess at a possible required gas

* internal/ethapi: update estimation tests with the error ratio

* eth/gasestimator: I hate you linter

* graphql: fix gas estimation test

---------

Co-authored-by: Oren <orenyomtov@users.noreply.github.com>
2023-11-28 22:31:47 +02:00
Martin Holst Swende 63979bc9cc
cmd/evm, core/state: fix post-exec dump of state (statetests, blockchaintests) (#28504)
There were several problems related to dumping state. 

- If a preimage was missing, even if we had set the `OnlyWithAddresses` to `false`, to export them anyway, the way the mapping was constructed (using `common.Address` as key) made the entries get lost anyway. Concerns both state- and blockchain tests. 
- Blockchain test execution was not configured to store preimages.

This changes makes it so that the block test executor takes a callback, just like the state test executor already does. This callback can be used to examine the post-execution state, e.g. to aid debugging of test failures.
2023-11-28 13:54:17 +01:00
jwasinger 3cfcd252db
cmd/geth: add support for --dev flag in dumpgenesis (#28463)
Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: lightclient <lightclient@protonmail.com>
2023-11-22 12:08:39 +01:00
jwasinger 104dbf7821
cmd/utils: validate pre-existing genesis in --dev mode (#28468)
geth --dev can be used with an existing data directory and genesis block. Since
dev mode only works with PoS, we need to verify that the merge has happened.

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-11-22 12:01:38 +01:00
Mario Vega 347fecd881
core/types: make 'v' optional for DynamicFeeTx and BlobTx (#28564)
This fixes an issue where transactions would not be accepted when they have only
'yParity' and not 'v'.
2023-11-22 11:00:44 +01:00
Marius van der Wijden 146e8d999c
core, trie, rpc: speed up tests (#28461)
* rpc: make subscription test faster

reduces time for TestClientSubscriptionChannelClose
from 25 sec to < 1 sec.

* trie: cache trie nodes for faster sanity check

This reduces the time spent on TestIncompleteSyncHash
from ~25s to ~16s.

* core/forkid: speed up validation test

This takes the validation test from > 5s to sub 1 sec

* core/state: improve snapshot test run
brings the time for TestSnapshotRandom from 13s down to 6s

* accounts/keystore: improve keyfile test

This removes some unnecessary waits and reduces the
runtime of TestUpdatedKeyfileContents from 5 to 3 seconds

* trie: remove resolver
* trie: only check ~5% of all trie nodes
2023-11-21 12:19:28 +01:00
rjl493456442 661bd45188
core/state/snapshot: print correct error from trie iterator (#28560) 2023-11-21 10:47:37 +08:00
jp-imx 14a1e96b68
core/txpool/legacypool: respect nolocals-setting (#28435)
This change adds a check to ensure that transactions added to the legacy pool are not treated as 'locals' if the global locals-management has been disabled. 

This change makes the pool enforce the --txpool.pricelimit setting.
2023-11-20 09:05:20 +01:00
Martin Holst Swende 2391fbc676
tests/fuzzers: move fuzzers into native packages (#28467)
This PR moves our fuzzers from tests/fuzzers into whatever their respective 'native' package is.

The historical reason why they were placed in an external location, is that when they were based on go-fuzz, they could not be "hidden" via the _test.go prefix. So in order to shove them away from the go-ethereum "production code", they were put aside.

But now we've rewritten them to be based on golang testing, and thus can be brought back. I've left (in tests/) the ones that are not production (bls128381), require non-standard imports (secp requires btcec, bn256 requires gnark/google/cloudflare deps).

This PR also adds a fuzzer for precompiled contracts, because why not.

This PR utilizes a newly rewritten replacement for go-118-fuzz-build, namely gofuzz-shim, which utilises the inputs from the fuzzing engine better.
2023-11-14 14:34:29 +01:00
Sina Mahmoodi e803ef09ad
eth/tracers/js: fix isPush for push0 (#28520)
Fixes so that `push0` opcode is correctly reported as `true` by the `IsPush` function

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-11-14 13:14:38 +01:00
Guillaume Ballet fa8d39807d
cmd, core, trie: verkle-capable `geth init` (#28270)
This change allows the creation of a genesis block for verkle testnets. This makes for a chunk of code that is easier to review and still touches many discussion points.
2023-11-14 13:09:40 +01:00
rjl493456442 326fa00759
core/rawdb: fsync the index file after each freezer write (#28483)
* core/rawdb: fsync the index and data file after each freezer write

* core/rawdb: fsync the data file in freezer after write
2023-11-10 12:56:39 +02:00
Péter Szilágyi 470dba8fc1
core/vm: set basefee to 0 internally on eth_call (#28470)
* core/vm: set basefee to 0 internally on eth_call

* core: nicer 0-basefee, make it work for blob fees too

* internal/ethapi: make tests a bit more complex

* core: fix blob fee checker

* core: make code a bit more readable

* core: fix some test error strings

* core/vm: Get rid of weird comment

* core: dict wrong typo
2023-11-08 11:22:08 +02:00
lmittmann a3be38127c
core/vm: performance tweak of `OpCode.String()` (#28453)
make `opCodeToString` a `[256]string` array

Co-authored-by: lmittmann <lmittmann@users.noreply.github.com>
2023-11-02 08:54:28 +02:00
Felix Lange bc42e88415
core: add basic chain history support in GenerateChain (#28428)
This change improves GenerateChain to support internal chain history access (ChainReader)
for the consensus engine and EVM.

GenerateChain takes a `parent` block and the number of blocks to create. With my changes,
the consensus engine and EVM can now access blocks from `parent` up to the block currently
being generated. This is required to make the BLOCKHASH instruction work, and also needed
to create real clique chains.  Clique uses chain history to figure out if the current signer is in-turn,
for example.

I've also added some more accessors to BlockGen. These are helpful when creating transactions:

- g.Signer returns a signer instance for the current block
- g.Difficulty returns the current block difficulty
- g.Gas returns the remaining gas amount

Another fix in this commit concerns the receipts returned by GenerateChain. The receipts now
have properly derived fields (BlockHash, etc.) and should generally match what would be
returned by the RPC API.
2023-10-31 12:39:25 +01:00
Jakub Freebit 447945e438
core/rawdb: add logging and fix comments around AncientRange function. (#28379)
This adds warning logs when the read does not match the expected count.
We can also remove the size limit since the function documentation explicitly states
that callers should limit the count.
2023-10-31 12:04:45 +01:00
Guillaume Ballet 4cbca5178a
core, cmd/geth: add --override.* flags to geth init (#28407)
* core, cmd/geth: add --override.* flags to geth init

* also apply overrides before genesis commit with new block

* review feedback
2023-10-27 10:15:14 +08:00
Martin Holst Swende 96b75033c0
trie: use explicit errors in stacktrie (instead of panic) (#28361)
This PR removes panics from stacktrie (mostly), and makes the Update return errors instead. While adding tests for this, I also found that one case of possible corruption was not caught, which is now fixed.
2023-10-25 14:53:50 +02:00
moomin a8617c6d4d
txpool/legacypool: remove already known error in legacypool (#28400) 2023-10-23 18:36:11 +03:00
rjl493456442 3853f50082
trie/triedb/pathdb, core/rawdb: enhance error message in freezer (#28198)
This PR adds more error message for debugging purpose.
2023-10-23 15:46:39 +02:00
hyunchel 062598bb40
core/blockchain: fix typos and grammar (#28295) 2023-10-23 15:23:41 +02:00
aaronbuchwald 6c6982163b
core/state/pruner: track number of skipped items during state pruning (#28368) 2023-10-23 13:24:32 +08:00
kevaundray a6a0ae45b6
crypto/kzg4844: use the new trusted setup file and format (#28383)
Changes the trusted_setup to the one created during the kzg-ceremony. The trusted setup file can be found in the consensus specs: https://github.com/ethereum/consensus-specs/blob/dev/presets/mainnet/trusted_setups/trusted_setup_4096.json
---------

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
2023-10-22 16:05:04 +02:00
Martin Holst Swende c1d5a012ea
core/state, tests: fix memory leak via fastcache (#28387)
This change fixes a memory leak, when running either state-tests or blockchain-tests, we allocate a `1MB` fastcache during snapshot generation. `fastcache` is a bit special, and requires a `Reset()` (it has it's own memory allocator). 

The `1MB` was hidden [here](https://github.com/ethereum/go-ethereum/blob/master/tests/state_test_util.go#L333) and [here](https://github.com/ethereum/go-ethereum/blob/master/tests/block_test_util.go#L146) respectively.
2023-10-20 13:35:49 +02:00
lightclient 20d5256e40
core/chain_makers: add SetParentBeaconRoot(..) to chain makers (#28252) 2023-10-17 14:43:46 +02:00
rjl493456442 1b1611b8d0
core, trie, eth: refactor stacktrie constructor (#28350)
This change enhances the stacktrie constructor by introducing an option struct. It also simplifies the `Hash` and `Commit` operations, getting rid of the special handling round root node.
2023-10-17 14:09:25 +02:00
Péter Szilágyi 509a64ffb9
cmd, core, ethdb: enable Pebble on 32 bits and OpenBSD too (#28335)
* cmd, core, ethdb: enable Pebble on 32 bits and OpenBSD too

* ethdb/pebble: use Pebble's internal constant calculation
2023-10-13 22:50:20 +03:00
Martin Holst Swende f62c58f8de
trie: make rhs-proof align with last key in range proofs (#28311)
During snap-sync, we request ranges of values: either a range of accounts or a range of storage values. For any large trie, e.g. the main account trie or a large storage trie, we cannot fetch everything at once.

Short version; we split it up and request in multiple stages. To do so, we use an origin field, to say "Give me all storage key/values where key > 0x20000000000000000". When the server fulfils this, the server provides the first key after origin, let's say 0x2e030000000000000 -- never providing the exact origin. However, the client-side needs to be able to verify that the 0x2e03.. indeed is the first one after 0x2000.., and therefore the attached proof concerns the origin, not the first key.

So, short-short version: the left-hand side of the proof relates to the origin, and is free-standing from the first leaf.

On the other hand, (pun intended), the right-hand side, there's no such 'gap' between "along what path does the proof walk" and the last provided leaf. The proof must prove the last element (unless there are no elements).

Therefore, we can simplify the semantics for trie.VerifyRangeProof by removing an argument. This doesn't make much difference in practice, but makes it so that we can remove some tests. The reason I am raising this is that the upcoming stacktrie-based verifier does not support such fancy features as standalone right-hand borders.
2023-10-13 16:05:29 +02:00
rjl493456442 1cb3b6aee4
eth/protocols/snap: fix snap sync failure on empty storage range (#28306)
This change addresses an issue in snap sync, specifically when the entire sync process can be halted due to an encountered empty storage range.

Currently, on the snap sync client side, the response to an empty (partial) storage range is discarded as a non-delivery. However, this response can be a valid response, when the particular range requested does not contain any slots.

For instance, consider a large contract where the entire key space is divided into 16 chunks, and there are no available slots in the last chunk [0xf] -> [end]. When the node receives a request for this particular range, the response includes:

    The proof with origin [0xf]
    A nil storage slot set

If we simply discard this response, the finalization of the last range will be skipped, halting the entire sync process indefinitely. The test case TestSyncWithUnevenStorage can reproduce the scenario described above.

In addition, this change also defines the common variables MaxAddress and MaxHash.
2023-10-13 09:08:26 +02:00
rjl493456442 eeb5dc3ccf
cmd, core: resolve scheme from a read-write database (#28313)
* cmd, core: resolve scheme from a read-write database

* cmd, core, eth: move the scheme check in the ethereum constructor

* cmd/geth: dump should in ro mode

* cmd: reverts
2023-10-11 11:27:44 +03:00
Martin Holst Swende 8976a0c97a
trie: remove owner and binary marshaling from stacktrie (#28291)
This change
  - Removes the owner-notion from a stacktrie; the owner is only ever needed for comitting to the database, but the commit-function, the `writeFn` is provided by the caller, so the caller can just set the owner into the `writeFn` instead of having it passed through the stacktrie.
  - Removes the `encoding.BinaryMarshaler`/`encoding.BinaryUnmarshaler` interface from stacktrie. We're not using it, and it is doubtful whether anyone downstream is either.
2023-10-11 06:12:45 +02:00
0xbstn fa6107c85e
core: fix typos (#28255)
fixes various typos in core
2023-10-10 10:22:03 +02:00
hyunchel 4e1e37323d
core/txpool/legacypool: fix typo (#28258) 2023-10-07 10:57:12 +08:00
Péter Szilágyi a8a9c8e4b0
core, eth, miner: start propagating and consuming blob txs (#28243)
* core, eth, miner: start propagating and consuming blob txs

* eth/protocols/eth: disable eth/67 if Cancun is enabled

* core/txpool, eth, miner: pass gas limit infos in lazy tx for mienr filtering

* core/txpool, miner: add lazy resolver for pending txs too

* core, eth: fix review noticed bugs

* eth, miner: minor polishes in the mining and announcing logs

* core/expool: unsubscribe the event scope
2023-10-04 12:36:36 +03:00
0xbstn 339a4cf056
core: fix typos (#28238) 2023-10-03 07:44:01 -04:00
Marius van der Wijden c39cbc1a78
core: implement BLOBBASEFEE opcode (0x4a) (#28098)
Implements "EIP-7516: BLOBBASEFEE opcode" for cancun, as per spec: https://eips.ethereum.org/EIPS/eip-7516
2023-10-02 05:49:29 -04:00
aaronbuchwald c5ff839fb2
core/state: small trie prefetcher nits (#28183)
Small trie prefetcher nits
2023-09-29 17:46:23 +03:00
Sina Mahmoodi 0ded110b80
core: infer blobGasUsed in chain maker (#28212)
Same way that the gasUsed in header is updated when a tx 
is added we should update blob gas used instead of requiring caller
to set it manually.
2023-09-29 10:44:28 +02:00
0xbstn 1f6e63900d
core: fix typos (#28218)
* fix(core/txpool): fix typos

* core/asm: fix typos

* core/bloombits: fix typos

* core/rawdb: fix typos
2023-09-29 10:52:22 +03:00
phenix3443 b9450bfcca
core, eth: typos and some code formatting (#28201)
* fix: typo

* feat: revert symbol name
2023-09-28 13:15:50 +03:00
rjl493456442 73f5bcb75b
core, accounts, eth, trie: handle genesis state missing (#28171)
* core, accounts, eth, trie: handle genesis state missing

* core, eth, trie: polish

* core: manage txpool subscription in mainpool

* eth/backend: fix test

* cmd, eth: fix test

* core/rawdb, trie/triedb/pathdb: address comments

* eth, trie: address comments

* eth: inline the function

* eth: use synced flag

* core/txpool: revert changes in txpool

* core, eth, trie: rename functions
2023-09-28 10:00:53 +03:00
0xbstn a081130081
core/txpool: fix typos (#28213)
fix(core/txpool): fix typos
2023-09-28 09:48:14 +08:00
bnovil 614804b33c
core/txpool: fix typos (#28208)
core/txpool:fix typos
2023-09-27 11:08:53 +08:00
hzysvilla 4de89e92e4
core/vm: minor code formatting (#28199)
Adding a space beween function opOrigin() and opcCaller() in instruciton.go.
Adding a space beween function opkeccak256()  and opAddress() in instruciton.go.
2023-09-26 11:58:01 +03:00
Marius van der Wijden 1fa3362ea7
core/forkid: add forkid test for holesky (#28193) 2023-09-25 17:02:19 +03:00
tokikuch c2cfe35f12
core/bloombits: fix deadlock when matcher session hits an error (#28184)
When MatcherSession encounters an error, it attempts to close the session.
Closing waits for all goroutines to finish, including the 'distributor'. However, the
distributor will not exit until all requests have returned.

This patch fixes the issue by delivering the (empty) result to the distributor
before calling Close().
2023-09-25 15:35:24 +02:00
Péter Szilágyi 323542af50
core, params: update Holesky testnet to relaunched spec (#28191) 2023-09-25 16:10:23 +03:00
Delweng f1b2ec0833
core/rawdb: use readonly file lock in readonly mode (#28180)
This allows using the freezer from multiple processes at once
in read-only mode.

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-09-22 12:10:50 +02:00
Péter Szilágyi 83f3fc2e80
core/state/snapshot: be very noisy if the generator hits a trie error (#28178) 2023-09-22 10:27:58 +03:00
rjl493456442 4773dcbc81
trie: remove internal nodes between shortNode and child in path mode (#28163)
* trie: remove internal nodes between shortNode and child in path mode

* trie: address comments

* core/rawdb, trie: address comments

* core/rawdb: delete unused func

* trie: change comments

* trie: add missing tests

* trie: fix lint
2023-09-22 09:31:10 +03:00
Delweng 545f4c5547
core/rawdb: no need to run truncateFile for readonly mode (#28145)
Avoid truncating files, if ancients are opened in readonly mode. With this change, we return error instead of trying (and failing)  to repair
2023-09-21 04:05:55 -04:00
Delweng ef76afad35
core/rawdb: fix typo in comment (#28140) 2023-09-19 13:43:37 +02:00
Felix Lange e9f78db79d
cmd/evm: fix some issues with the evm run command (#28109)
* cmd/evm: improve flags handling

This fixes some issues with flags in cmd/evm. The supported flags did not
actually show up in help output because they weren't categorized. I'm also
adding the VM-related flags to the run command here so they can be given
after the subcommand name. So it can be run like this now:

   ./evm run --code 6001 --debug

* cmd/evm: enable all forks by default in run command

The default genesis was just empty with no forks at all, which is annoying because
contracts will be relying on opcodes introduced in a fork. So this changes the default to
have all forks enabled.

* core/asm: fix some issues in the assembler

This fixes minor bugs in the old assembler:

- It is now possible to have comments on the same line as an instruction.
- Errors for invalid numbers in the jump instruction are reported better
- Line numbers in errors were off by one
2023-09-19 13:41:16 +02:00
rjl493456442 c53b0fef2a
core, eth/downloader: fix genesis state missing due to state sync (#28124)
* core: fix chain repair corner case in path-based scheme

* eth/downloader: disable trie database whenever state sync is launched
2023-09-17 17:35:09 +03:00
Darioush Jalali 48fdb79de5
core/state: check err for iter.Error in fastDeleteStorage (#28122)
core/state: check err for iter.Error
2023-09-15 14:09:07 +08:00
Felix Lange 909dd4a109
rlp/rlpgen: remove build tag (#28106)
* rlp/rlpgen: remove build tag

This tag was supposed to prevent unstable output when types reference each other. Imagine
there are two struct types A and B, where a reference to type B is in A. If I run rlpgen
on type B first, and then on type A, the generator will see the B.EncodeRLP method and
call it. However, if I run rlpgen on type A first, it will inline the encoding of B.

The solution I chose for the initial release of rlpgen was to just ignore methods
generated by rlpgen using a build tag. But there is a problem with this: if any code in
the package calls EncodeRLP explicitly, the package can't be loaded without errors anymore
in rlpgen, because the loader ignores it. Would be nice if there was a way to just make it
ignore invalid functions during type checking (they're not necessary for rlpgen), but
golang.org/x/tools/go/packages does not provide a way of ignoring them.

Luckily, the types we use rlpgen with do not reference each other right now, so we can
just remove the build tags for now.
2023-09-14 12:28:40 +02:00
Martin Holst Swende 8b6cf128af
metrics: refactor metrics (#28035)
This change includes a lot of things, listed below. 

### Split up interfaces, write vs read

The interfaces have been split up into one write-interface and one read-interface, with `Snapshot` being the gateway from write to read. This simplifies the semantics _a lot_. 

Example of splitting up an interface into one readonly 'snapshot' part, and one updatable writeonly part: 

```golang
type MeterSnapshot interface {
	Count() int64
	Rate1() float64
	Rate5() float64
	Rate15() float64
	RateMean() float64
}

// Meters count events to produce exponentially-weighted moving average rates
// at one-, five-, and fifteen-minutes and a mean rate.
type Meter interface {
	Mark(int64)
	Snapshot() MeterSnapshot
	Stop()
}
```

### A note about concurrency

This PR makes the concurrency model clearer. We have actual meters and snapshot of meters. The `meter` is the thing which can be accessed from the registry, and updates can be made to it. 

- For all `meters`, (`Gauge`, `Timer` etc), it is assumed that they are accessed by different threads, making updates. Therefore, all `meters` update-methods (`Inc`, `Add`, `Update`, `Clear` etc) need to be concurrency-safe. 
- All `meters` have a `Snapshot()` method. This method is _usually_ called from one thread, a backend-exporter. But it's fully possible to have several exporters simultaneously: therefore this method should also be concurrency-safe. 

TLDR: `meter`s are accessible via registry, all their methods must be concurrency-safe. 

For all `Snapshot`s, it is assumed that an individual exporter-thread has obtained a `meter` from the registry, and called the `Snapshot` method to obtain a readonly snapshot. This snapshot is _not_ guaranteed to be concurrency-safe. There's no need for a snapshot to be concurrency-safe, since exporters should not share snapshots. 

Note, though: that by happenstance a lot of the snapshots _are_ concurrency-safe, being unmutable minimal representations of a value. Only the more complex ones are _not_ threadsafe, those that lazily calculate things like `Variance()`, `Mean()`.

Example of how a background exporter typically works, obtaining the snapshot and sequentially accessing the non-threadsafe methods in it: 
```golang
		ms := metric.Snapshot()
                ...
		fields := map[string]interface{}{
			"count":    ms.Count(),
			"max":      ms.Max(),
			"mean":     ms.Mean(),
			"min":      ms.Min(),
			"stddev":   ms.StdDev(),
			"variance": ms.Variance(),
```

TLDR: `snapshots` are not guaranteed to be concurrency-safe (but often are).

### Sample changes

I also changed the `Sample` type: previously, it iterated the samples fully every time `Mean()`,`Sum()`, `Min()` or `Max()` was invoked. Since we now have readonly base data, we can just iterate it once, in the constructor, and set all four values at once. 

The same thing has been done for runtimehistogram. 

### ResettingTimer API

Back when ResettingTImer was implemented, as part of https://github.com/ethereum/go-ethereum/pull/15910, Anton implemented a `Percentiles` on the new type. However, the method did not conform to the other existing types which also had a `Percentiles`. 

1. The existing ones, on input, took `0.5` to mean `50%`. Anton used `50` to mean `50%`. 
2. The existing ones returned `float64` outputs, thus interpolating between values. A value-set of `0, 10`, at `50%` would return `5`, whereas Anton's would return either `0` or `10`. 

This PR removes the 'new' version, and uses only the 'legacy' percentiles, also for the ResettingTimer type. 

The resetting timer snapshot was also defined so that it would expose the internal values. This has been removed, and getters for `Max, Min, Mean` have been added instead. 

### Unexport types

A lot of types were exported, but do not need to be. This PR unexports quite a lot of them.
2023-09-13 13:13:47 -04:00
Péter Szilágyi 8d38b1fe62
core/rawdb: skip pathdb state inspection in hashdb mode (#28108) 2023-09-13 15:13:10 +03:00
lightclient 1efd12f695
core: fix calculation of blob gasprice in tx receipt (#28082)
This fixes the derived value BlobGasPrice on the receipt of EIP-4844 transactions, which was previously erroneously set to the price cap.
2023-09-11 02:14:20 -04:00
rjl493456442 a7842c9cae
core, trie: cleanup trie database (#28062) 2023-09-07 21:17:14 +08:00
Delweng 2e02c1ffd9
core/rawdb: don't warn for missing "unclean shutdown markers" (#28014)
This changes removes the warn-printout about not finding unclean shutdown markers, which always happens on fresh installs / wiped databases.
2023-09-06 06:29:51 -04:00
lightclient 25733a4aad
params: update 4844 parameters (#28026)
On ACD 163, it was agreed to bump the target and max blob values from `2/4` to `3/6` for future devnets until we could decide on final mainnet number. This change contains said update, making master pass all the hive tests. The final decision for mainnet cancun is still to be made.
---------

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-09-05 04:49:51 -04:00
lightclient eff7c3bda0
core/forkid: skip genesis forks by time (#28034)
* core/forkid: skip genesis forks by time

* core/forkid: add comment about skipping non-zero fork times

* core/forkid: skip all time based forks in genesis using loop

* core/forkid: simplify logic for dropping time-based forks
2023-09-04 09:32:14 -04:00
rjl493456442 0acc0a1f86
core/state: simplify storage trie update and commit (#28030)
This change improves function description and simplifies logic in statedb update and commit operations.
2023-08-31 14:33:18 -04:00
Jorge 53f3c2ae65
metrics, cmd/geth: informational metrics (prometheus, influxdb, opentsb) (#24877)
This chang creates a GaugeInfo metrics type for registering informational (textual) metrics, e.g. geth version number. It also improves the testing for backend-exporters, and uses a shared subpackage in 'internal' to provide sample datasets and ordered registry. 

Implements #21783

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-08-31 13:37:17 -04:00
lightclient 41ee96fdfe
core/txpool/blobpool: fix rlp decoding flaw during offload (#28027) 2023-08-30 03:28:03 -04:00
Martin Holst Swende d4e345c7d4
core/state: fix missing import (#28010) 2023-08-26 04:43:36 -04:00
rjl493456442 3ff6b3c31e
core/state: implement fast storage deletion (#27955)
This changes implements faster post-selfdestruct iteration of storage slots for deletion, by using snapshot-storage+stacktrie to recover the trienodes to be deleted. This mechanism is only implemented for path-based schema. 

For hash-based schema, the entire post-selfdestruct storage iteration is skipped, with this change, since hash-based does not actually perform deletion anyway. 

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-08-26 04:13:22 -04:00
Martin Holst Swende 6aa88ccdd2
beacon/engine, eth/catalyst, miner: EIP-4788 CL/EL protocol updates (#27872)
This PR makes EIP-4788 work in the engine API and miner. It also fixes some bugs related to 
EIP-4844 block processing and mining. Changes in detail:

- Header.BeaconRoot has been renamed to ParentBeaconRoot.
- The engine API now implements forkchoiceUpdatedV3
- newPayloadV3 method has been updated with the parentBeaconBlockRoot parameter
- beacon root is now applied to new blocks in miner
- For EIP-4844, block creation now updates the blobGasUsed field of the header
2023-08-26 04:52:12 +02:00
Felix Lange 9bbb9df185
core/types: transaction and receipt encoding/decoding optimizations (#27976)
Just some minor optimizations I figured out a while ago. By using ReadBytes instead of
Bytes on the rlp stream, we can save the allocation of a temporary buffer for the typed tx
payload.

If kind == rlp.Byte, the size reported by Stream.Kind will be zero, but we need a buffer
of size 1 for ReadBytes. Since typed txs always have to be longer than 1 byte, we can just
return an error for kind == rlp.Byte.

There is a also a small change for Log: since the first three fields of Log are the ones that 
should appear in the canon encoding, we can simply ignore the remaining fields via 
struct tag. Doing this removes an indirection through the rlpLog type.

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-08-25 20:03:41 +02:00
Péter Szilágyi 6b98d18789
cmd, core, params: add support for the Holesky testnet (#28007)
* cmd, core, params: add support for the Holesky testnet

* cmd/devp2p: add support for holesky for the dns crawler
2023-08-25 18:11:40 +03:00
Felix Lange 6dc9cdf15b
core: support null balance in genesis (#28005) 2023-08-25 15:59:40 +02:00
aaronbuchwald 56d2366699
core/state/snapshot: replace diffToDisk ideal batch size with 64MB (#27977) 2023-08-25 15:48:10 +03:00
Guido Vranken 0ba2d3cfa4
core/vm/runtime: Add Random field to config (#28001) 2023-08-25 04:05:33 -04:00
Felix Lange 9b46986edc
all: use rlp.DecodeBytes instead of rlp.Decode where possible (#27994) 2023-08-24 11:47:42 +03:00
lightclient feb8f416ac
miner: add to build block with EIP-4844 blobs (#27875)
---------

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-08-23 23:16:14 +02:00
Felix Lange d1f6735171
core/rawdb: fix 32bit build (#27995) 2023-08-23 23:14:36 +02:00
Martin Holst Swende eb6cbe37e1
core/state: remove public method ForEachStorage (#27986)
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-08-23 21:53:31 +02:00
Marius van der Wijden 2f4dbb4f90
core/rawdb: allocate database keys with explicit size to avoid slice growth (#27772) 2023-08-23 21:41:56 +02:00
Martin Holst Swende f0f8703bf2
core/rawdb, ethdb/pebble: avoid fsync db in tests (#27836)
Adds an option to disable fsync for database operations.
This is to make tests faster.
2023-08-23 20:43:55 +02:00
Péter Szilágyi 00fead91c4
cmd/utils: fix a startup issue on deleted chaindata but dangling ancients (#27989) 2023-08-23 16:42:37 +03:00
Péter Szilágyi 0c6bbeb423
core, eth, trie: expose more detailed dirty ram tracking for diff layers (#27971) 2023-08-23 14:08:39 +03:00
Felix Lange 4c7053baf1
core: add BeaconRoot to header in GenerateChain (#27974)
Fixes an error in tests of internal/ethapi.
2023-08-22 18:29:17 +02:00
Martin Holst Swende b8d38e76ef
core, params, beacon/engine: implement EIP 4788 BeaconRoot (#27849)
This change implements "EIP 4788 : Beacon block root in the EVM". It implements version-2 of EPI-4788, main difference being that the contract is an actual contract rather than a precompile, as in #27289.
2023-08-22 08:33:39 -04:00
Delweng 7c0d90c8c9
core: ensure txindex will be triggered at least once (#27847)
Currently, we trigger the logic to (un)index transactions when the node receives a new
block. However, in some cases the node may not receive new blocks (eg, when the Geth node
is configured without peer discovery, or when it acts as an RPC node for historical-only
data).

In these situations, the Geth node user may not have previously configured txlookuplimit
(i.e. the default of around one year), but later realizes they need to index all
historical blocks. However, adding txlookuplimit=0 and restarting geth has no effect. This
change makes it check for required indexing work once, on startup, to fix the issue.

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-08-22 02:58:20 +02:00
Delweng 9f4a528793
internal/ethapi: add testcases for blobTx (#27818)
Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-08-22 02:53:29 +02:00
Martin Holst Swende 1aa5520d75
core/txpool/legacypool: protect cache with mutex (#27898)
This change fixes the a potential race by using mutexes when the m.cache is read or modified.
2023-08-17 05:22:18 -04:00
lightclient 32fde3f838
core/forkid: correctly compute forkid when timestamp fork is activated in genesis (#27895)
This changes the forkID calculation to ignore time-based forks that occurred before the
genesis block. It's supposed to be done this way because the spec says:

> If a chain is configured to start with a non-Frontier ruleset already in its genesis, that is NOT considered a fork.
2023-08-16 23:31:02 +02:00
lightclient a3e35414b7
core/types: fix unmarshalling of BlobTx values (#27939)
FromBig returns true *when overflow occurs*
2023-08-16 23:22:15 +02:00
Felix Lange 2a6beb6a39
core/types: support for optional blob sidecar in BlobTx (#27841)
This PR removes the newly added txpool.Transaction wrapper type, and instead adds a way
of keeping the blob sidecar within types.Transaction. It's better this way because most
code in go-ethereum does not care about blob transactions, and probably never will. This
will start mattering especially on the client side of RPC, where all APIs are based on
types.Transaction. Users need to be able to use the same signing flows they already
have.

However, since blobs are only allowed in some places but not others, we will now need to
add checks to avoid creating invalid blocks. I'm still trying to figure out the best place
to do some of these. The way I have it currently is as follows:

- In block validation (import), txs are verified not to have a blob sidecar.
- In miner, we strip off the sidecar when committing the transaction into the block.
- In TxPool validation, txs must have a sidecar to be added into the blobpool.
  - Note there is a special case here: when transactions are re-added because of a chain
    reorg, we cannot use the transactions gathered from the old chain blocks as-is,
    because they will be missing their blobs. This was previously handled by storing the
    blobs into the 'blobpool limbo'. The code has now changed to store the full
    transaction in the limbo instead, but it might be confusing for code readers why we're
    not simply adding the types.Transaction we already have.

Code changes summary:

- txpool.Transaction removed and all uses replaced by types.Transaction again
- blobpool now stores types.Transaction instead of defining its own blobTx format for storage
- the blobpool limbo now stores types.Transaction instead of storing only the blobs
- checks to validate the presence/absence of the blob sidecar added in certain critical places
2023-08-14 10:13:34 +02:00
Péter Szilágyi be65b47645
all: update golang/x/ext and fix slice sorting fallout (#27909)
The Go authors updated golang/x/ext to change the function signature of the slices sort method. 
It's an entire shitshow now because x/ext is not tagged, so everyone's codebase just 
picked a new version that some other dep depends on, causing our code to fail building.

This PR updates the dep on our code too and does all the refactorings to follow upstream...
2023-08-12 00:04:12 +02:00
DongXi Huang 80b76a9527
core/types: remove duplication in eip2930 signer (#27860)
Remove duplication in signer
---------

Co-authored-by: GDdark <huangdongxi@minijoy.work>
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-08-11 09:50:57 -04:00
imulmat4 6ddb92cac3
core/txpool/blobpool: fix metrics name for prometheus export (#27901) 2023-08-11 10:12:26 +03:00
rjl493456442 503f1f7ada
all: activate pbss as experimental feature (#26274)
* all: activate pbss

* core/rawdb: fix compilation error

* cma, core, eth, les, trie: address comments

* cmd, core, eth, trie: polish code

* core, cmd, eth: address comments

* cmd, core, eth, les, light, tests: address comment

* cmd/utils: shorten log message

* trie/triedb/pathdb: limit node buffer size to 1gb

* cmd/utils: fix opening non-existing db

* cmd/utils: rename flag name

* cmd, core: group chain history flags and fix tests

* core, eth, trie: fix memory leak in snapshot generation

* cmd, eth, internal: deprecate flags

* all: enable state tests for pathdb, fixes

* cmd, core: polish code

* trie/triedb/pathdb: limit the node buffer size to 256mb

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2023-08-10 22:21:36 +03:00
Martin Holst Swende 8d1db1601d
eth/filters, core/rawdb: remove unused param, refactor filtering-loop (#27891)
This change removes a chainconfig parameter passed into rawdb.ReadLogs, which is not used nor needed.
It also modifies the filter loop slightly, avoiding a labeled break and instead using a method.

This change does not modify any behaviour.
2023-08-10 06:49:05 -04:00
Guillaume Ballet 67979022aa
core/state: move UpdateContractCode before the trie hash is computed (#27853)
Context: The UpdateContractCode method was introduced for the state storage commitment
schemes that include the whole code for their commitment computation. It must therefore be called
before the root hash is computed at the end of IntermediateRoot.

This should have no impact on the MPT since, in this context, the method is a no-op.
2023-08-09 18:02:45 +02:00
Martin Holst Swende e13fa32cea
core/vm: update 4844 - point evaluation precompile address (#27874) 2023-08-08 15:54:19 +03:00
Felix Lange bb148dd342
core/types: support yParity field in JSON transactions (#27744)
This adds support for the "yParity" field in transaction objects returned by RPC
APIs. We somehow forgot to add this field even though it has been in the spec for
a long time.
2023-08-05 00:54:55 +02:00
Darioush Jalali 57cdbaef30
all: remove trailing whitespace (#27741) 2023-08-05 00:24:32 +02:00
Felix Lange df544350bc
core/types: fix immutability guarantees in Block (#27844)
This change rearranges the accessor methods in block.go and fixes some minor issues with
the copy-on-write logic of block data. Fixed issues:

- Block.WithWithdrawals did not create a shallow copy of the block.

- Block.WithBody copied the header unnecessarily, and did not preserve withdrawals.

However, the bugs did not affect any code in go-ethereum because blocks are *always*
created using NewBlockWithHeader().WithBody().WithWithdrawals()
2023-08-04 14:16:23 +02:00
rjl493456442 7de748d3f6
all: implement path-based state scheme (#25963)
* all: implement path-based state scheme

* all: edits from review

* core/rawdb, trie/triedb/pathdb: review changes

* core, light, trie, eth, tests: reimplement pbss history

* core, trie/triedb/pathdb: track block number in state history

* trie/triedb/pathdb: add history documentation

* core, trie/triedb/pathdb: address comments from Peter's review

Important changes to list:

- Cache trie nodes by path in clean cache
- Remove root->id mappings when history is truncated

* trie/triedb/pathdb: fallback to disk if unexpect node in clean cache

* core/rawdb: fix tests

* trie/triedb/pathdb: rename metrics, change clean cache key

* trie/triedb: manage the clean cache inside of disk layer

* trie/triedb/pathdb: move journal function

* trie/triedb/path: fix tests

* trie/triedb/pathdb: fix journal

* trie/triedb/pathdb: fix history

* trie/triedb/pathdb: try to fix tests on windows

* core, trie: address comments

* trie/triedb/pathdb: fix test issues

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-08-01 15:17:32 +03:00
Delweng 7c95ebd63d
consensus/misc: move eip1559 into a package (#27828)
* consensus/misc: move eip1559 as a sub directory

Signed-off-by: jsvisa <delweng@gmail.com>

* consensus/misc: package name

Signed-off-by: jsvisa <delweng@gmail.com>

* all: eip1559

Signed-off-by: jsvisa <delweng@gmail.com>

---------

Signed-off-by: jsvisa <delweng@gmail.com>
2023-08-01 12:58:45 +03:00
rjl493456442 2fd77a6a7e
core/txpool/blobpool: remove error log when finalized block is nil (#27822)
* core/txpool/blobpool: remove error log when finalized block is nil

* core/txpool/blobpool: take peter's suggestion
2023-08-01 10:09:20 +03:00
Delweng 3ca92f70e5
all: replace data gas to blob gas in comments (#27825)
* eth: excessDataGas -> excessBlobGas

* consensus: data gas -> blob gas

* core: data gas -> blob gas

* params: data gas -> blob gas
2023-08-01 10:07:25 +03:00