Commit Graph

2299 Commits

Author SHA1 Message Date
rjl493456442 b946b7a13b
core, miner: drop transactions from the same sender when error occurs (#27038)
This PR unifies the error handling in miner. 

Whenever an error occur while applying a transaction, the transaction should be regarded as invalid and all following transactions from the same sender not executable because of the nonce restriction. The only exception is the `nonceTooLow` error which is handled separately.
2023-04-05 07:09:25 -04:00
Marius van der Wijden 230df98e4d
core/txpool: disallow future churn by remote txs (#26907)
Prior to this change, it was possible that transactions are erroneously deemed as 'future' although they are in fact 'pending', causing them to be dropped due to 'future' not being allowed to replace 'pending'. 

This change fixes that, by doing a more in-depth inspection of the queue.
2023-04-05 04:59:32 -04:00
Martin Holst Swende ab1a404b01
all: remove debug-field from vm config (#27048)
This PR removes the Debug field from vmconfig, making it so that if a tracer is set, debug=true is implied.

---------

Co-authored-by: 0xTylerHolmes <tyler@ethereum.org>
Co-authored-by: Sina Mahmoodi <1591639+s1na@users.noreply.github.com>
2023-04-04 09:50:13 -04:00
Péter Szilágyi 91faf2c559
consensus, core/typer: add 4844 excessDataGas to header, tie it to Cancun (#27046) 2023-04-04 10:02:50 +03:00
joohhnnn 9b1a82c600
core/vm: clarify comment (#27045) 2023-04-04 02:59:40 -04:00
Martin Holst Swende beda6c41ad
core/txpool: move some validation to outside of mutex (#27006)
Currently, most of transaction validation while holding the txpool mutex: one exception being an early-on signature check. 

This PR changes that, so that we do all non-stateful checks before we entering the mutex area. This means they can be performed in parallel, and to enable that, certain fields have been made atomic bools and uint64.
2023-04-03 07:16:57 -04:00
s7v7nislands 949cee2fe3
core: use atomic type (#27011) 2023-03-30 06:53:32 -04:00
David Murdoch 7ca4f60a1a
docs: update outdated DeriveSha docs comment (#26968) 2023-03-28 08:59:37 -04:00
s7v7nislands fb8a3aaf1e
core/state: use atomic.Bool (#26992) 2023-03-28 03:06:50 -04:00
s7v7nislands 79532a25b1
core/bloombits: use atomic type (#26993) 2023-03-28 03:05:29 -04:00
s7v7nislands 881fed032c
core/vm: use atomic.Bool (#26951)
Make use of new atomic types
---------

Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-03-27 09:40:20 -04:00
Guillaume Ballet 41f89ca944
core/state, trie: remove Try prefix in Trie accessors (#26975)
This change renames StateTrie methods to remove the Try* prefix. 

We added the Trie methods with prefix 'Try' a long time ago, working
around the problem that most existing methods of Trie did not return the
database error. This weird naming convention has persisted until now.

Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2023-03-27 10:48:46 +02:00
Guillaume Ballet 8990c92aea
core/state: add account address to Trie slot accessors (#26934)
This changes the Trie interface to add the plain account address as a
parameter to all storage-related methods.

After the introduction of the TryAccount* functions, TryGet, TryUpdate and
TryDelete are now only meant to read an account's storage. In their current
form, they assume that an account storage is stored in a separate trie, and
that the hashing of the slot is independent of its account's address.

The proposed structure for a stateless storage breaks these two
assumptions: the hashing of a slot key requires the address and all slots
and accounts are stored in a single trie.

This PR therefore adds an address parameter to the interface. It is ignored
in the MPT version, so this change has no functional impact, however it
will reduce the diff size when merging verkle trees.
2023-03-23 11:52:22 +01:00
Delweng 7f3fc15a8b
core/rawdb: update freezertable read meter (#26946)
The meter for "for measuring the effective amount of data read" within the freezertable was never updated. This change remedies that. 
---------

Signed-off-by: jsvisa <delweng@gmail.com>
2023-03-23 03:34:40 -04:00
Martin Holst Swende b3f43c89b3
core/vm: expose jumptable constructors (#26880)
When interacting with geth as a library to e.g. produce state tests, it is desirable to obtain the consensus-correct jumptable definition for a given fork. This changes adds accessors so the instructionset can be obtained and characteristics about opcodes can be inspected.
2023-03-21 07:14:47 -04:00
s7v7nislands 905a723fae
core/rawdb: use atomic int added in go1.19 (#26935) 2023-03-21 07:10:23 -04:00
Marius van der Wijden e6b6a8b738
core/txpool: allow future local transactions (#26930)
Local transactions should not be subject to the "future shouldn't churn pending txs" rule
2023-03-20 09:15:18 -04:00
Marius van der Wijden 5d23d21fff
params: schedule shanghai fork on mainnet (#26908)
Schedules the shanghai hardfork on timestamp 1681338455 as discussed on ACDE 157: https://github.com/ethereum/execution-specs/pull/727
2023-03-20 07:38:34 -04:00
s7v7nislands 80ff0b4e6a
core/txpool: use atomic int added in go1.19 (#26913)
Makes use of atomic.Uint64 instead of atomic by pointer
2023-03-20 04:12:24 -04:00
rjl493456442 48d1bf0678
consensus: improve consensus engine definition (#26871)
Makes clear the distinction between Finalize and FinalizedAndAssemble:

- In Finalize function, a series of state operations are applied according to consensus rules. The statedb is mutated and the root hash can be checked and compared afterwards.

This function should be used in block processing(receive afrom network and apply it locally) but not block generation.

- In FinalizeAndAssemble function, after applying state mutations, the block is also to be assembled with the latest
  state root computed, updating the header. 

 This function should be used in block generation only.
2023-03-16 15:34:25 -04:00
rjl493456442 bba2a1bac5
core: show db error-info in case of mismatched hash root (#26870)
When a database failure occurs, bubble it up a into statedb, and report it in suitable places, such as during a 'bad block' report.
2023-03-16 03:12:34 -04:00
rjl493456442 c8a6b7100c
core/state, trie: port changes from PBSS (#26763) 2023-03-14 04:50:53 -04:00
ucwong d1c5f918a3
core/txpool: use priceList.Put instead of heap.Push (#26863)
Minor refactor to use the 'intended' accessor
2023-03-13 04:45:25 -04:00
s7v7nislands a20e38720c
core: minor code refactor (#26852)
* core: refactor code

* core: drop it from this anonymous goroutine func
2023-03-13 04:02:50 -04:00
xiyang ca61048178
code/vm: fix comment typo (#26865)
it should be constantinople rather than contantinople
2023-03-13 09:30:32 +02:00
Marius van der Wijden 6cf2e921a7
core/txpool: implement additional DoS defenses (#26648)
This adds two new rules to the transaction pool:

- A future transaction can not evict a pending transaction.
- A transaction can not overspend available funds of a sender.

---

Co-authored-by: dwn1998 <42262393+dwn1998@users.noreply.github.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-03-10 18:30:26 +01:00
Felix Lange 564db9a95f
core: add Timestamp method in BlockGen (#26844)
Since forks are now scheduled by block time, it can be necessary
to check the timestamp of a block while generating transactions.
2023-03-10 14:03:27 +01:00
Roberto Bayardo 67ac5f0ae7
core, core/types: plain Message struct (#25977)
Here, the core.Message interface turns into a plain struct and
types.Message gets removed.

This is a breaking change to packages core and core/types. While we do
not promise API stability for package core, we do for core/types. An
exception can be made for types.Message, since it doesn't have any
purpose apart from invoking the state transition in package core.
types.Message was also marked deprecated by the same commit it
got added in, 4dca5d4db7 (November 2016).

The core.Message interface was added in December 2014, in commit
db494170dc, for the purpose of 'testing' state transitions. It's the
same change that made transaction struct fields private. Before that,
the state transition used *types.Transaction directly.

Over time, multiple implementations of the interface accrued across
different packages, since constructing a Message is required whenever
one wants to invoke the state transition. These implementations all
looked very similar, a struct with private fields exposing the fields
as accessor methods.

By changing Message into a struct with public fields we can remove all
these useless interface implementations. It will also hopefully
simplify future changes to the type with less updates to apply across
all of go-ethereum when a field is added to Message.

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-03-09 14:19:12 +01:00
Felix Lange 5395362e0f
core/forkid: fix issue in validation test (#26544)
This changes the test to match the comment description. Using timestampedConfig in this test case is incorrect, the comment says 'local is at Gray Glacier' and isn't aware of more forks.
2023-03-09 06:37:44 -05:00
xiyang 1bf1168432
core/vm: fix typo in comment (#26838)
fixes eip 220 ->  2200
2023-03-09 04:39:17 -05:00
Martin Holst Swende b80f05bde2
core/vm: use golang native big.Int (#26834)
reverts #26021, to use the upstream bigint instead.
2023-03-08 13:12:53 -05:00
rjl493456442 f7661a662a
core/rawdb: find smallest block stored in key-value store when chain gapped (#26719)
This change prints out more information about the problem, in the case where geth detects a gap between leveldb and ancients, so we can determine more exactly where the gap is (what the first missing is). Also prints out more metadata. 

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-03-08 02:39:13 -05:00
rjl493456442 77e33e5a49
core, miner: revert block gas counter in case of invalid transaction (#26799)
This change fixes a flaw where, in certain scenarios, the block sealer did not accurately reset the remaining gas after failing to include an invalid transaction. Fixes #26791
2023-03-07 05:23:52 -05:00
Marius van der Wijden 5bc2ef984f
core, eth/catalyst: fix race conditions in tests (#26790)
Fixes a race in TestNewPayloadOnInvalidTerminalBlock where setting the TTD raced with
the miner. Solution: set the TTD on the blockchain config not the genesis config.

Also fixes a race in CopyHeader which resulted in race reports all over the place.
2023-03-06 23:32:27 +01:00
Felix Lange 87186148e0
core/types: add EffectiveGasPrice in Receipt (#26713)
This change adds a struct field EffectiveGasPrice in types.Receipt. The field is present
in RPC responses, but not in the Go struct, and thus can't easily be accessed via ethclient.

Co-authored-by: PulsarAI <dev@pulsar-systems.fi>
2023-03-06 17:19:41 +01:00
Marius van der Wijden d865a5d6ae
core, params: schedule Shanghai on goerli (#26795)
* core: params: schedule Shanghai on goerli

* core/forkid: fix comment
2023-03-06 09:26:43 +02:00
Péter Szilágyi 010189538e
core: fix a merge fault (#26802) 2023-03-03 12:11:39 +02:00
Péter Szilágyi 19f74fa3c0
core/rawdb, ethdb/pebble: disable pebble on openbsd (#26801) 2023-03-03 12:05:00 +02:00
Péter Szilágyi cd31f2dee2
all: change chain head markers from block to header (#26777) 2023-03-02 08:29:15 +02:00
Dan Cline 2ea48f8a22
core: improve withdrawal index assignment in GenerateChain (#26756)
This fixes an issue where the withdrawal index was not calculated correctly
for multiple withdrawals in a single block.

Co-authored-by: Gary Rong <garyrong0905@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-02-28 11:46:32 +01:00
Martin Holst Swende 09a9ccdbce
core/rawdb, node: use standalone flock dependency (#26633) 2023-02-23 09:11:50 +02:00
rjl493456442 fe01a2f63b
all: use unified emptyRootHash and emptyCodeHash (#26718)
The EmptyRootHash and EmptyCodeHash are defined everywhere in the codebase, this PR replaces all of them with unified one defined in core/types package, and also defines constants for TxRoot, WithdrawalsRoot and UncleRoot
2023-02-21 06:12:27 -05:00
Péter Szilágyi 2f20fd31ee
core/rawdb: expose chain freezer constructor without internals (#26748) 2023-02-21 13:10:01 +02:00
Martin Holst Swende 6d2d126100
core: fix accessor mismatch for genesis state (#26747) 2023-02-21 12:18:33 +02:00
Péter Szilágyi 90d25514af
core, eth: merge snap-sync chain download progress logs (#26676) 2023-02-21 12:17:34 +02:00
rjl493456442 13ef21d467
Revert "core/trie: remove trie tracer (#26665)" (#26732)
This reverts commit 7c749c947a.
2023-02-20 09:54:52 -05:00
Martin Holst Swende 7c749c947a
core/trie: remove trie tracer (#26665)
This PR contains a small portion of the full pbss PR, namely

    Remove the tracer from trie (and comitter), and instead using an accessList.
    Related changes to the Nodeset.


---------

Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2023-02-19 14:10:19 -05:00
Martin Holst Swende 4d3525610e
all: remove deprecated uses of math.rand (#26710)
This PR is a (superior) alternative to https://github.com/ethereum/go-ethereum/pull/26708, it handles deprecation, primarily two specific cases. 

`rand.Seed` is typically used in two ways
- `rand.Seed(time.Now().UnixNano())` -- we seed it, just to be sure to get some random, and not always get the same thing on every run. This is not needed, with global seeding, so those are just removed. 
- `rand.Seed(1)` this is typically done to ensure we have a stable test. If we rely on this, we need to fix up the tests to use a deterministic prng-source. A few occurrences like this has been replaced with a proper custom source. 

`rand.Read` has been replaced by `crypto/rand`.`Read` in this PR.
2023-02-16 14:36:58 -05:00
Martin Holst Swende 645e3e86c4
core, eth/downloader: make body validation more strict (#26704) 2023-02-16 12:10:16 +02:00
rjl493456442 08bf8a60c3
core: check genesis state presence by disk read (#26703) 2023-02-16 12:09:41 +02:00
Marius van der Wijden 7fb42e6db2
eth/downloader: handle missing withdrawals if empty list is expected (#26675)
This PR relaxes the block body ingress handling a bit: if block body withdrawals are missing (but expected to be empty), the body withdrawals are set to 'empty list' before being passed to upper layers. 

This fixes an issue where a block passed from EthereumJS to geth was deemed invalid.
2023-02-15 07:23:07 -05:00
Sina Mahmoodi 2def62b99b
eth/filters: avoid block body retrieval when no matching logs (#25199)
Logs stored on disk have minimal information. Contextual information such as block
number, index of log in block, index of transaction in block are filled in upon request.
We can fill in all these fields only having the block header and list of receipts.
But determining the transaction hash of a log requires the block body.

The goal of this PR is postponing this retrieval until we are sure we the transaction hash.
It happens often that the header bloom filter signals there might be matches in a block,
but after actually checking them reveals the logs do not match. We want to avoid fetching
the body in this case.

Note that this changes the semantics of Backend.GetLogs. Downstream callers of
GetLogs now assume log context fields have not been derived, and need to call
DeriveFields on the logs if necessary.
2023-02-13 10:59:27 +01:00
Marius van der Wijden 241cf62b5c
params: schedule shanghai fork on sepolia (#26662)
* params: schedule shanghai fork on sepolia

* params: u64 -> newUint64
2023-02-10 13:34:45 +02:00
Chris Ziogas b0cd8c4a5c
core/vm: set tracer-observable `value` of a delegatecall to match parent `value` (#26632)
This is a breaking change in the tracing hooks API as well as semantics of the callTracer:

- CaptureEnter hook provided a nil value argument in case of DELEGATECALL. However to stay consistent with how delegate calls behave in EVM this hook is changed to pass in the value of the parent call.
- callTracer will return parent call's value for DELEGATECALL frames.

---------

Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
2023-02-10 10:26:43 +01:00
Martin Holst Swende 22c3ad1d12
core/state, trie: remove unused error-return from trie Commit operation (#26641) 2023-02-09 08:56:59 -05:00
Péter Szilágyi bf1798e04e
common/prque: generic priority queue (#26290)
* common, core, eth, les, trie: make prque generic

* les/vflux/server: fixed issues in priorityPool

* common, core, eth, les, trie: make priority also generic in prque

* les/flowcontrol: add test case for priority accumulator overflow

* les/flowcontrol: avoid priority value overflow

* common/prque: use int priority in some tests

No need to convert to int64 when we can just change the type used by the
queue.

* common/prque: remove comment about int64 range

---------

Co-authored-by: Zsolt Felfoldi <zsfelfoldi@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-02-09 13:03:54 +02:00
Martin Holst Swende ed51b8c5d3
ethdb: pebble backend (64bit platforms only) (#26517)
* ethdb: use pebble

Co-authored-by: Gary Rong <garyrong0905@gmail.com>

foo

update

* apply suggested changes

* flags: go format

node: fix ddir lookup mistake

accounts/abi/bind: fix go.mod replacement for generated binding

deps: update pebble + with fix 32-bit build

* ethdb/pebble: respect max memtable size

* core/rawdb, ethdb: enable pebble on non-32bit platforms only

* core/rawdb: fix build tags, fix some review concerns

* core/rawdb: refactor methods for database opening

* core/rawdb: remove erroneous build tag

* cmd/geth: fix the flag default handling + testcase

* cmd/geth: improve testing regarding custom backends

* ethdb/pebble, deps: update pebble dependency

* core/rawdb: replace method with Open

* ethdb/pebble: several updates for pebble (#49)

* ethdb/pebble: fix size count in batch

* ethdb/pebble: disable seek compaction

* ethdb/pebble: more fixes

* ethdb, core, cmd: polish and fixes (#50)

* cmd/utils, core/rawdb, ethdb/pebble: address some review concerns

* Update flags.go

* ethdb/pebble: minor refactors

* ethdb/pebble: avoid copy on batch replay

* ethdb: fix compilation flaw

* cmd: fix test fail due to mismatching error message

* cmd/geth, node: rename backingdb to db.engine

---------

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2023-02-09 10:48:34 +02:00
Péter Szilágyi 095e365fac
all: remove support for Ropsten (#26644) 2023-02-09 10:03:00 +02:00
rjl493456442 9842301376
all: remove database commit callback, rework noderesolver (#26637)
This change ports some changes from the main PBSS PR:

  - get rid of callback function in `trie.Database.Commit` which is not required anymore
  - rework the `nodeResolver` in `trie.Iterator` to make it compatible with multiple state scheme
  - some other shallow changes in tests and typo-fixes
2023-02-08 06:14:34 -05:00
Martin Holst Swende 91cb6f863a
tests: update (#26548)
This updates the reference tests to the latest version.

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-02-06 14:52:51 -05:00
Felföldi Zsolt 3a5aceed8f
beacon/engine: move core/beacon to beacon/engine (#26616)
This PR moves core/beacon to beacon/engine so that beacon-chain related code has its own top level package which also can house the the beacon lightclient-code.
2023-02-06 10:37:58 -05:00
Martin Holst Swende 8860b39754
all: prepare for path-based trie storage (#26603)
This PR moves some trie-related db accessor methods to a different file, and also removes the schema type. Instead of the schema type, a string is used to distinguish between hashbased/pathbased db accessors.
This also moves some code from trie package to rawdb package.

This PR is intended to be a no-functionality-change prep PR for #25963 .

---------

Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2023-02-06 10:28:40 -05:00
raulk 918aed4e31
core/vm: add bn256ScalarMul testcase for zero scalar value (#26607)
EIP-196 allows a zero value in the scalar argument to precompile `0x07`. This change adds a test for that case.
2023-02-06 07:41:29 -05:00
Marius van der Wijden 9826cd65bc
eth/catalyst: implement engine_getPayloadBodiesByHash/Range methods (#26232)
This change implements engine_getPayloadBodiesByHash and engine_getPayloadBodiesByRange, according to the specification at https://github.com/ethereum/execution-apis/blob/main/src/engine/shanghai.md#specification-4 .

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-02-06 04:21:40 -05:00
lmittmann 877d2174fb
core/vm: improve EVM instance reusability (#26341)
This change improves reusability of the EVM struct. Two methods are added:

- SetBlockContext(...)
- SetTracer(...)

Other attributes like the TransactionContext and the StateDB can already be updated.
BlockContext and Tracer are partially not updateable right now. This change fixes it and
opens the potential to reuse an EVM struct in more ways.

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-02-05 15:11:25 +01:00
Marius van der Wijden a8cf4399a9
eth/catalyst: return invalid params instead of invalid payload params (#26591) 2023-02-02 11:52:26 +02:00
Martin Holst Swende 5c8cc10d1e
core: improve ambiguous block validation message (#26582) 2023-02-01 10:08:25 -05:00
Marius van der Wijden 2a2b0419fb
all: implement withdrawals (EIP-4895) (#26484)
This change implements withdrawals as specified in EIP-4895.

Co-authored-by: lightclient@protonmail.com <lightclient@protonmail.com>
Co-authored-by: marioevz <marioevz@gmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-01-25 15:32:25 +01:00
Martin Holst Swende 2b57a27d9e
all: make timestamp-based fork checks based on uint64 (#26474)
This PR changes the API so that uint64 is used for fork timestamps.
It's a good choice because types.Header also uses uint64 for time.

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-01-25 12:12:28 +01:00
rjl493456442 50e65392aa
params: define cancun and prague as timestamp based forks (#26481)
* params: define cancun and osaka as timestamp based forks

* core, params: change osaka to prague

* params: fix
2023-01-20 11:57:50 +02:00
Zachinquarantine 690338f0fa
all: remove Kiln testnet (#26522)
Kiln was deprecated after the merge.
2023-01-19 10:49:48 +01:00
Marius van der Wijden a35b654f25
core/txpool: check if initcode size is exceeded (#26504)
* core/txpool: check if initcode size is exceeded

* core/txpool: move check
2023-01-18 03:47:42 -05:00
Martin Holst Swende 0b53b29078
core/rawdb: fix cornercase shutdown behaviour in freezer (#26485)
This PR does a few things. 
It fixes a shutdown-order flaw in the chainfreezer. Previously, the chain-freezer would shutdown the freezer backend first, and then signal for the loop to exit. This can lead to a scenario where the freezer tries to fsync closed files, which is an error-conditon that could lead to exit via log.Crit. 

It also makes the printout more detailed when truncating 'dangling' items, by showing the exact number instead of approximate MB.

This PR also adds calls to fsync files before closing them, and also makes the `db inspect` command slightly more robust.
2023-01-16 03:57:27 -05:00
rjl493456442 e04d63ebd3
core/rawdb: fsync head data file before closing it (#26490)
This PR fixes an issue which might result in data lost in freezer.

Whenever mutation happens in freezer, all data will be written into head data file
and it will be rotated with a new one in case the size of file reaches the threshold.

Theoretically, the rotated old data file should be fsync'd to prevent data loss.
In freezer.Sync function, we only fsync: (1) index file (2) meta file and (3) head
data file. So this PR forcibly fsync the head data file if mutation happens in the
boundary of data file.
2023-01-13 05:55:50 -05:00
Andrew Ashikhmin c125e6e00c
core/vm: enable EIP-3855 (PUSH0) in Shanghai (#26475) 2023-01-12 11:00:08 +01:00
Andrei Maiboroda 793f0f9ec8
core/vm: implement EIP-3860: Limit and meter initcode (#23847)
Implementation of https://eips.ethereum.org/EIPS/eip-3860, limit and meter initcode. This PR enables EIP-3860 as part of the Shanghai fork. 


Co-authored-by: lightclient@protonmail.com <lightclient@protonmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
2023-01-11 04:05:47 -05:00
Martin Holst Swende 2c6dda5ad7
eth/tracers: use non-threaded tracechain (#24283)
This makes non-JS tracers execute all block txs on a single goroutine.
In the previous implementation, we used to prepare every tx pre-state
on one goroutine, and then run the transactions again with tracing enabled.
Native tracers are usually faster, so it is faster overall to use their output as
the pre-state for tracing the next transaction.

Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
2023-01-10 15:11:53 +01:00
Martin Holst Swende 7a489623ac
core/state: remove notion of fake storage (#24916)
This PR removes the notion of fakeStorage from the state objects, and instead, for any state modifications that are needed, it simply makes the changes.
2023-01-10 08:24:30 -05:00
Péter Szilágyi 2189773093
Merge pull request #25878 from MariusVanDerWijden/shanghai-by-time
params: core: enable shanghai based on timestamps
2023-01-06 16:10:12 +02:00
Péter Szilágyi b56c796220
cmd, core, eth, els, params: disallow setheads below genesis, tweaks 2023-01-06 15:29:58 +02:00
Péter Szilágyi fcf3d00488
eth, les: polish forkid a bit, fix races and transition validation 2023-01-06 15:29:57 +02:00
rjl493456442 6c149fd4ad
core, eth, trie, light: clean up trie interface (#26388)
* all: cleanup trie interface

* eth, trie: address comments
2023-01-03 15:41:40 +02:00
Péter Szilágyi d021157820
core, les, light: implement timestamp based sethead and genesis rewinds 2023-01-03 12:57:07 +02:00
Péter Szilágyi 08481028fe
core, les, params: add timestamp based fork compatibility checks 2023-01-03 12:57:06 +02:00
Marius van der Wijden a4e19c5ca3
all: implement forkid changes for shanghai 2023-01-03 12:57:06 +02:00
Marius van der Wijden efc9409ca9
params: core: enable shanghai based on timestamps 2023-01-03 12:56:25 +02:00
Marius van der Wijden db82ea2ee3
core: reset txpool on sethead (#26392)
This change sends a HeadChainEvent when SetHead is invoked. The txpool will then reset the txnoncer on receiving the event.
2022-12-29 07:24:34 -05:00
rjl493456442 c87f321b8f
core/state: maintain destruction flag by default (#26371)
This changes moves the tracking of "deleted in this block" out from snap-only domain, so that it happens regardless of whether the execution is snapshot-backed or trie-backed.
2022-12-28 08:53:43 -05:00
rjl493456442 01808421e2
core/state: return error when storage trie can't be opened (#26350)
This changes the StorageTrie method to return an error when the trie
is not available. It used to return an 'empty trie' in this case, but that's
not possible anymore under PBSS.
2022-12-21 10:21:21 +01:00
Martin Holst Swende b818e73ef3
tests: update tests (#26314)
This PR builds on #26299, but also updates the tests to the most recent version, which includes tests regarding TheMerge.

This change adds checks to the beacon consensus engine, making it more strict in validating the pre- and post-headers, and not relying on the caller to have already correctly sanitized the headers/blocks.
2022-12-20 09:56:52 -05:00
rjl493456442 79a478bb61
core/rawdb: implement resettable freezer (#26324)
This PR implements resettable freezer by adding a ResettableFreezer wrapper.

The resettable freezer wraps the original freezer in a way that makes it possible to ensure atomic resets. Implementation wise, it relies on the os.Rename and os.RemoveAll to atomically delete the original freezer data and re-create a new one from scratch.
2022-12-19 04:59:12 -05:00
Ikko Ashimine f51f6edb40
core: fix typo in blockchain_snapshot_test.go (#26357)
correspnding -> corresponding
2022-12-14 11:48:12 +02:00
rjl493456442 cda051eba7
core, cmd: fill blockNumber in logs (#26345)
* core, cmd: fill blockNumber in logs

* Update core/state/statedb.go

Co-authored-by: Martin Holst Swende <martin@swende.se>

* core/types: revert

* core/state: improve comments

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-12-13 07:54:16 -05:00
Alex Stokes 250a80a50e
core: fix comment about recipient for contract creation (#26349)
A comment suggests that contract creation happens if the recipient of a call is 0x00..00 ("zero address") but in fact the sender must be nil. The zero address is a regular valid address that is commonly used as a "burn" address.
2022-12-13 03:33:40 -05:00
Felix Lange 3315bad256
core: pass block into collectLogs (#26335)
While investigating another issue, I found that all callers of collectLogs have the
complete block available. rawdb.ReadReceipts loads the block from the database,
so it is better to use ReadRawReceipts here, and derive the receipt information using
the block which is already in memory.
2022-12-09 16:14:33 +01:00
Sina Mahmoodi 711afbc7fd
core,eth: add `debug_setTrieFlushInterval` to change trie flush frequency (#24785)
This PR makes it possible to modify the flush interval time via RPC. On one extreme, `0s`, it would act as an archive node. If set to `1h`, means that after one hour of effective block processing time, the trie would be flushed. If one block takes 200ms, this means that a flush would occur every `5*3600=18000`  blocks -- however, if the memory size of the cached states grows too large, it will flush sooner. 

Essentially, this makes it possible to configure the node to be more or less "archive:ish", and without restarting the node while reconfiguring it.
2022-12-09 07:40:17 -05:00
Sina Mahmoodi 6891288787
core: fix state flushing for catalyst mode (#26319)
The gcproc field tracks the amount of time spent processing blocks,
and is used to trigger a state flush to disk when a certain threshold is
reached. After the merge, single block insertion by CL is the most
common source of block processing time, but this time was not added
into gcproc.
2022-12-07 17:36:54 +01:00
Martin Holst Swende 1f35988a00
eth/tracers, core/vm: remove `time` from trace output and tracing interface (#26291)
This removes the 'time' field from logs, as well as from the tracer interface. This change makes the trace output deterministic.  If a tracer needs the time they can measure it themselves. No need for evm to do this.

Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
2022-12-05 16:57:47 +01:00
Håvard Anda Estensen 06632da2bb
all: assign zero after resize in implementations of heap.Interface (#26296)
This changes the Pop method to assign the zero value before
reducing slice size. Doing so ensures the backing array does not
reference removed item values.
2022-12-05 13:49:54 +01:00
Sina Mahmoodi 10347c6b54
core: drop legacy receipt types (#26225)
This PR drops the legacy receipt types, the freezer-migrate command and the startup check. The previous attempt #22852 at this failed because there were users who still had legacy receipts in their db, so it had to be reverted #23247. Since then we added a command to migrate legacy dbs #24028.

As of the last hardforks all users either must have done the migration, or used the --ignore-legacy-receipts flag which will stop working now.
2022-12-03 21:42:11 +01:00
rjl493456442 743e404906
core, eth, les, tests, trie: abstract node scheme (#25532)
This PR introduces a node scheme abstraction. The interface is only implemented by `hashScheme` at the moment, but will be extended by `pathScheme` very soon.

Apart from that, a few changes are also included which is worth mentioning:

-  port the changes in the stacktrie, tracking the path prefix of nodes during commit
-  use ethdb.Database for constructing trie.Database. This is not necessary right now, but it is required for path-based used to open reverse diff freezer
2022-11-28 14:31:28 +01:00
setunapo 0dc9b01c11
core: clarify code in forkchoice (#26257)
refactoring without logic change
2022-11-28 11:07:25 +01:00
rjl493456442 add1bff13f
core/rawdb: fix freezer validation (#26251)
* core/rawdb: fix freezer validation

* core/rawdb: address comment
2022-11-25 09:10:31 +01:00
Felix Lange 193f350eb9
core/rawdb: improve freezerTable.Sync (#26245)
While investigating #22374, I noticed that the Sync operation of the
freezer does not take the table lock. It also doesn't call sync for all files
if there is an error with one of them. I doubt this will fix anything, but
didn't want to drop the fix on the floor either.
2022-11-24 10:50:28 +01:00
Marius van der Wijden ec2ec2d08e
core: implement EIP-3651, warm coinbase (#25819)
Implements EIP-3651, "Warm Coinbase", for Shanghai hardfork. Specification: https://eips.ethereum.org/EIPS/eip-3651.
2022-11-22 22:39:52 +01:00
Mark Tyneway b4ea2bf7dd
all: implement EIP-1153 transient storage (#26003)
Implements TSTORE and TLOAD as specified by the following EIP:

https://eips.ethereum.org/EIPS/eip-1153
https://ethereum-magicians.org/t/eip-1153-transient-storage-opcodes/553


Co-authored-by: Sara Reynolds <snreynolds2506@gmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2022-11-16 10:18:52 +01:00
Felix Lange 9afc6816d2
common/lru: add generic LRU implementation (#26162)
It seems there is no fully typed library implementation of an LRU cache.
So I wrote one. Method names are the same as github.com/hashicorp/golang-lru,
and the new type can be used as a drop-in replacement.

Two reasons to do this:

- It's much easier to understand what a cache is for when the types are right there.
- Performance: the new implementation is slightly faster and performs zero memory
   allocations in Add when the cache is at capacity. Overall, memory usage of the cache
   is much reduced because keys are values are no longer wrapped in interface.
2022-11-14 15:41:56 +01:00
Martin Holst Swende 5fded04037
core/state: replace fastcache code cache with gc-friendly structure (#26092)
This PR replaces fastcache with a pretty simple LRU which does not require explicit closing.
2022-11-09 08:06:02 +01:00
yihuang 7dc5e785a8
core/vm: deepcopy jumptable when enabling extra eips (#26137)
When the interpreter is configured to use extra-eips, this change makes it so that all the opcodes are deep-copied, to prevent accidental modification of the 'base' jumptable. 

Closes: #26136

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-11-09 07:52:13 +01:00
protolambda a51188a163
params: make eip1559 params non-global (#25994)
This PR changes geth to read the eip1559 params from the chain config instead of the globals.

This way the parameters may be changed by forking the chain config code, without creating a large diff throughout the past and future usages of the parameters.

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-11-04 16:27:15 +01:00
rjl493456442 60e30a940b
core/rawdb: refactor db inspector for extending multiple ancient store (#25896)
This PR ports a few changes from PBSS:

- Fix the snapshot generator waiter in case the generation is not even initialized
- Refactor db inspector for ancient store
2022-10-28 10:23:49 +02:00
Martin Holst Swende 9d795d0836
core/vm: use optimized bigint (#26021) 2022-10-27 10:39:01 +02:00
Péter Szilágyi c4a662176e
core, eth: for types with accurate size calcs, return uint64, not float (#26046)
* core, eth: for types with accurate size calcs, return uint64, not float

* core/types: proper tx size tests

* core/types: extend tx size test with decoded sizes, fix error

* core/txpool: fix linter

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-10-26 15:23:07 +03:00
Péter Szilágyi 68ba845bb5
core/types: rename tx files to group them better together 2022-10-26 13:30:51 +03:00
Péter Szilágyi a6dda03644
all: refactor txpool into it's own package in prep for 4844 2022-10-24 16:35:53 +03:00
rjl493456442 b9ba6f6e4d
core/rawdb: open meta file in read only mode (#26009) 2022-10-19 21:15:43 +02:00
s7v7nislands d86fe26f67
core/rawdb: refactor db key prefix (#26000)
Co-authored-by: seven <seven@nodereal.io>
2022-10-19 09:53:09 +02:00
Martin Holst Swende 6069d8294e
cmd/utils: enable snapshot generation in import-mode (#25990)
This PR fixes a regression causing snapshots not to be generated in "geth --import" mode.  It also fixes the geth export command to be truly readonly, and adds a new test for geth export.
2022-10-19 08:20:39 +02:00
Martin Holst Swende bed3b10086
common/math: optimized modexp (+ fuzzer) (#25525)
This adds a 
* core/vm, tests: optimized modexp + fuzzer

* common/math: modexp optimizations

* core/vm: special case base 1 in big modexp

* core/vm: disable fastexp
2022-10-12 10:34:52 +02:00
Marius van der Wijden a007ab786c
core/types: add more context around ErrInvalidChainID (#25367)
This changes the error message for mismatching chain ID to show
the given and expected value. Callers expecting this error must be
changed to use errors.Is.
2022-10-12 10:27:39 +02:00
Martin Holst Swende 28d076d37e
core/rawdb: provide more info on 'gap in the chain' error (#25938) 2022-10-12 09:35:09 +02:00
Martin Holst Swende 5a02b2d6d0
all: fix spelling mistakes (#25961) 2022-10-11 09:37:00 +02:00
Martin Holst Swende 067bac3f24
core, ethclient/gethclient: improve flaky tests (#25918)
* ethclient/gethclient: improve time-sensitive flaky test

* eth/catalyst: fix (?) flaky test

* core: stop blockchains in tests after use

* core: fix dangling blockchain instances

* core: rm whitespace

* eth/gasprice, eth/tracers, consensus/clique: stop dangling blockchains in tests

* all: address review concerns

* core: goimports

* eth/catalyst: fix another time-sensitive test

* consensus/clique: add snapshot test run function

* core: rename stop() to stopWithoutSaving()

Co-authored-by: Felix Lange <fjl@twurst.com>
2022-10-06 13:39:20 +02:00
Martin Holst Swende deead99731
core/bloombits: speed up windows-test (#25844)
core/bloombits: remove micro-sleep
2022-10-06 11:41:07 +02:00
Martin Holst Swende e14164d516
core: fix linter complaint (#25927) 2022-10-04 18:09:05 +02:00
Péter Szilágyi 4f7a425aa8
Merge pull request #25924 from holiman/comments_fix
all: fix docstrings
2022-10-04 13:30:00 +03:00
Martin Holst Swende ee301c750b
all: fix docstrings 2022-10-04 09:18:02 +02:00
zhiqiangxu a724163e59
core/types: add a testcase to TestLegacyReceiptDecoding (#25909)
add a testcase to TestLegacyReceiptDecoding
2022-10-03 12:56:21 +02:00
rjl493456442 052c634917
core: rework tx indexer (#25723)
This PR reworks tx indexer a bit. Compared to the original version, one scenario is no longer handled - upgrading  from legacy geth without indexer support. 

The tx indexer was introduced in 2020 and have been present through hardforks, so it can be assumed that all Geth nodes have tx indexer already. So we can simplify the tx indexer logic a bit:

-    If the tail flag is not present, it means node is just initialized may or may not with an ancient store attached. In this case all blocks are regarded as unindexed
-   If the tail flag is present, it means blocks below tail are unindexed, blocks above tail are indexed

This change also address some weird cornercases that could make the indexer not work after a crash.
2022-09-29 09:50:24 +02:00
lmittmann 2b1299b1c0
core/vm: minor trivial clean up (#25880)
* dropped `vm.keccakState` for `crypto.KeccakState`

* cleaned up `OpCode.IsPush()`
2022-09-28 13:21:54 +02:00
rjl493456442 bff84a99fe
cmd, core, eth, les, light: track deleted nodes (#25757)
* cmd, core, eth, les, light: track deleted nodes

* trie: add docs

* trie: address comments

* cmd, core, eth, les, light, trie: trie id

* trie: add tests

* trie, core: updates

* trie: fix imports

* trie: add utility print-method for nodeset

* trie: import err

* trie: fix go vet warnings

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-09-27 10:01:02 +02:00
jwasinger 7227c9ef07
cmd/geth: make dumpgenesis load genesis datadir if it exists (#25135)
`geth dumpgenesis` currently does not respect the content of the data directory. Instead, it outputs the genesis block created by command-line flags. This PR fixes it to read the genesis from the database, if the database already exists.


Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-09-26 13:55:18 +02:00
Leon c55c56cf0a
core/vm: better handle error on eip activation check (#25131)
* core/vm: correct logic for eip check of NewEVMInterpreter

* refactor
2022-09-26 13:20:38 +02:00
Martin Holst Swende e004e7d256
core: fix datarace in txpool, fixes #25870 and #25869 (#25872)
core: fix datarace in txpool pendingnoce, fixes #25870
2022-09-26 11:34:15 +02:00
zhiqiangxu 367e60549a
core/types: improve accuracy of header.Size() (#25859)
The header.Size() method did not take the basefee into account.
2022-09-26 09:10:39 +02:00
rjl493456442 3da42f85d9
all: clean up the configs for pruner and snapshotter (#22396)
This PR cleans up the configurations for pruner and snapshotter by passing a config struct.

And also, this PR disables the snapshot background generation if the chain is opened in "read-only" mode. The read-only mode is necessary in some cases. For example, we have a list of commands to open the etheruem node in "read-only" mode, like export-chain. In these cases, the snapshot background generation is non expected and should be banned explicitly.
2022-09-23 20:20:36 +02:00
lightclient 338bb2e36c
core/blockchain: downgrade tx indexing and unindexing logs (#25832)
core/blockchain: downgrade tx indexing and unindexing logs from info to debug

If a user has a finite tx lookup limit, they will see an "unindexing" info level log each time a block is imported. This information might help a user understand that they are removing the index each block and some txs may not be retrievable by hash, but overall it is generally more of a nuisance than a benefit. This change downgrades the log to a debug log.
2022-09-21 09:49:17 +02:00
jwasinger d8ada03eac
core/vm: return copy of input slice in identity precompile, avoid returndata copy (#25183)
* core/vm: return copy of input slice in identity precompile.  don't deep copy return data slice upon call completion

* make use of common.CopyBytes
2022-09-20 14:58:03 +02:00
Felix Lange 468d1844c7
core: move build version reading to its own package (#25806)
This fixes the build with Go 1.17, which does not have BuildInfo.Settings yet.
2022-09-19 10:04:16 +02:00
lightclient 8e5201551d
core: more compact bad block report (#25714)
This shortens the chain config summary in bad block reports,
and adds go-ethereum version information as well.

Co-authored-by: Felix Lange <fjl@twurst.com>
2022-09-17 21:27:10 +02:00
Seungbae Yu ada603fab5
core: don't cache zero nonce in txNoncer (#25603)
This changes the nonce cache used by TxPool to not store cached
nonces for non-existing accounts.
2022-09-16 18:23:13 +02:00
ucwong 83989a19be
core/rawdb: fix leak of backoff timer (#25776) 2022-09-16 11:33:48 +02:00
Felix Lange 0ee8b273f2
core: fix log order in "Imported new chain segment" (#25768) 2022-09-15 13:18:23 +02:00
Seungbae Yu 3a4cef5402
core: preallocate maps in TxPool helper methods (#25737) 2022-09-12 15:02:41 +02:00
Felix Lange b628d72766
build: upgrade to go 1.19 (#25726)
This changes the CI / release builds to use the latest Go version. It also
upgrades golangci-lint to a newer version compatible with Go 1.19.

In Go 1.19, godoc has gained official support for links and lists. The
syntax for code blocks in doc comments has changed and now requires a
leading tab character. gofmt adapts comments to the new syntax
automatically, so there are a lot of comment re-formatting changes in this
PR. We need to apply the new format in order to pass the CI lint stage with
Go 1.19.

With the linter upgrade, I have decided to disable 'gosec' - it produces
too many false-positive warnings. The 'deadcode' and 'varcheck' linters
have also been removed because golangci-lint warns about them being
unmaintained. 'unused' provides similar coverage and we already have it
enabled, so we don't lose much with this change.
2022-09-10 13:25:40 +02:00
Martin Holst Swende 389021a5af
core: place a cap on reorglogs (#25711)
This PR makes the event-sending for deleted and new logs happen in batches, to prevent OOM situation due to large reorgs. 

Co-authored-by: Felix Lange <fjl@twurst.com>
2022-09-09 15:25:55 +02:00
rjl493456442 dea1fb3cfc
all: cleanup tests (#25641)
Follow-up to PR #25523 to cleanup all relevant tests.
2022-09-07 20:21:59 +02:00
Marius van der Wijden d30e39b2f8
core/vm: rename RANDOM to PREVRANDAO (#25691)
See: https://github.com/ethereum/EIPs/blob/master/EIPS/eip-4399.md

>  This EIP supplants the semantics of the return value of existing DIFFICULTY (0x44) opcode and renames the opcode to PREVRANDAO (0x44).
2022-09-07 13:09:52 +02:00
rjl493456442 5ddedd2f83
core, light, trie: remove DiskDB function from trie database (#25690) 2022-09-07 09:08:56 +02:00
rjl493456442 d46184c969
core/state: filter out nil trie for copy (#25575) 2022-09-01 09:34:32 +02:00