Commit Graph

2299 Commits

Author SHA1 Message Date
Marius van der Wijden 7fb42e6db2
eth/downloader: handle missing withdrawals if empty list is expected (#26675)
This PR relaxes the block body ingress handling a bit: if block body withdrawals are missing (but expected to be empty), the body withdrawals are set to 'empty list' before being passed to upper layers. 

This fixes an issue where a block passed from EthereumJS to geth was deemed invalid.
2023-02-15 07:23:07 -05:00
Sina Mahmoodi 2def62b99b
eth/filters: avoid block body retrieval when no matching logs (#25199)
Logs stored on disk have minimal information. Contextual information such as block
number, index of log in block, index of transaction in block are filled in upon request.
We can fill in all these fields only having the block header and list of receipts.
But determining the transaction hash of a log requires the block body.

The goal of this PR is postponing this retrieval until we are sure we the transaction hash.
It happens often that the header bloom filter signals there might be matches in a block,
but after actually checking them reveals the logs do not match. We want to avoid fetching
the body in this case.

Note that this changes the semantics of Backend.GetLogs. Downstream callers of
GetLogs now assume log context fields have not been derived, and need to call
DeriveFields on the logs if necessary.
2023-02-13 10:59:27 +01:00
Marius van der Wijden 241cf62b5c
params: schedule shanghai fork on sepolia (#26662)
* params: schedule shanghai fork on sepolia

* params: u64 -> newUint64
2023-02-10 13:34:45 +02:00
Chris Ziogas b0cd8c4a5c
core/vm: set tracer-observable `value` of a delegatecall to match parent `value` (#26632)
This is a breaking change in the tracing hooks API as well as semantics of the callTracer:

- CaptureEnter hook provided a nil value argument in case of DELEGATECALL. However to stay consistent with how delegate calls behave in EVM this hook is changed to pass in the value of the parent call.
- callTracer will return parent call's value for DELEGATECALL frames.

---------

Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
2023-02-10 10:26:43 +01:00
Martin Holst Swende 22c3ad1d12
core/state, trie: remove unused error-return from trie Commit operation (#26641) 2023-02-09 08:56:59 -05:00
Péter Szilágyi bf1798e04e
common/prque: generic priority queue (#26290)
* common, core, eth, les, trie: make prque generic

* les/vflux/server: fixed issues in priorityPool

* common, core, eth, les, trie: make priority also generic in prque

* les/flowcontrol: add test case for priority accumulator overflow

* les/flowcontrol: avoid priority value overflow

* common/prque: use int priority in some tests

No need to convert to int64 when we can just change the type used by the
queue.

* common/prque: remove comment about int64 range

---------

Co-authored-by: Zsolt Felfoldi <zsfelfoldi@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-02-09 13:03:54 +02:00
Martin Holst Swende ed51b8c5d3
ethdb: pebble backend (64bit platforms only) (#26517)
* ethdb: use pebble

Co-authored-by: Gary Rong <garyrong0905@gmail.com>

foo

update

* apply suggested changes

* flags: go format

node: fix ddir lookup mistake

accounts/abi/bind: fix go.mod replacement for generated binding

deps: update pebble + with fix 32-bit build

* ethdb/pebble: respect max memtable size

* core/rawdb, ethdb: enable pebble on non-32bit platforms only

* core/rawdb: fix build tags, fix some review concerns

* core/rawdb: refactor methods for database opening

* core/rawdb: remove erroneous build tag

* cmd/geth: fix the flag default handling + testcase

* cmd/geth: improve testing regarding custom backends

* ethdb/pebble, deps: update pebble dependency

* core/rawdb: replace method with Open

* ethdb/pebble: several updates for pebble (#49)

* ethdb/pebble: fix size count in batch

* ethdb/pebble: disable seek compaction

* ethdb/pebble: more fixes

* ethdb, core, cmd: polish and fixes (#50)

* cmd/utils, core/rawdb, ethdb/pebble: address some review concerns

* Update flags.go

* ethdb/pebble: minor refactors

* ethdb/pebble: avoid copy on batch replay

* ethdb: fix compilation flaw

* cmd: fix test fail due to mismatching error message

* cmd/geth, node: rename backingdb to db.engine

---------

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2023-02-09 10:48:34 +02:00
Péter Szilágyi 095e365fac
all: remove support for Ropsten (#26644) 2023-02-09 10:03:00 +02:00
rjl493456442 9842301376
all: remove database commit callback, rework noderesolver (#26637)
This change ports some changes from the main PBSS PR:

  - get rid of callback function in `trie.Database.Commit` which is not required anymore
  - rework the `nodeResolver` in `trie.Iterator` to make it compatible with multiple state scheme
  - some other shallow changes in tests and typo-fixes
2023-02-08 06:14:34 -05:00
Martin Holst Swende 91cb6f863a
tests: update (#26548)
This updates the reference tests to the latest version.

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-02-06 14:52:51 -05:00
Felföldi Zsolt 3a5aceed8f
beacon/engine: move core/beacon to beacon/engine (#26616)
This PR moves core/beacon to beacon/engine so that beacon-chain related code has its own top level package which also can house the the beacon lightclient-code.
2023-02-06 10:37:58 -05:00
Martin Holst Swende 8860b39754
all: prepare for path-based trie storage (#26603)
This PR moves some trie-related db accessor methods to a different file, and also removes the schema type. Instead of the schema type, a string is used to distinguish between hashbased/pathbased db accessors.
This also moves some code from trie package to rawdb package.

This PR is intended to be a no-functionality-change prep PR for #25963 .

---------

Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2023-02-06 10:28:40 -05:00
raulk 918aed4e31
core/vm: add bn256ScalarMul testcase for zero scalar value (#26607)
EIP-196 allows a zero value in the scalar argument to precompile `0x07`. This change adds a test for that case.
2023-02-06 07:41:29 -05:00
Marius van der Wijden 9826cd65bc
eth/catalyst: implement engine_getPayloadBodiesByHash/Range methods (#26232)
This change implements engine_getPayloadBodiesByHash and engine_getPayloadBodiesByRange, according to the specification at https://github.com/ethereum/execution-apis/blob/main/src/engine/shanghai.md#specification-4 .

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-02-06 04:21:40 -05:00
lmittmann 877d2174fb
core/vm: improve EVM instance reusability (#26341)
This change improves reusability of the EVM struct. Two methods are added:

- SetBlockContext(...)
- SetTracer(...)

Other attributes like the TransactionContext and the StateDB can already be updated.
BlockContext and Tracer are partially not updateable right now. This change fixes it and
opens the potential to reuse an EVM struct in more ways.

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-02-05 15:11:25 +01:00
Marius van der Wijden a8cf4399a9
eth/catalyst: return invalid params instead of invalid payload params (#26591) 2023-02-02 11:52:26 +02:00
Martin Holst Swende 5c8cc10d1e
core: improve ambiguous block validation message (#26582) 2023-02-01 10:08:25 -05:00
Marius van der Wijden 2a2b0419fb
all: implement withdrawals (EIP-4895) (#26484)
This change implements withdrawals as specified in EIP-4895.

Co-authored-by: lightclient@protonmail.com <lightclient@protonmail.com>
Co-authored-by: marioevz <marioevz@gmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-01-25 15:32:25 +01:00
Martin Holst Swende 2b57a27d9e
all: make timestamp-based fork checks based on uint64 (#26474)
This PR changes the API so that uint64 is used for fork timestamps.
It's a good choice because types.Header also uses uint64 for time.

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-01-25 12:12:28 +01:00
rjl493456442 50e65392aa
params: define cancun and prague as timestamp based forks (#26481)
* params: define cancun and osaka as timestamp based forks

* core, params: change osaka to prague

* params: fix
2023-01-20 11:57:50 +02:00
Zachinquarantine 690338f0fa
all: remove Kiln testnet (#26522)
Kiln was deprecated after the merge.
2023-01-19 10:49:48 +01:00
Marius van der Wijden a35b654f25
core/txpool: check if initcode size is exceeded (#26504)
* core/txpool: check if initcode size is exceeded

* core/txpool: move check
2023-01-18 03:47:42 -05:00
Martin Holst Swende 0b53b29078
core/rawdb: fix cornercase shutdown behaviour in freezer (#26485)
This PR does a few things. 
It fixes a shutdown-order flaw in the chainfreezer. Previously, the chain-freezer would shutdown the freezer backend first, and then signal for the loop to exit. This can lead to a scenario where the freezer tries to fsync closed files, which is an error-conditon that could lead to exit via log.Crit. 

It also makes the printout more detailed when truncating 'dangling' items, by showing the exact number instead of approximate MB.

This PR also adds calls to fsync files before closing them, and also makes the `db inspect` command slightly more robust.
2023-01-16 03:57:27 -05:00
rjl493456442 e04d63ebd3
core/rawdb: fsync head data file before closing it (#26490)
This PR fixes an issue which might result in data lost in freezer.

Whenever mutation happens in freezer, all data will be written into head data file
and it will be rotated with a new one in case the size of file reaches the threshold.

Theoretically, the rotated old data file should be fsync'd to prevent data loss.
In freezer.Sync function, we only fsync: (1) index file (2) meta file and (3) head
data file. So this PR forcibly fsync the head data file if mutation happens in the
boundary of data file.
2023-01-13 05:55:50 -05:00
Andrew Ashikhmin c125e6e00c
core/vm: enable EIP-3855 (PUSH0) in Shanghai (#26475) 2023-01-12 11:00:08 +01:00
Andrei Maiboroda 793f0f9ec8
core/vm: implement EIP-3860: Limit and meter initcode (#23847)
Implementation of https://eips.ethereum.org/EIPS/eip-3860, limit and meter initcode. This PR enables EIP-3860 as part of the Shanghai fork. 


Co-authored-by: lightclient@protonmail.com <lightclient@protonmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
2023-01-11 04:05:47 -05:00
Martin Holst Swende 2c6dda5ad7
eth/tracers: use non-threaded tracechain (#24283)
This makes non-JS tracers execute all block txs on a single goroutine.
In the previous implementation, we used to prepare every tx pre-state
on one goroutine, and then run the transactions again with tracing enabled.
Native tracers are usually faster, so it is faster overall to use their output as
the pre-state for tracing the next transaction.

Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
2023-01-10 15:11:53 +01:00
Martin Holst Swende 7a489623ac
core/state: remove notion of fake storage (#24916)
This PR removes the notion of fakeStorage from the state objects, and instead, for any state modifications that are needed, it simply makes the changes.
2023-01-10 08:24:30 -05:00
Péter Szilágyi 2189773093
Merge pull request #25878 from MariusVanDerWijden/shanghai-by-time
params: core: enable shanghai based on timestamps
2023-01-06 16:10:12 +02:00
Péter Szilágyi b56c796220
cmd, core, eth, els, params: disallow setheads below genesis, tweaks 2023-01-06 15:29:58 +02:00
Péter Szilágyi fcf3d00488
eth, les: polish forkid a bit, fix races and transition validation 2023-01-06 15:29:57 +02:00
rjl493456442 6c149fd4ad
core, eth, trie, light: clean up trie interface (#26388)
* all: cleanup trie interface

* eth, trie: address comments
2023-01-03 15:41:40 +02:00
Péter Szilágyi d021157820
core, les, light: implement timestamp based sethead and genesis rewinds 2023-01-03 12:57:07 +02:00
Péter Szilágyi 08481028fe
core, les, params: add timestamp based fork compatibility checks 2023-01-03 12:57:06 +02:00
Marius van der Wijden a4e19c5ca3
all: implement forkid changes for shanghai 2023-01-03 12:57:06 +02:00
Marius van der Wijden efc9409ca9
params: core: enable shanghai based on timestamps 2023-01-03 12:56:25 +02:00
Marius van der Wijden db82ea2ee3
core: reset txpool on sethead (#26392)
This change sends a HeadChainEvent when SetHead is invoked. The txpool will then reset the txnoncer on receiving the event.
2022-12-29 07:24:34 -05:00
rjl493456442 c87f321b8f
core/state: maintain destruction flag by default (#26371)
This changes moves the tracking of "deleted in this block" out from snap-only domain, so that it happens regardless of whether the execution is snapshot-backed or trie-backed.
2022-12-28 08:53:43 -05:00
rjl493456442 01808421e2
core/state: return error when storage trie can't be opened (#26350)
This changes the StorageTrie method to return an error when the trie
is not available. It used to return an 'empty trie' in this case, but that's
not possible anymore under PBSS.
2022-12-21 10:21:21 +01:00
Martin Holst Swende b818e73ef3
tests: update tests (#26314)
This PR builds on #26299, but also updates the tests to the most recent version, which includes tests regarding TheMerge.

This change adds checks to the beacon consensus engine, making it more strict in validating the pre- and post-headers, and not relying on the caller to have already correctly sanitized the headers/blocks.
2022-12-20 09:56:52 -05:00
rjl493456442 79a478bb61
core/rawdb: implement resettable freezer (#26324)
This PR implements resettable freezer by adding a ResettableFreezer wrapper.

The resettable freezer wraps the original freezer in a way that makes it possible to ensure atomic resets. Implementation wise, it relies on the os.Rename and os.RemoveAll to atomically delete the original freezer data and re-create a new one from scratch.
2022-12-19 04:59:12 -05:00
Ikko Ashimine f51f6edb40
core: fix typo in blockchain_snapshot_test.go (#26357)
correspnding -> corresponding
2022-12-14 11:48:12 +02:00
rjl493456442 cda051eba7
core, cmd: fill blockNumber in logs (#26345)
* core, cmd: fill blockNumber in logs

* Update core/state/statedb.go

Co-authored-by: Martin Holst Swende <martin@swende.se>

* core/types: revert

* core/state: improve comments

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-12-13 07:54:16 -05:00
Alex Stokes 250a80a50e
core: fix comment about recipient for contract creation (#26349)
A comment suggests that contract creation happens if the recipient of a call is 0x00..00 ("zero address") but in fact the sender must be nil. The zero address is a regular valid address that is commonly used as a "burn" address.
2022-12-13 03:33:40 -05:00
Felix Lange 3315bad256
core: pass block into collectLogs (#26335)
While investigating another issue, I found that all callers of collectLogs have the
complete block available. rawdb.ReadReceipts loads the block from the database,
so it is better to use ReadRawReceipts here, and derive the receipt information using
the block which is already in memory.
2022-12-09 16:14:33 +01:00
Sina Mahmoodi 711afbc7fd
core,eth: add `debug_setTrieFlushInterval` to change trie flush frequency (#24785)
This PR makes it possible to modify the flush interval time via RPC. On one extreme, `0s`, it would act as an archive node. If set to `1h`, means that after one hour of effective block processing time, the trie would be flushed. If one block takes 200ms, this means that a flush would occur every `5*3600=18000`  blocks -- however, if the memory size of the cached states grows too large, it will flush sooner. 

Essentially, this makes it possible to configure the node to be more or less "archive:ish", and without restarting the node while reconfiguring it.
2022-12-09 07:40:17 -05:00
Sina Mahmoodi 6891288787
core: fix state flushing for catalyst mode (#26319)
The gcproc field tracks the amount of time spent processing blocks,
and is used to trigger a state flush to disk when a certain threshold is
reached. After the merge, single block insertion by CL is the most
common source of block processing time, but this time was not added
into gcproc.
2022-12-07 17:36:54 +01:00
Martin Holst Swende 1f35988a00
eth/tracers, core/vm: remove `time` from trace output and tracing interface (#26291)
This removes the 'time' field from logs, as well as from the tracer interface. This change makes the trace output deterministic.  If a tracer needs the time they can measure it themselves. No need for evm to do this.

Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
2022-12-05 16:57:47 +01:00
Håvard Anda Estensen 06632da2bb
all: assign zero after resize in implementations of heap.Interface (#26296)
This changes the Pop method to assign the zero value before
reducing slice size. Doing so ensures the backing array does not
reference removed item values.
2022-12-05 13:49:54 +01:00
Sina Mahmoodi 10347c6b54
core: drop legacy receipt types (#26225)
This PR drops the legacy receipt types, the freezer-migrate command and the startup check. The previous attempt #22852 at this failed because there were users who still had legacy receipts in their db, so it had to be reverted #23247. Since then we added a command to migrate legacy dbs #24028.

As of the last hardforks all users either must have done the migration, or used the --ignore-legacy-receipts flag which will stop working now.
2022-12-03 21:42:11 +01:00
rjl493456442 743e404906
core, eth, les, tests, trie: abstract node scheme (#25532)
This PR introduces a node scheme abstraction. The interface is only implemented by `hashScheme` at the moment, but will be extended by `pathScheme` very soon.

Apart from that, a few changes are also included which is worth mentioning:

-  port the changes in the stacktrie, tracking the path prefix of nodes during commit
-  use ethdb.Database for constructing trie.Database. This is not necessary right now, but it is required for path-based used to open reverse diff freezer
2022-11-28 14:31:28 +01:00
setunapo 0dc9b01c11
core: clarify code in forkchoice (#26257)
refactoring without logic change
2022-11-28 11:07:25 +01:00
rjl493456442 add1bff13f
core/rawdb: fix freezer validation (#26251)
* core/rawdb: fix freezer validation

* core/rawdb: address comment
2022-11-25 09:10:31 +01:00
Felix Lange 193f350eb9
core/rawdb: improve freezerTable.Sync (#26245)
While investigating #22374, I noticed that the Sync operation of the
freezer does not take the table lock. It also doesn't call sync for all files
if there is an error with one of them. I doubt this will fix anything, but
didn't want to drop the fix on the floor either.
2022-11-24 10:50:28 +01:00
Marius van der Wijden ec2ec2d08e
core: implement EIP-3651, warm coinbase (#25819)
Implements EIP-3651, "Warm Coinbase", for Shanghai hardfork. Specification: https://eips.ethereum.org/EIPS/eip-3651.
2022-11-22 22:39:52 +01:00
Mark Tyneway b4ea2bf7dd
all: implement EIP-1153 transient storage (#26003)
Implements TSTORE and TLOAD as specified by the following EIP:

https://eips.ethereum.org/EIPS/eip-1153
https://ethereum-magicians.org/t/eip-1153-transient-storage-opcodes/553


Co-authored-by: Sara Reynolds <snreynolds2506@gmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2022-11-16 10:18:52 +01:00
Felix Lange 9afc6816d2
common/lru: add generic LRU implementation (#26162)
It seems there is no fully typed library implementation of an LRU cache.
So I wrote one. Method names are the same as github.com/hashicorp/golang-lru,
and the new type can be used as a drop-in replacement.

Two reasons to do this:

- It's much easier to understand what a cache is for when the types are right there.
- Performance: the new implementation is slightly faster and performs zero memory
   allocations in Add when the cache is at capacity. Overall, memory usage of the cache
   is much reduced because keys are values are no longer wrapped in interface.
2022-11-14 15:41:56 +01:00
Martin Holst Swende 5fded04037
core/state: replace fastcache code cache with gc-friendly structure (#26092)
This PR replaces fastcache with a pretty simple LRU which does not require explicit closing.
2022-11-09 08:06:02 +01:00
yihuang 7dc5e785a8
core/vm: deepcopy jumptable when enabling extra eips (#26137)
When the interpreter is configured to use extra-eips, this change makes it so that all the opcodes are deep-copied, to prevent accidental modification of the 'base' jumptable. 

Closes: #26136

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-11-09 07:52:13 +01:00
protolambda a51188a163
params: make eip1559 params non-global (#25994)
This PR changes geth to read the eip1559 params from the chain config instead of the globals.

This way the parameters may be changed by forking the chain config code, without creating a large diff throughout the past and future usages of the parameters.

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-11-04 16:27:15 +01:00
rjl493456442 60e30a940b
core/rawdb: refactor db inspector for extending multiple ancient store (#25896)
This PR ports a few changes from PBSS:

- Fix the snapshot generator waiter in case the generation is not even initialized
- Refactor db inspector for ancient store
2022-10-28 10:23:49 +02:00
Martin Holst Swende 9d795d0836
core/vm: use optimized bigint (#26021) 2022-10-27 10:39:01 +02:00
Péter Szilágyi c4a662176e
core, eth: for types with accurate size calcs, return uint64, not float (#26046)
* core, eth: for types with accurate size calcs, return uint64, not float

* core/types: proper tx size tests

* core/types: extend tx size test with decoded sizes, fix error

* core/txpool: fix linter

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-10-26 15:23:07 +03:00
Péter Szilágyi 68ba845bb5
core/types: rename tx files to group them better together 2022-10-26 13:30:51 +03:00
Péter Szilágyi a6dda03644
all: refactor txpool into it's own package in prep for 4844 2022-10-24 16:35:53 +03:00
rjl493456442 b9ba6f6e4d
core/rawdb: open meta file in read only mode (#26009) 2022-10-19 21:15:43 +02:00
s7v7nislands d86fe26f67
core/rawdb: refactor db key prefix (#26000)
Co-authored-by: seven <seven@nodereal.io>
2022-10-19 09:53:09 +02:00
Martin Holst Swende 6069d8294e
cmd/utils: enable snapshot generation in import-mode (#25990)
This PR fixes a regression causing snapshots not to be generated in "geth --import" mode.  It also fixes the geth export command to be truly readonly, and adds a new test for geth export.
2022-10-19 08:20:39 +02:00
Martin Holst Swende bed3b10086
common/math: optimized modexp (+ fuzzer) (#25525)
This adds a 
* core/vm, tests: optimized modexp + fuzzer

* common/math: modexp optimizations

* core/vm: special case base 1 in big modexp

* core/vm: disable fastexp
2022-10-12 10:34:52 +02:00
Marius van der Wijden a007ab786c
core/types: add more context around ErrInvalidChainID (#25367)
This changes the error message for mismatching chain ID to show
the given and expected value. Callers expecting this error must be
changed to use errors.Is.
2022-10-12 10:27:39 +02:00
Martin Holst Swende 28d076d37e
core/rawdb: provide more info on 'gap in the chain' error (#25938) 2022-10-12 09:35:09 +02:00
Martin Holst Swende 5a02b2d6d0
all: fix spelling mistakes (#25961) 2022-10-11 09:37:00 +02:00
Martin Holst Swende 067bac3f24
core, ethclient/gethclient: improve flaky tests (#25918)
* ethclient/gethclient: improve time-sensitive flaky test

* eth/catalyst: fix (?) flaky test

* core: stop blockchains in tests after use

* core: fix dangling blockchain instances

* core: rm whitespace

* eth/gasprice, eth/tracers, consensus/clique: stop dangling blockchains in tests

* all: address review concerns

* core: goimports

* eth/catalyst: fix another time-sensitive test

* consensus/clique: add snapshot test run function

* core: rename stop() to stopWithoutSaving()

Co-authored-by: Felix Lange <fjl@twurst.com>
2022-10-06 13:39:20 +02:00
Martin Holst Swende deead99731
core/bloombits: speed up windows-test (#25844)
core/bloombits: remove micro-sleep
2022-10-06 11:41:07 +02:00
Martin Holst Swende e14164d516
core: fix linter complaint (#25927) 2022-10-04 18:09:05 +02:00
Péter Szilágyi 4f7a425aa8
Merge pull request #25924 from holiman/comments_fix
all: fix docstrings
2022-10-04 13:30:00 +03:00
Martin Holst Swende ee301c750b
all: fix docstrings 2022-10-04 09:18:02 +02:00
zhiqiangxu a724163e59
core/types: add a testcase to TestLegacyReceiptDecoding (#25909)
add a testcase to TestLegacyReceiptDecoding
2022-10-03 12:56:21 +02:00
rjl493456442 052c634917
core: rework tx indexer (#25723)
This PR reworks tx indexer a bit. Compared to the original version, one scenario is no longer handled - upgrading  from legacy geth without indexer support. 

The tx indexer was introduced in 2020 and have been present through hardforks, so it can be assumed that all Geth nodes have tx indexer already. So we can simplify the tx indexer logic a bit:

-    If the tail flag is not present, it means node is just initialized may or may not with an ancient store attached. In this case all blocks are regarded as unindexed
-   If the tail flag is present, it means blocks below tail are unindexed, blocks above tail are indexed

This change also address some weird cornercases that could make the indexer not work after a crash.
2022-09-29 09:50:24 +02:00
lmittmann 2b1299b1c0
core/vm: minor trivial clean up (#25880)
* dropped `vm.keccakState` for `crypto.KeccakState`

* cleaned up `OpCode.IsPush()`
2022-09-28 13:21:54 +02:00
rjl493456442 bff84a99fe
cmd, core, eth, les, light: track deleted nodes (#25757)
* cmd, core, eth, les, light: track deleted nodes

* trie: add docs

* trie: address comments

* cmd, core, eth, les, light, trie: trie id

* trie: add tests

* trie, core: updates

* trie: fix imports

* trie: add utility print-method for nodeset

* trie: import err

* trie: fix go vet warnings

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-09-27 10:01:02 +02:00
jwasinger 7227c9ef07
cmd/geth: make dumpgenesis load genesis datadir if it exists (#25135)
`geth dumpgenesis` currently does not respect the content of the data directory. Instead, it outputs the genesis block created by command-line flags. This PR fixes it to read the genesis from the database, if the database already exists.


Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-09-26 13:55:18 +02:00
Leon c55c56cf0a
core/vm: better handle error on eip activation check (#25131)
* core/vm: correct logic for eip check of NewEVMInterpreter

* refactor
2022-09-26 13:20:38 +02:00
Martin Holst Swende e004e7d256
core: fix datarace in txpool, fixes #25870 and #25869 (#25872)
core: fix datarace in txpool pendingnoce, fixes #25870
2022-09-26 11:34:15 +02:00
zhiqiangxu 367e60549a
core/types: improve accuracy of header.Size() (#25859)
The header.Size() method did not take the basefee into account.
2022-09-26 09:10:39 +02:00
rjl493456442 3da42f85d9
all: clean up the configs for pruner and snapshotter (#22396)
This PR cleans up the configurations for pruner and snapshotter by passing a config struct.

And also, this PR disables the snapshot background generation if the chain is opened in "read-only" mode. The read-only mode is necessary in some cases. For example, we have a list of commands to open the etheruem node in "read-only" mode, like export-chain. In these cases, the snapshot background generation is non expected and should be banned explicitly.
2022-09-23 20:20:36 +02:00
lightclient 338bb2e36c
core/blockchain: downgrade tx indexing and unindexing logs (#25832)
core/blockchain: downgrade tx indexing and unindexing logs from info to debug

If a user has a finite tx lookup limit, they will see an "unindexing" info level log each time a block is imported. This information might help a user understand that they are removing the index each block and some txs may not be retrievable by hash, but overall it is generally more of a nuisance than a benefit. This change downgrades the log to a debug log.
2022-09-21 09:49:17 +02:00
jwasinger d8ada03eac
core/vm: return copy of input slice in identity precompile, avoid returndata copy (#25183)
* core/vm: return copy of input slice in identity precompile.  don't deep copy return data slice upon call completion

* make use of common.CopyBytes
2022-09-20 14:58:03 +02:00
Felix Lange 468d1844c7
core: move build version reading to its own package (#25806)
This fixes the build with Go 1.17, which does not have BuildInfo.Settings yet.
2022-09-19 10:04:16 +02:00
lightclient 8e5201551d
core: more compact bad block report (#25714)
This shortens the chain config summary in bad block reports,
and adds go-ethereum version information as well.

Co-authored-by: Felix Lange <fjl@twurst.com>
2022-09-17 21:27:10 +02:00
Seungbae Yu ada603fab5
core: don't cache zero nonce in txNoncer (#25603)
This changes the nonce cache used by TxPool to not store cached
nonces for non-existing accounts.
2022-09-16 18:23:13 +02:00
ucwong 83989a19be
core/rawdb: fix leak of backoff timer (#25776) 2022-09-16 11:33:48 +02:00
Felix Lange 0ee8b273f2
core: fix log order in "Imported new chain segment" (#25768) 2022-09-15 13:18:23 +02:00
Seungbae Yu 3a4cef5402
core: preallocate maps in TxPool helper methods (#25737) 2022-09-12 15:02:41 +02:00
Felix Lange b628d72766
build: upgrade to go 1.19 (#25726)
This changes the CI / release builds to use the latest Go version. It also
upgrades golangci-lint to a newer version compatible with Go 1.19.

In Go 1.19, godoc has gained official support for links and lists. The
syntax for code blocks in doc comments has changed and now requires a
leading tab character. gofmt adapts comments to the new syntax
automatically, so there are a lot of comment re-formatting changes in this
PR. We need to apply the new format in order to pass the CI lint stage with
Go 1.19.

With the linter upgrade, I have decided to disable 'gosec' - it produces
too many false-positive warnings. The 'deadcode' and 'varcheck' linters
have also been removed because golangci-lint warns about them being
unmaintained. 'unused' provides similar coverage and we already have it
enabled, so we don't lose much with this change.
2022-09-10 13:25:40 +02:00
Martin Holst Swende 389021a5af
core: place a cap on reorglogs (#25711)
This PR makes the event-sending for deleted and new logs happen in batches, to prevent OOM situation due to large reorgs. 

Co-authored-by: Felix Lange <fjl@twurst.com>
2022-09-09 15:25:55 +02:00
rjl493456442 dea1fb3cfc
all: cleanup tests (#25641)
Follow-up to PR #25523 to cleanup all relevant tests.
2022-09-07 20:21:59 +02:00
Marius van der Wijden d30e39b2f8
core/vm: rename RANDOM to PREVRANDAO (#25691)
See: https://github.com/ethereum/EIPs/blob/master/EIPS/eip-4399.md

>  This EIP supplants the semantics of the return value of existing DIFFICULTY (0x44) opcode and renames the opcode to PREVRANDAO (0x44).
2022-09-07 13:09:52 +02:00
rjl493456442 5ddedd2f83
core, light, trie: remove DiskDB function from trie database (#25690) 2022-09-07 09:08:56 +02:00
rjl493456442 d46184c969
core/state: filter out nil trie for copy (#25575) 2022-09-01 09:34:32 +02:00
rjl493456442 d10c280309
all: move genesis initialization to blockchain (#25523)
* all: move genesis initialization to blockchain

* core: add one more check

* core: fix tests
2022-08-30 18:22:28 +02:00
Abirdcfly c394c308e6
all: remove duplicate word in comments (#25618)
Signed-off-by: Abirdcfly <fp544037857@gmail.com>

Signed-off-by: Abirdcfly <fp544037857@gmail.com>
2022-08-29 11:16:34 +03:00
Péter Szilágyi 5758d1fb11
core/state, trie: fix trie flush order for proper pruning 2022-08-23 21:17:12 +03:00
Péter Szilágyi 81bd998353
core, eth/downloader: handle spurious junk bodies from racey rollbacks (#25578)
* eth/downloader: handle junkbodies/receipts in the beacon sync

* core: check for header presence when checking for blocks
2022-08-23 14:02:51 +03:00
Justin Traglia 2c5648d891
all: fix some typos (#25551)
* Fix some typos

* Fix some mistakes

* Revert 4byte.json

* Fix an incorrect fix

* Change files to fails
2022-08-19 09:00:21 +03:00
Guillaume Ballet 23ac8df153
cmd. core: save preimages on genesis creation (#25538)
force preimage dump for genesis
2022-08-18 13:28:06 +02:00
Seungbae Yu a50c006b49
core: make tx journal check and open atomic (#25530)
* core: reduce system call about `os`

* avoid deprecated method
2022-08-17 14:16:18 +03:00
Guillaume Ballet 6da5c1644d
core/state, trie, light: add a TryDeleteAccount method (#25531)
* core/state, trie, light: Add a DeleteAccount method

* review feedback

* Update database.go

* pr triage feedback

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
2022-08-17 14:14:49 +03:00
Guillaume Ballet 12185e40e0
core, trie: flush preimages to db on blockchain close (#25533)
* core, trie: flush preimages to db on database close

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>

* rename Close to CommitPreimages for clarity

* core, trie: nitpick fixes

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2022-08-17 14:12:10 +03:00
ycyraum 141cd42531
core/genesis: remove calaverasAllocData (#25516)
core/genesis: calaverasAllocData no longer used
2022-08-12 13:58:06 +02:00
ycyraum 877ef7f09e
core: remove unused bc ChainContext in applyTransaction 2022-08-10 08:02:37 +02:00
rjl493456442 a41ea8a97c
all: cleanup the APIs for initializing genesis (#25473)
* all: polish tests

* core: apply feedback from Guillaume

* core: fix comment
2022-08-09 12:44:39 +03:00
Sina Mahmoodi 86de2e516e
eth/tracers: add onlyTopCall option to callTracer (#25430)
This PR allows users to pass in a config object directly to the tracers. Previously only the struct logger was configurable.

It also adds an option to the call tracer which if enabled makes it ignore any subcall and collect only information about the top-level call. See #25419 for discussion.

The tracers will silently ignore if they are passed a config they don't care about.
2022-08-09 11:04:57 +02:00
int88 e4b3bd6f26
core: fix uncle creation in TestFastVsFullChains (#25476)
Co-authored-by: Felix Lange <fjl@twurst.com>
2022-08-09 00:20:46 +02:00
rjl493456442 e44d6551c3
cmd, core, ethdb, node: move chain freezer one folder deeper (#25487)
* cmd, core, ethdb, node: create chain freezer in a sub folder

* core/rawdb: remove unused code

* core, ethdb, node: add AncientDatadir API back

* cmd, core: extend freezer info dump for sub-ancient-store

* core/rawdb: rework freezer inspector

* core/rawdb: address comments from Peter

* core/rawdb: fix build issue
2022-08-08 12:08:36 +03:00
Guillaume Ballet f67e54c92f
core: use TryGetAccount to read what TryUpdateAccount has written (#25458)
* core: use TryGetAccount to read where TryUpdateAccount has been used to write

* Gary's review feedback

* implement Gary's suggestion

* fix bug + rename NewSecure into NewStateTrie

* trie: add backwards-compatibility aliases for SecureTrie

* Update database.go

* make the linter happy

Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
2022-08-04 16:13:18 +02:00
rjl493456442 8b53b92eb4
core, trie: rework trie committer (#25320)
* all: rework trie and trie committer

* all: get rid of internal cache in trie

* all: fixes

* trie: polish

* core, trie: address comments

* trie: fix imports

* core/state: address comments

* core/state/snapshot: polish

* trie: remove unused code

* trie: update tests

* trie: don't set db as nil

* trie: address comments

* trie: unskip test
2022-08-04 11:03:20 +03:00
Seungbae Yu 5fb463dddc
core: preallocate batch size in bloomIndexer (#25289)
This change reduces allocations when committing bloombits indexes
by creating the database batch with a larger initial size.
2022-08-03 17:02:09 +02:00
yong 6b6261b51f
core/types: fix typo in comment (#25359) 2022-08-03 16:56:57 +02:00
Péter Szilágyi 6fd06ab075
cmd, core, eth, les, params: add merge-passed chain config (#24538)
* cmd, core, eth, les, params: add merge-passed chain config

* eth/catalyst, params: add various warning on malfunctioning beacons

* eth/catalyst: fix warning for beacons without transition exchanges
2022-08-01 15:13:25 +03:00
rjl493456442 9d76a9b94f
core, trie, eth, cmd: rework preimage store (#25287)
* core, trie, eth, cmd: rework preimage store

* trie: address comment
2022-07-27 20:37:04 +02:00
Marius van der Wijden c6dcd018d2
core: eth: rpc: implement safe rpc block (#25165)
* core: eth: rpc: implement safe rpc block

* core: fix setHead, panics
2022-07-25 18:42:05 +03:00
Péter Szilágyi f3af3fd8df
eth: support bubbling up bad blocks from sync to the engine API (#25190)
* eth: support bubbling up bad blocks from sync to the engine API

* eth/catalyst: fix typo

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>

* eth/catalyst: fix typo

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>

* Update eth/catalyst/api.go

* eth/catalyst: when forgetting bad hashes, also forget descendants

* eth/catalyst: minor bad block tweaks for resilience

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-07-25 16:51:04 +03:00
Delweng b196ad1c16
all: add whitespace linter (#25312)
* golangci: typo

Signed-off-by: Delweng <delweng@gmail.com>

* golangci: add whietspace

Signed-off-by: Delweng <delweng@gmail.com>

* *: rm whitesapce using golangci-lint

Signed-off-by: Delweng <delweng@gmail.com>

* cmd/puppeth: revert accidental resurrection

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2022-07-25 13:14:03 +03:00
Marius van der Wijden 22d71afc95
params: set sepolia mergeNetsplitBlock to 1735371 (#25372) 2022-07-23 09:54:11 +02:00
rjl493456442 1657e43931
core, les, eth: port snap sync changes (#24898)
core, eth, les, trie: rework snap sync
2022-07-15 14:55:51 +03:00
Lee Bousfield 1c9afc56ae
core: prevent negative fee during RPC calls (#25214)
During RPC calls such as eth_call and eth_estimateGas, st.evm.Config.NoBaseFee is set
which allows the gas price to be below the base fee. This results the tip being negative,
and balance being subtracted from the coinbase instead of added to it, which results in a
potentially negative coinbase balance interestingly. This can't happen during normal chain
processing as outside of RPC calls the gas price is required to be at least the base fee,
as NoBaseFee is false.

This change prevents this behavior by disabling fee payment when the fee is not set.

Co-authored-by: lightclient@protonmail.com <lightclient@protonmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
2022-07-15 11:32:54 +02:00
henridf 4766b1107f
core: remove lock in BlockChain.ExportN (#25254)
* Remove locking in (*BlockChain).ExportN

Since ExportN is read-only, it shouldn't need the lock. (?)

* Add hash check to detect reorgs during export.

* fix check order

* Update blockchain.go

* Update blockchain.go

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
2022-07-14 15:55:54 +03:00
Seungbae.yu 44893be0d6
core, eth: pre-allocate map in storage copy (#25279) 2022-07-12 10:08:45 +03:00
Seungbae.yu e394d01f2a
core/types: fix typo in comment (#25249) 2022-07-06 15:16:05 +02:00
Marius van der Wijden 87bb5db675
core: allow external processor (#25233) 2022-07-05 10:02:49 +03:00
Marius van der Wijden 5f6e870ee6
core: apply ttd override to uninitialized db (#25136)
* core: apply ttd override to genesis block

* core: apply overrides properly
2022-07-04 12:25:17 +03:00
Marius van der Wijden e537193421
core: allow external code to set the block validator for malicious tests (#25119)
* core: don't validate state

* core: allow external validator

* core: revert

* core: comments

* Update blockchain_reader.go

* core: move SetValidator to blockchain.go

* core: rename method
2022-07-04 12:24:06 +03:00
Seungbae.yu 2697e44d81
all: change format `0x%x` to `%#x` (#25221) 2022-07-04 11:03:32 +03:00
Marius van der Wijden d12b1a91cd
consensus/beacon: check that only the latest pow block is valid ttd block (#25187)
* consensus/beacon: check that only the latest pow block is valid ttd block

* consensus/beacon: move verification to async function

* consensus/beacon: fix verifyTerminalPoWBlock, add test cases

* consensus/beacon: cosmetic changes

* consensus/beacon: apply karalabe's fixes
2022-06-29 15:13:19 +03:00
aaronbuchwald 300f6121ad
core/rawdb: simplify TestDiskSeek to use memorydb (#25182) 2022-06-29 11:54:42 +02:00
Ruohui Wang 3f712e7447
core/rawdb: fix typo in comment (#25191) 2022-06-29 11:47:33 +02:00
aaronbuchwald 68420e1aa5
core: fix typo in txpool (#25149)
Fix typo in txPool truncateQueue comment
2022-06-22 22:51:45 +02:00
Marius van der Wijden d8f963811d
cmd, params: implement Gray Glacier hard-fork (EIP-5133) (#25088)
* cmd/geth, params: implement Gray Glacier (EIP-5133)

* cmd/evm: add gray glacier tests

* params: nitpicks

* params: fixes
2022-06-15 14:10:38 +03:00
lmittmann bc013bc42e
all: prefer `new(big.Int)` over `big.NewInt(0)` (#25087)
minor performance improvement: `big.NewInt(0).Xxx` -> `new(big.Int).Xxx`
2022-06-14 15:09:48 +03:00
Martin Holst Swende a907d7e81a
all: more linters (#24783)
This enables the following linters

- typecheck
- unused
- staticcheck
- bidichk
- durationcheck
- exportloopref
- gosec

WIth a few exceptions.

- We use a deprecated protobuf in trezor. I didn't want to mess with that, since I cannot meaningfully test any changes there.
- The deprecated TypeMux is used in a few places still, so the warning for it is silenced for now.
- Using string type in context.WithValue is apparently wrong, one should use a custom type, to prevent collisions between different places in the hierarchy of callers. That should be fixed at some point, but may require some attention.
- The warnings for using weak random generator are squashed, since we use a lot of random without need for cryptographic guarantees.
2022-06-13 16:24:45 +02:00
Gustavo Silva eb94896270
Chore: Minimal gramatical errors (signleton -> singleton) (#25057)
core: fix typos
2022-06-10 18:47:06 +03:00
int88 594e321662
core/evm: fix error in comment (#25040)
Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-06-08 18:40:37 +02:00
henridf f5037185aa
core/types: improve LogForStorage and ReceiptForStorage comments (#25032)
Co-authored-by: Felix Lange <fjl@twurst.com>
2022-06-08 18:32:07 +02:00
Rajaram Gaunker 6160296445
core/types: remove unused field 'td' in Block (#25010) 2022-06-08 18:31:43 +02:00
Péter Szilágyi 41e75480df
eth, les, params: log chain config a bit saner (#24904)
Previously on Geth startup we just logged the chain config is a semi-json-y format. Whilst that worked while we had a handful of hard-forks defined, currently it's kind of unwieldy. 

This PR converts that original data dump and converts it into a user friendly - alas multiline - log output.
2022-06-07 12:49:07 +02:00
rjl493456442 22defa5af7
all: introduce trie owner notion (#24750)
* cmd, core/state, light, trie, eth: add trie owner notion

* all: refactor

* tests: fix goimports

* core/state/snapshot: fix ineffasigns

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-06-06 17:14:55 +02:00
Martin Holst Swende c375ee91e9
cmd/geth, core/state/snapshot: rework journal loading, implement account-check (#24765)
* cmd/geth, core/state/snapshot: rework journal loading, implement account-check

* core/state/snapshot, cmd/geth: polish code (#37)

* core/state/snapshot: minor nits

* core/state/snapshot: simplify error logic

* cmd/geth: go format

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
2022-06-06 17:09:39 +02:00
Paweł Bylica 997f1c4f0a
core/vm: optimize jumpdest analysis (#23500)
core/vm: optimize PUSH opcode discrimination
2022-06-03 10:40:14 +02:00
Martin Holst Swende d8a2305565
eth/tracers: add support for block overrides in debug_traceCall (#24871)
This PR adds support for block overrides when doing debug_traceCall.

- Previously, debug_traceCall against pending erroneously used a common.Hash{} stateroot when looking up the state, meaning that a totally empty state was used -- so it always failed,
- With this change, we reject executing debug_traceCall against pending.
- And we add ability to override all evm-visible header fields.
2022-06-02 11:39:36 +02:00
rjl493456442 f9806dc872
core: fix canonical hash marker update (#24996)
* core: fix reorg

* core: revert change for memory efficiency

* core: revert changes
2022-06-01 11:03:24 +02:00
Marius van der Wijden 8c0c0434c9
core/beacon: prevent invalid logsBloom length panic (#24946)
* core/beacon: prevent invalid logsBloom length panic

* core/beacon: prevent negative baseFeePerGas

* Update core/beacon/types.go

Co-authored-by: Martin Holst Swende <martin@swende.se>

* eth/catalys: go format

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-05-31 11:11:50 +02:00
Marius van der Wijden 86af788790
core: use less memory during reorgs (#24616)
This PR significantly reduces the memory consumption of a long reorg
2022-05-30 08:42:06 +02:00
Felix Lange 9244d5cd61
all: update license headers and AUTHORS from git history (#24947) 2022-05-24 20:39:40 +02:00
Péter Szilágyi 64d6c787b3
Merge pull request #24867 from holiman/no_preop_memexpand
core/vm: for tracing, do not report post-op memory
2022-05-24 11:53:11 +03:00
rjl493456442 59ac229f87
core/state/snapshot: detect and clean up dangling storage snapshot in generation (#24811)
* core/state/snapshot: check dangling storages when generating snapshot

* core/state/snapshot: polish

* core/state/snapshot: wipe the last part of the dangling storages

* core/state/snapshot: fix and add tests

* core/state/snapshot: fix comment

* README: remove mentions of fast sync (#24656)

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>

* core, cmd: expose dangling storage detector for wider usage

* core/state/snapshot: rename variable

* core, ethdb: use global iterators for snapshot generation

* core/state/snapshot: polish

* cmd, core/state/snapshot: polish

* core/state/snapshot: polish

* Update core/state/snapshot/generate.go

Co-authored-by: Martin Holst Swende <martin@swende.se>

* ethdb: extend db test suite and fix memorydb iterator

* ethdb/dbtest: rollback changes

* ethdb/memorydb: simplify iteration

* core/state/snapshot: update dangling counter

* core/state/snapshot: release iterators

* core/state/snapshot: update metrics

* core/state/snapshot: update time metrics

* metrics/influxdb: temp solution to present counter meaningfully, remove it

* add debug log, revert later

* core/state/snapshot: fix iterator panic

* all: customized snapshot iterator for backward iteration

* core, ethdb: polish

* core/state/snapshot: remove debug log

* core/state/snapshot: address comments from peter

* core/state/snapshot: reopen the iterator at the next position

* ethdb, core/state/snapshot: address comment from peter

* core/state/snapshot: reopen exhausted iterators

Co-authored-by: Tbnoapi <63448616+nuoomnoy02@users.noreply.github.com>
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-05-23 13:26:22 +03:00
Péter Szilágyi 34bda5eae3
core, eth, les, rpc: polish catalyst errors, add context 2022-05-20 12:15:54 +03:00
Marius van der Wijden 4a4d531052
eth/catalyst: update implementation to spec (#24802)
* eth/catalyst: return invalid payload attributes error

* eth/catalyst: implement LVH as specified, add tests

* eth/catalyst: return current block hash not header hash

* eth/catalyst: fix test

* eth/catalyst: bring error codes in line with spec
2022-05-20 11:12:12 +03:00
zhaochonghe 2bfd9a28d1
core: fix the order of address in queue (#24907)
reverse the order of address in queue
2022-05-19 15:25:22 +08:00
Sina Mahmoodi bf693228a3
eth/tracers/js: goja tracer (#23773)
This adds a JS tracer runtime environment based on the Goja VM. The new
runtime replaces the duktape runtime, which will be removed soon.

Goja is implemented in Go and is faster for cases where the Go <-> JS
transition overhead dominates overall performance. It is faster because
duktape is written in C, and the transition cost includes the cost of using
cgo. Another reason for using Goja is that go-duktape is not maintained
anymore.

We expect the performace of JS tracing to be at least as good or better with
this change.
2022-05-18 16:34:18 +02:00
Marius van der Wijden e6fa102eb0
core, eth, internal, rpc: implement final block (#24282)
* eth: core: implement finalized block

* eth/catalyst: fix final block

* eth/catalyst: update finalized head gauge

* internal/jsre/deps: updated web3.js to allow for finalized block

* eth/catalyst: make sure only one thread can call fcu

* eth/catalyst: nitpicks

* eth/catalyst: use plain mutex

* eth: nitpicks
2022-05-18 17:30:42 +03:00
Håvard Anda Estensen 4b309c7006
all: replace non-trivial uses of package ioutil with os (#24886)
Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-05-17 14:27:04 +02:00
Håvard Anda Estensen e644d45c14
all: use strings.EqualFold for string comparison (#24890) 2022-05-17 13:56:52 +02:00
Marius van der Wijden 381c66caf0
eth/catalyst: set the correct LatestValidHash (#24855)
* eth/catalyst: set the correct LatestValidHash

* eth/catalyst: core: return LVH during reorg, rework invalid teminal block

* eth/catalyst: nitpicks
2022-05-17 12:32:55 +03:00
Qian Bin fe5a26733c
core/vm: reduce overhead in instructions-benchmark (#24860)
* core/vm: reduce footprint of OP benchmark

* core/vm: for opBenchmark, add code to detect inputs mutation

* Update core/vm/instructions_test.go

Co-authored-by: Martin Holst Swende <martin@swende.se>

* core/vm: opBenchmark, stop timer before sanity-test code

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-05-17 09:21:43 +02:00
Håvard Anda Estensen 07508ac0e9
all: replace uses of ioutil with io and os (#24869) 2022-05-16 11:59:35 +02:00
s7v7nislands 330e53fbb9
core/asm: use strings.Builder and fix godoc issues (#24861) 2022-05-16 11:39:07 +02:00
Martin Holst Swende 97f308a98f
core/vm: for tracing, do not report post-op memory 2022-05-12 21:20:40 +02:00
Qian Bin e0a1fd5fdc
core/vm: optimize Memory.Set32 (#24847)
* core/vm: remove unnecessary memset for Memory.Set32

* core/vm: optimize Memory.Set32
2022-05-11 11:00:29 +02:00
s7v7nislands 539bbd6349
core/vm: separate opcode group for 0x20 range (#24850) 2022-05-11 08:04:16 +03:00
s7v7nislands 24c590cbec
core/vm: clean up some dead functions (#24851) 2022-05-11 08:03:35 +03:00
aaronbuchwald 52eb87d87c
core/vm: update benchmark to use Errorf instead of Sprintf (#24845) 2022-05-10 17:26:48 +03:00
Marius van der Wijden 86d5477079
core/state/snapshot: fix race condition (#24685)
Fixes three race conditions found through fuzzing by David Theodore
2022-05-06 17:20:41 +02:00
Alex Beregszaszi 3b967d16ca
core/vm: implement EIP-3855: PUSH0 instruction (#24039)
* core/vm: Implement PUSH0

* Move PUSH0 to enable3855

* Add method doc
2022-05-06 14:36:45 +02:00
rjl493456442 1941c5e6c9
core/rawdb: untie freezer and ancient chain data (#24684)
Previously freezer has only been used for storing ancient chain data, while obviously it can be used more. This PR unties the chain data and freezer, keep the minimal freezer structure and move all other logic (like incrementally freezing block data) into a separate structure called ChainFreezer.

This PR also extends the database interface by adding a new ancient store function AncientDatadir which can return the root directory of ancient store. The ancient root directory can be used when we want to open some other ancient-stores (e.g. reverse diff freezer).
2022-05-06 13:28:42 +02:00
rjl493456442 7f6f01d46f
core: recover state when beacon sets canonical head if it's missing (#24613)
* core: recover the state in SetChainHead if the head state is missing

* core: disable test logging

* core: address comment from martin

* core: improve log level in case state is recovered

* core, eth, les, light: rename SetChainHead to SetCanonical
2022-05-05 10:36:26 +03:00
milesvant 11b56ace2a
core: fix WriteBlockAndSetHead documentation (#24818) 2022-05-05 10:22:00 +03:00
s7v7nislands 5a584c2133
all: use common.FileExist for checking file existence (#24748) 2022-04-27 11:48:02 +02:00
tia-99 1f5943e4f9
core/types: fix unhandled errors in TestTransactionCoding (#24692) 2022-04-27 10:56:50 +02:00
Martin Holst Swende 0914234d10
cmd/geth, core/state/snapshot: fix flaw in dangling-storage check + inspect difflayers (#24677)
This PR fixes the flaw that @rjl493456442 found in https://github.com/ethereum/go-ethereum/pull/#issuecomment-1093817551 , namely, that the snapshot iterator uses the combined (disk + difflayers) 'view', wheres the raw iterator uses only the disk 'view'.

This PR instead splits up the work: one phase is iterating the disk layer data, another phase is loading the journalled difflayers and performing the same check there.
2022-04-26 09:08:43 +02:00
s7v7nislands 4024c1e869
fix typo (#24731) 2022-04-25 09:28:03 +02:00
Enrique Ortiz 637cf34ded
core/vm: fix typo (#24714)
Was just browsing the code and found this.
2022-04-20 16:13:47 +02:00
ucwong d4d288e3f1
build: add imports for go generate tools (#24682)
This adds a tools.go file to import all command packages used for
go:generate. Doing so makes it possible to execute go-based code
generators using 'go run', locking in the tool version using go.mod.

Co-authored-by: Felix Lange <fjl@twurst.com>
2022-04-12 20:24:02 +02:00
rjl493456442 195c979168
core: fix benchmark panic (#24657)
This PR fixes a few panics in the chain marker benchmarks. The root
cause for panic is in chain marker the genesis header/block is not
accessible, while it's expected to be obtained in tests. So this PR
avoids touching genesis header at all to avoid panic.
2022-04-12 11:34:07 +03:00
Felix Lange 59f0e8ae60
core/types: make "miner" optional in Header JSON (#24666)
"miner" is not set for pending block responses in some cases.

Fixes #24632
2022-04-11 21:19:13 +03:00
Eng Zer Jun 8d066f1f42
all: use T.TempDir to create temporary test directories (#24633)
This commit replaces ioutil.TempDir with t.TempDir in tests. The
directory created by t.TempDir is automatically removed when the test
and all its subtests complete.

Prior to this commit, temporary directory created using ioutil.TempDir
had to be removed manually by calling os.RemoveAll, which is omitted in
some tests. The error handling boilerplate e.g.

	defer func() {
		if err := os.RemoveAll(dir); err != nil {
			t.Fatal(err)
		}
	}

is also tedious, but t.TempDir handles this for us nicely.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-04-08 15:44:55 +02:00
Péter Szilágyi f6891ba40d
core/state/snapshot: remove noop map item assignment 2022-04-07 09:36:44 +03:00
Sina Mahmoodi 3fd16af5a9
core,eth: implement tx-level hooks for tracers (#24510)
* core,eth: add empty tx logger hooks

* core,eth: add initial and remaining gas to tx hooks

* store tx gasLimit in js tracer

* use gasLimit to compute intrinsic cost for js tracer

* re-use rules in transitiondb

* rm logs

* rm logs

* Mv some fields from Start to TxStart

* simplify sender lookup in prestate tracer

* mv env to TxStart

* Revert "mv env to TxStart"

This reverts commit 656939634b9aff19f55a1cd167345faf8b1ec310.

* Revert "simplify sender lookup in prestate tracer"

This reverts commit ab65bce48007cab99e68232e7aac2fe008338d50.

* Revert "Mv some fields from Start to TxStart"

This reverts commit aa50d3d9b2559addc80df966111ef5fb5d0c1b6b.

* fix intrinsic gas for prestate tracer

* add comments

* refactor

* fix test case

* simplify consumedGas calc in prestate tracer
2022-03-31 11:51:44 +02:00
rjl493456442 da16d089c0
trie, les, tests, core: implement trie tracer (#24403)
Trie tracer is an auxiliary tool to capture all deleted nodes
which can't be captured by trie.Committer. The deleted nodes
can be removed from the disk later.
2022-03-31 09:28:32 +02:00
Guruprasad Kamath 8cacb42278
core: verify genesis extradata for clique (#24470)
* Add extra-data checks for clique genesis

* Update genesis.go

* Update genesis.go

* core: simplify clique genesis check

Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-03-30 10:27:25 +02:00
rjl493456442 25bd17d725
core/state/snapshot: clean up the generation code (#24479) 2022-03-29 20:37:35 +02:00
Péter Szilágyi 1027cb52c4
Merge pull request #24605 from karalabe/revert-setting-ttd
core: set ttd override on all chain variations
2022-03-29 11:39:45 +03:00
Zhang Zhuo b06e8c4a8a
core/vm: fix sstore gas comment type (#24583) 2022-03-29 11:21:11 +03:00
Péter Szilágyi b45d82e94a
core: set ttd override on all chain variations 2022-03-29 11:03:01 +03:00
Sina Mahmoodi fd4f60f49b
core/rawdb: simple legacy receipt converter (#24028)
* cmd,core: add simple legacy receipt converter

core/rawdb: use forEach in migrate

core/rawdb: batch reads in forEach

core/rawdb: make forEach anonymous fn

cmd/geth: check for legacy receipts on node startup

fix err msg

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>

fix log

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>

fix some review comments

add warning to cmd

drop isLegacy fn from migrateTable params

add test for windows rename

test replacing in windows case

* minor fix

* sanity check for tail-deletion

* add log before moving files around

* speed-up hack for mainnet

* fix mainnet check, use networkid instead

* check mainnet genesis

* review fixes

* resume previous migration attempt

* core/rawdb: lint fix

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-03-23 20:57:32 +01:00
rjl493456442 e0e8bf31c5
eth/downloader: ignore zero size header batch for importing (#24569)
* eth/downloader: ignore zero size header batch for importing

* core, light: reject empty header batch for importing
2022-03-22 18:58:05 +01:00
rjl493456442 7ae6c4a790
core: store genesis allocation and recommit them if necessary (#24460)
* core: store genesis allocation and recommit them if necessary

* core: recover predefined genesis allocation if possible
2022-03-22 10:53:22 +01:00
rjl493456442 6afb717be5
core/rawdb: fix db commands (#24540) 2022-03-22 10:19:04 +01:00
Marius van der Wijden 667e1c038e
core, params: add kiln flag (#24548) 2022-03-17 17:34:22 +02:00
Marius van der Wijden 4f4622bc8b
core: eth: implement Kiln-v2 spec (#24506)
* core/beacon: eth/catalyst: updated engine api to new version

* core: implement exchangeTransitionConfig

* core/beacon: prevRandao instead of Random

* eth/catalyst: Fix ExchangeTransitionConfig, add test

* eth/catalyst: stop external miners on TTD reached

* node: implement --authrpc.vhosts flag

* core: allow for config override on non-mainnet networks

* eth/catalyst: fix peters comments

* eth/catalyst: make stop remote sealer more explicit

* eth/catalyst: add log output

* cmd/utils: rename authrpc.host to authrpc.addr

* eth/catalyst: disable the disabling of the miner

* eth: core: remove notion of terminal pow block

* eth: les: more of peters nitpicks
2022-03-17 17:20:03 +02:00