Commit Graph

2069 Commits

Author SHA1 Message Date
s7v7nislands 4024c1e869
fix typo (#24731) 2022-04-25 09:28:03 +02:00
Enrique Ortiz 637cf34ded
core/vm: fix typo (#24714)
Was just browsing the code and found this.
2022-04-20 16:13:47 +02:00
ucwong d4d288e3f1
build: add imports for go generate tools (#24682)
This adds a tools.go file to import all command packages used for
go:generate. Doing so makes it possible to execute go-based code
generators using 'go run', locking in the tool version using go.mod.

Co-authored-by: Felix Lange <fjl@twurst.com>
2022-04-12 20:24:02 +02:00
rjl493456442 195c979168
core: fix benchmark panic (#24657)
This PR fixes a few panics in the chain marker benchmarks. The root
cause for panic is in chain marker the genesis header/block is not
accessible, while it's expected to be obtained in tests. So this PR
avoids touching genesis header at all to avoid panic.
2022-04-12 11:34:07 +03:00
Felix Lange 59f0e8ae60
core/types: make "miner" optional in Header JSON (#24666)
"miner" is not set for pending block responses in some cases.

Fixes #24632
2022-04-11 21:19:13 +03:00
Eng Zer Jun 8d066f1f42
all: use T.TempDir to create temporary test directories (#24633)
This commit replaces ioutil.TempDir with t.TempDir in tests. The
directory created by t.TempDir is automatically removed when the test
and all its subtests complete.

Prior to this commit, temporary directory created using ioutil.TempDir
had to be removed manually by calling os.RemoveAll, which is omitted in
some tests. The error handling boilerplate e.g.

	defer func() {
		if err := os.RemoveAll(dir); err != nil {
			t.Fatal(err)
		}
	}

is also tedious, but t.TempDir handles this for us nicely.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-04-08 15:44:55 +02:00
Péter Szilágyi f6891ba40d
core/state/snapshot: remove noop map item assignment 2022-04-07 09:36:44 +03:00
Sina Mahmoodi 3fd16af5a9
core,eth: implement tx-level hooks for tracers (#24510)
* core,eth: add empty tx logger hooks

* core,eth: add initial and remaining gas to tx hooks

* store tx gasLimit in js tracer

* use gasLimit to compute intrinsic cost for js tracer

* re-use rules in transitiondb

* rm logs

* rm logs

* Mv some fields from Start to TxStart

* simplify sender lookup in prestate tracer

* mv env to TxStart

* Revert "mv env to TxStart"

This reverts commit 656939634b9aff19f55a1cd167345faf8b1ec310.

* Revert "simplify sender lookup in prestate tracer"

This reverts commit ab65bce48007cab99e68232e7aac2fe008338d50.

* Revert "Mv some fields from Start to TxStart"

This reverts commit aa50d3d9b2559addc80df966111ef5fb5d0c1b6b.

* fix intrinsic gas for prestate tracer

* add comments

* refactor

* fix test case

* simplify consumedGas calc in prestate tracer
2022-03-31 11:51:44 +02:00
rjl493456442 da16d089c0
trie, les, tests, core: implement trie tracer (#24403)
Trie tracer is an auxiliary tool to capture all deleted nodes
which can't be captured by trie.Committer. The deleted nodes
can be removed from the disk later.
2022-03-31 09:28:32 +02:00
Guruprasad Kamath 8cacb42278
core: verify genesis extradata for clique (#24470)
* Add extra-data checks for clique genesis

* Update genesis.go

* Update genesis.go

* core: simplify clique genesis check

Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-03-30 10:27:25 +02:00
rjl493456442 25bd17d725
core/state/snapshot: clean up the generation code (#24479) 2022-03-29 20:37:35 +02:00
Péter Szilágyi 1027cb52c4
Merge pull request #24605 from karalabe/revert-setting-ttd
core: set ttd override on all chain variations
2022-03-29 11:39:45 +03:00
Zhang Zhuo b06e8c4a8a
core/vm: fix sstore gas comment type (#24583) 2022-03-29 11:21:11 +03:00
Péter Szilágyi b45d82e94a
core: set ttd override on all chain variations 2022-03-29 11:03:01 +03:00
Sina Mahmoodi fd4f60f49b
core/rawdb: simple legacy receipt converter (#24028)
* cmd,core: add simple legacy receipt converter

core/rawdb: use forEach in migrate

core/rawdb: batch reads in forEach

core/rawdb: make forEach anonymous fn

cmd/geth: check for legacy receipts on node startup

fix err msg

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>

fix log

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>

fix some review comments

add warning to cmd

drop isLegacy fn from migrateTable params

add test for windows rename

test replacing in windows case

* minor fix

* sanity check for tail-deletion

* add log before moving files around

* speed-up hack for mainnet

* fix mainnet check, use networkid instead

* check mainnet genesis

* review fixes

* resume previous migration attempt

* core/rawdb: lint fix

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-03-23 20:57:32 +01:00
rjl493456442 e0e8bf31c5
eth/downloader: ignore zero size header batch for importing (#24569)
* eth/downloader: ignore zero size header batch for importing

* core, light: reject empty header batch for importing
2022-03-22 18:58:05 +01:00
rjl493456442 7ae6c4a790
core: store genesis allocation and recommit them if necessary (#24460)
* core: store genesis allocation and recommit them if necessary

* core: recover predefined genesis allocation if possible
2022-03-22 10:53:22 +01:00
rjl493456442 6afb717be5
core/rawdb: fix db commands (#24540) 2022-03-22 10:19:04 +01:00
Marius van der Wijden 667e1c038e
core, params: add kiln flag (#24548) 2022-03-17 17:34:22 +02:00
Marius van der Wijden 4f4622bc8b
core: eth: implement Kiln-v2 spec (#24506)
* core/beacon: eth/catalyst: updated engine api to new version

* core: implement exchangeTransitionConfig

* core/beacon: prevRandao instead of Random

* eth/catalyst: Fix ExchangeTransitionConfig, add test

* eth/catalyst: stop external miners on TTD reached

* node: implement --authrpc.vhosts flag

* core: allow for config override on non-mainnet networks

* eth/catalyst: fix peters comments

* eth/catalyst: make stop remote sealer more explicit

* eth/catalyst: add log output

* cmd/utils: rename authrpc.host to authrpc.addr

* eth/catalyst: disable the disabling of the miner

* eth: core: remove notion of terminal pow block

* eth: les: more of peters nitpicks
2022-03-17 17:20:03 +02:00
aaronbuchwald c3701b265e
core/rawdb: add specified key length iterator (#24535) 2022-03-15 10:28:26 +02:00
Péter Szilágyi 70da74e73a
Merge pull request #24515 from karalabe/pending-statedb-accesslist-init
core/statedb: always clear out access list when setting a new one
2022-03-14 10:51:32 +02:00
Péter Szilágyi 8f66ea3786
eth/downloader: implement beacon sync (#23982)
* eth/downloader: implement beacon sync

* eth/downloader: fix a crash if the beacon chain is reduced in length

* eth/downloader: fix beacon sync start/stop thrashing data race

* eth/downloader: use a non-nil pivot even in degenerate sync requests

* eth/downloader: don't touch internal state on beacon Head retrieval

* eth/downloader: fix spelling mistakes

* eth/downloader: fix some typos

* eth: integrate legacy/beacon sync switchover and UX

* eth: handle UX wise being stuck on post-merge TTD

* core, eth: integrate the beacon client with the beacon sync

* eth/catalyst: make some warning messages nicer

* eth/downloader: remove Ethereum 1&2 notions in favor of merge

* core/beacon, eth: clean up engine API returns a bit

* eth/downloader: add skeleton extension tests

* eth/catalyst: keep non-kiln spec, handle mining on ttd

* eth/downloader: add beacon header retrieval tests

* eth: fixed spelling, commented failing tests out

* eth/downloader: review fixes

* eth/downloader: drop peers failing to deliver beacon headers

* core/rawdb: track beacon sync data in db inspect

* eth: fix review concerns

* internal/web3ext: nit

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
2022-03-11 14:14:45 +02:00
rjl493456442 538a868384
core/rawdb, cmd, ethdb, eth: implement freezer tail deletion (#23954)
* core/rawdb, cmd, ethdb, eth: implement freezer tail deletion

* core/rawdb: address comments from martin and sina

* core/rawdb: fixes cornercase in tail deletion

* core/rawdb: separate metadata into a standalone file

* core/rawdb: remove unused code

* core/rawdb: add random test

* core/rawdb: polish code

* core/rawdb: fsync meta file before manipulating the index

* core/rawdb: fix typo

* core/rawdb: address comments
2022-03-10 09:37:23 +01:00
rjl493456442 8c8a9e5ca1
core, ethdb, tests, trie: introduce database snapshot (#24486) 2022-03-10 09:35:22 +01:00
Justus d1f6a9f544
core/types: improve error for too short transaction / receipt encoding (#24256)
Co-authored-by: Felix Lange <fjl@twurst.com>
2022-03-09 10:44:53 +01:00
Ceyhun Onur 8401e4277a
core/rawdb: add HasCode, HashTrieNode and use them where possible (#24454) 2022-03-09 00:39:34 +01:00
Péter Szilágyi 48605b5f61
core/statedb: always clear out access list when setting a new one 2022-03-08 12:00:29 +02:00
Felix Lange d6f49bf764
core/types: faster RLP encoding of Header, StateAcccount, ReceiptForStorage (#24420)
This change makes use of the new code generator rlp/rlpgen to improve the
performance of RLP encoding for Header and StateAccount. It also speeds up
encoding of ReceiptForStorage using the new rlp.EncoderBuffer API.

The change is much less transparent than I wanted it to be, because Header and
StateAccount now have an EncodeRLP method defined with pointer receiver. It
used to be possible to encode non-pointer values of these types, but the new
method prevents that and attempting to encode unadressable values (even if
part of another value) will return an error. The error can be surprising and may
pop up in places that previously didn't expect any errors.

To make things work, I also needed to update all code paths (mostly in unit tests)
that lead to encoding of non-pointer values, and pass a pointer instead.

Benchmark results:

    name                             old time/op    new time/op    delta
    EncodeRLP/legacy-header-8           328ns ± 0%     237ns ± 1%   -27.63%  (p=0.000 n=8+8)
    EncodeRLP/london-header-8           353ns ± 0%     247ns ± 1%   -30.06%  (p=0.000 n=8+8)
    EncodeRLP/receipt-for-storage-8     237ns ± 0%     123ns ± 0%   -47.86%  (p=0.000 n=8+7)
    EncodeRLP/receipt-full-8            297ns ± 0%     301ns ± 1%    +1.39%  (p=0.000 n=8+8)

    name                             old speed      new speed      delta
    EncodeRLP/legacy-header-8        1.66GB/s ± 0%  2.29GB/s ± 1%   +38.19%  (p=0.000 n=8+8)
    EncodeRLP/london-header-8        1.55GB/s ± 0%  2.22GB/s ± 1%   +42.99%  (p=0.000 n=8+8)
    EncodeRLP/receipt-for-storage-8  38.0MB/s ± 0%  64.8MB/s ± 0%   +70.48%  (p=0.000 n=8+7)
    EncodeRLP/receipt-full-8          910MB/s ± 0%   897MB/s ± 1%    -1.37%  (p=0.000 n=8+8)

    name                             old alloc/op   new alloc/op   delta
    EncodeRLP/legacy-header-8           0.00B          0.00B           ~     (all equal)
    EncodeRLP/london-header-8           0.00B          0.00B           ~     (all equal)
    EncodeRLP/receipt-for-storage-8     64.0B ± 0%      0.0B       -100.00%  (p=0.000 n=8+8)
    EncodeRLP/receipt-full-8             320B ± 0%      320B ± 0%      ~     (all equal)
2022-02-18 08:10:26 +01:00
rjl493456442 4d086430bd
core, ethdb, tests, trie: implement NewBatchWithSize API for batcher (#24392)
This PR adds an addtional API called `NewBatchWithSize` for db
batcher. It turns out that leveldb batch memory allocation is
super inefficient. The main reason is the allocation step of
leveldb Batch is too small when the batch size is large. It can
take a few second to build a leveldb batch with 100MB size.

Luckily, leveldb also offers another API called MakeBatch which can
pre-allocate the memory area. So if the approximate size of batch is
known in advance, this API can be used in this case.

It's needed in new state scheme PR which needs to commit a batch of
trie nodes in a single batch. Implement the feature in a seperate PR.
2022-02-15 15:15:13 +02:00
Martin Holst Swende 51e7968b8b
core/state: fix read-meters + simplify code (#24304) 2022-02-14 09:22:57 +01:00
rjl493456442 9da25c5db7
all: separate catalyst package (#24280)
* all: seperate catalyst package

* eth/catalyst: moved some methods, added docs

* eth/catalyst, les/catalyst: add method docs

* core, eth, les, miner: move common function to beacon package

* eth/catalyst: goimported

* cmd/utils, miner/stress/beacon: naming nitpicks

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2022-01-31 14:22:35 +02:00
Marius van der Wijden 29cb5deea3
core/rawdb: fix typo (#24289) 2022-01-25 11:36:51 +01:00
Martin Holst Swende 0e35192797
core/rawdb: do prefixed lookup first 2022-01-25 10:51:18 +01:00
Shihao Xia eef7a33135
core, miner, rpc, eth: fix goroutine leaks in tests (#24211)
* fix blocking and non-blocking issues

* core: revert change in blockchain.go

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-01-21 12:22:44 +01:00
Di Peng c029cdc90b
core: fix typo in blockchain test (#24263) 2022-01-21 09:12:40 +01:00
Martin Holst Swende 51eb5f8ca8
cmd/geth: add db cmd to show metadata (#23900)
* cmd/geth: add db cmd to show metadata

* cmd/geth: better output generator status

Co-authored-by: Sina Mahmoodi <1591639+s1na@users.noreply.github.com>

* cmd: minor

Co-authored-by: Sina Mahmoodi <1591639+s1na@users.noreply.github.com>
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
2022-01-18 12:30:41 +02:00
Sina Mahmoodi 4aab440ee2
core/rawdb: enforce readonly in freezer instantiation (#24119)
* freezer: add readonly flag to table

* freezer: enforce readonly in table repair

* freezer: enforce readonly in newFreezer

* minor fix

* minor

* core/rawdb: test that writing during readonly fails

* rm unused log

* check readonly on batch append

* minor

* Revert "check readonly on batch append"

This reverts commit 2ddb5ec4ba.

* review fixes

* minor test refactor

* attempt at fixing windows issue

* add comment re windows sync issue

* k->kind

* open readonly db for genesis check

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-01-18 11:29:38 +02:00
Péter Szilágyi acd7b36999
Merge pull request #24197 from rjl493456442/periodically-flush-batch
core: periodically flush the transaction indexes
2022-01-11 11:32:43 +02:00
Marius van der Wijden b1e72f7ea9
core/evm: RANDOM opcode (EIP-4399) (#24141)
* core: implement eip-4399 random opcode

* core: make vmconfig threadsafe

* core: miner: pass vmConfig by value not reference

* all: enable 4399 by Rules

* core: remove diff (f)

* tests: set proper difficulty (f)

* smaller diff (f)

* eth/catalyst: nit

* core: make RANDOM a pointer which is only set post-merge

* cmd/evm/internal/t8ntool: fix t8n tracing of 4399

* tests: set difficulty

* cmd/evm/internal/t8ntool: check that baserules are london before applying the merge chainrules
2022-01-10 09:44:21 +02:00
Sam 7403a38ab7
core: fix a typo (#24198) 2022-01-06 16:02:23 +02:00
Péter Szilágyi af2ca5a654
Merge pull request #24117 from holiman/db_has
trie, core, eth: use db.has over db.get where possible
2022-01-06 11:30:11 +02:00
Gary Rong 4bd2d0eccf core: periodically flush the transaction indexes 2022-01-05 15:00:03 +08:00
Martin Holst Swende 66a908c5e8
core/rawdb: fix double-lock causing hang (#24189)
Fixes #24159

Co-authored-by: Felix Lange <fjl@twurst.com>
2022-01-04 19:02:37 +01:00
zgfzgf 356bbe343a
core/asm: change order of items in stringtokenTypes (#24153)
This orders the items in slice definition same as the enum values.
2021-12-26 14:58:17 +01:00
Felix Lange 980b7682b4
core/types: document JSON field name equivalents of DynamicFeeTx (#24143) 2021-12-22 15:32:17 +01:00
xwjack 8bbf83e7a4
core: ignore basefee when comparing with pool gasprice in txpool (#24080)
This reverts commit 9489853321.
2021-12-17 15:44:57 +01:00
Marius van der Wijden 2295640ebd
eth/catalyst: implement kintsugi-spec v3 (#24067) 2021-12-17 15:38:58 +01:00
Sina Mahmoodi ada9c774e9
eth, les: update unclean shutdown markers regularly (#24077)
Fixes #22580

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-12-17 15:18:51 +01:00
Andrei Maiboroda 3e47e38a4e
core/vm: Make INVALID a defined opcode (#24017)
* core/vm: Define 0xfe opcode as INVALID

* core/vm: Remove opInvalid as opUndefined handles it

Co-authored-by: Alex Beregszaszi <alex@rtfs.hu>
2021-12-17 13:44:05 +01:00
Paweł Bylica 81ec6b1d4c
core/vm: reverse bit order in bytes of code bitmap (#24120)
* core/vm: reverse bit order in bytes of code bitmap

This bit order is more natural for bit manipulation operations and we
can eliminate some small number of CPU instructions.

* core/vm: drop lookup table
2021-12-17 10:32:00 +01:00
Martin Holst Swende 893502e561
trie, core, eth: use db.has over db.get where possible 2021-12-15 16:16:45 +01:00
Martin Holst Swende 155795be99
core/vm: avoid memory expansion check for trivial ops (#24048) 2021-12-14 11:30:20 +01:00
ucwong adec878c1d
core/rawdb: fix error message fields in ReadLogs (#24104) 2021-12-14 11:11:05 +01:00
Alex Beregszaszi ae8ff2661d
core/vm: remove unused code (IsStaticJump) (#24085) 2021-12-09 13:55:06 +01:00
Martin Holst Swende db03faa10d
core, eth: improve delivery speed on header requests (#23105)
This PR reduces the amount of work we do when answering header queries, e.g. when a peer
is syncing from us.

For some items, e.g block bodies, when we read the rlp-data from database, we plug it
directly into the response package. We didn't do that for headers, but instead read
headers-rlp, decode to types.Header, and re-encode to rlp. This PR changes that to keep it
in RLP-form as much as possible. When a node is syncing from us, it typically requests 192
contiguous headers. On master it has the following effect:

- For headers not in ancient: 2 db lookups. One for translating hash->number (even though
  the request is by number), and another for reading by hash (this latter one is sometimes
  cached).
  
- For headers in ancient: 1 file lookup/syscall for translating hash->number (even though
  the request is by number), and another for reading the header itself. After this, it
  also performes a hashing of the header, to ensure that the hash is what it expected. In
  this PR, I instead move the logic for "give me a sequence of blocks" into the lower
  layers, where the database can determine how and what to read from leveldb and/or
  ancients.

There are basically four types of requests; three of them are improved this way. The
fourth, by hash going backwards, is more tricky to optimize. However, since we know that
the gap is 0, we can look up by the parentHash, and stlil shave off all the number->hash
lookups.

The gapped collection can be optimized similarly, as a follow-up, at least in three out of
four cases.

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-12-07 17:50:58 +01:00
Péter Szilágyi a25906e4c0
Merge pull request #24066 from holiman/no_no_recursion
core/vm: remove no-recursion option from config
2021-12-06 12:47:25 +02:00
rjl493456442 69686fa328
core, core/rawdb: fix transaction indexing (#24024)
This PR fixes a special corner case in transaction indexing.

When the chain is rewound by SetHead to a historical point which is even lower than the transaction indexes tail, then system will report Failed to decode block body error all the time, because the relevant blocks are already deleted.

In order to avoid this "non-critical-but-annoying" issue, we can recap the indexing target to head+1(to is excluded, so it means indexing transactions from 0 to head).
2021-12-06 11:26:03 +01:00
Martin Holst Swende a95675d50f
core/vm: remove no-recursion option from config 2021-12-06 09:55:50 +01:00
Péter Szilágyi cca482b4b1
Merge pull request #24047 from karalabe/delete-dead-sync-bloom
core, eth, les, trie: remove the sync bloom, used by fast sync
2021-12-03 13:55:17 +02:00
Péter Szilágyi 58d1988349
core, eth, les, trie: remove the sync bloom, used by fast sync 2021-12-03 12:32:41 +02:00
Andrei Maiboroda b02fe5317f
core/vm: move interpreter interruption check to jump instructions (#24026)
* core/vm: Remove interpreter loop interruption check

* core/vm: Unit test for interpreter loop interruption

* core/vm: Check for interpreter loop abort on every jump
2021-12-03 11:10:26 +01:00
Paweł Bylica 9331fe28e8
core/vm: fill gaps in jump table with opUndefined (#24031) 2021-12-03 11:04:54 +01:00
Alex Beregszaszi a0f7771962
core/vm: remove stack.pushN (#24040) 2021-12-03 10:16:49 +01:00
Taeik Lim 85064ed09b
all: fix 'the the' in comments (#24036) 2021-12-02 15:42:09 +01:00
Alex Beregszaszi 2be129b5cf
core/vm: rename opSuicide to opSelfdestruct (#24022)
The opcode was renamed in the codebase in 2017, but the functions were kept unchanged.
2021-12-01 10:33:29 +01:00
Alex Beregszaszi 9393d1fb5d
core/vm: Move interpreter.ReadOnly check into the opcode implementations (#23970)
* core/vm: Move interpreter.ReadOnly check into the opcode implementations

Also remove the same check from the interpreter inner loop.

* core/vm: Remove obsolete operation.writes flag

* core/vm: Capture fault states in logger

Co-authored-by: Martin Holst Swende <martin@swende.se>

* core/vm: Remove panic added for testing

Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-12-01 10:21:21 +01:00
Andrei Maiboroda 163f1665dd
core/vm: don't copy JumpTable when no EIP mods are needed (#23977) 2021-11-30 14:21:40 +02:00
Alex Beregszaszi a69d4b273d
core/vm: Rename SHA3 instruction to KECCAK256 (#23976)
This was proposed in 2016, Solidity uses this since 2017, and evmone and other VMs use the keccak256 name. This brings geth in line with those.
2021-11-30 10:34:34 +01:00
Paweł Bylica 1fa91729f2
core/vm: simplify error handling in interpreter loop (#23952)
* core/vm: break loop on any error

* core/vm: move ErrExecutionReverted to opRevert()

* core/vm: use "stop token" to stop the loop

* core/vm: unconditionally pc++ in the loop

* core/vm: set return data in instruction impls
2021-11-29 14:46:24 +01:00
Péter Szilágyi c10a0a62c3
eth: request id dispatcher and direct req/reply APIs (#23576)
* eth: request ID based message dispatcher

* eth: fix dispatcher cancellation, rework fetchers idleness tracker

* eth/downloader: drop peers who refuse to serve advertised chains
2021-11-26 13:26:03 +02:00
Marius van der Wijden 3038e480f5
all: core rework for the merge transition (#23761)
* all: work for eth1/2 transtition

* consensus/beacon, eth: change beacon difficulty to 0

* eth: updates

* all: add terminalBlockDifficulty config, fix rebasing issues

* eth: implemented merge interop spec

* internal/ethapi: update to v1.0.0.alpha.2

                                                                 This commit updates the code to the new spec, moving payloadId into
                                                                 it's own object. It also fixes an issue with finalizing an empty blockhash.
                                                                 It also properly sets the basefee

* all: sync polishes, other fixes + refactors

* core, eth: correct semantics for LeavePoW, EnterPoS

* core: fixed rebasing artifacts

* core: light: performance improvements

* core: use keyed field (f)

* core: eth: fix compilation issues + tests

* eth/catalyst: dbetter error codes

* all: move Merger to consensus/, remove reliance on it in bc

* all: renamed EnterPoS and LeavePoW to ReachTDD and FinalizePoS

* core: make mergelogs a function

* core: use InsertChain instead of InsertBlock

* les: drop merger from lightchain object

* consensus: add merger

* core: recoverAncestors in catalyst mode

* core: fix nitpick

* all: removed merger from beacon, use TTD, nitpicks

* consensus: eth: add docstring, removed unnecessary code duplication

* consensus/beacon: better comment

* all: easy to fix nitpicks by karalabe

* consensus/beacon: verify known headers to be sure

* core: comments

* core: eth: don't drop peers who advertise blocks, nitpicks

* core: never add beacon blocks to the future queue

* core: fixed nitpicks

* consensus/beacon: simplify IsTTDReached check

* consensus/beacon: correct IsTTDReached check

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2021-11-26 13:23:02 +02:00
Andrei Maiboroda 519cf98b69
core/vm: simplify op lookup in contract (#23974) 2021-11-25 20:10:01 +01:00
Péter Szilágyi 4ebeca19d7
Merge pull request #23967 from ipsilon/evm_jumptable_type
core/vm: use proper JumpTable type
2021-11-25 14:18:34 +02:00
Sina Mahmoodi 1876cb443b
all: move loggers to eth/tracers (#23892)
* all: mv loggers to eth/tracers

* core/vm: minor

* eth/tracers: tmp comment out testStoreCapture

* eth/tracers: uncomment and fix logger test

* eth/tracers: simplify test

* core/vm: re-add license

* core/vm: minor

* rename LogConfig to Config
2021-11-25 14:17:09 +02:00
Martin Holst Swende 9055cc14ec
core/vm, core/state/snapshot: remove unused code (#23956)
* core/state/snapshot: remove wiper functionality

* core/vm: remove unused 'unofficial' opcodes
2021-11-25 10:37:47 +02:00
Paweł Bylica 10b1cd9b1b
core/vm: use proper JumpTable type 2021-11-24 16:02:12 +01:00
Martin Holst Swende 347c37b362
core/rawdb: use AncientRange when initializing leveldb from freezer (#23612)
* core/rawdb: utilize AncientRange when initiating from freezer

* core/rawdb: remove debug sanity check
2021-11-23 12:37:26 +01:00
Péter Szilágyi 50e07a1e16
Merge pull request #23928 from holiman/no_iota
core/vm: don't use iota for opcode definitions
2021-11-23 11:50:17 +02:00
Péter Szilágyi d9c13d407f
core, eth/downloader: fix resetting below freezer threshold 2021-11-22 11:12:51 +02:00
Martin Holst Swende ab31fbbde1
core/vm: don't use iota for opcode definitions 2021-11-18 09:50:52 +01:00
Jesse Tane 33f2813809
cmd/geth: add flag --dev.gaslimit for dev mode (#23686)
* cmd, core: add flag --dev.gaslimit to allow configuring initial block gas limit in dev mode

* core: use provided gaslimit

Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-11-16 13:45:02 +01:00
Martin Holst Swende b7a6409cc1
core/rawdb: better error message in freezer (#23901)
* core/rawdb: better error message in freezer

* Apply suggestions from code review
2021-11-16 11:33:56 +02:00
Sina Mahmoodi c57df9ca28
core/rawdb: add slow path for getting legacy logs (#23879)
* eth/tracers: add slow path for getting legacy logs

* core/rawdb: fix test
2021-11-11 15:04:06 +01:00
Andrei Maiboroda f32feeb260
core/vm: implement EIP-2681: Limit account nonce to 2^64-1 (#23853)
This retroactively implements requirements or EIP-2681 for the account nonce upper limit.
2021-11-11 15:00:58 +01:00
Martin Holst Swende 6b9c77f060
eth/tracers: package restructuring (#23857)
* eth/tracers: restructure tracer package

* core/vm/runtime: load js tracers

* eth/tracers: mv bigint js code to own file

* eth/tracers: add method docs for native tracers

* eth/tracers: minor doc fix

* core,eth: cancel evm on nativecalltracer stop

* core/vm: fix failing test

Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
2021-11-09 12:09:35 +01:00
Felix Lange 9489853321
core: check effective tip in txpool pricelimit validation (#23855)
The price limit is supposed to exclude transactions with too low fee
amount. Before EIP-1559, it was sufficient to check the limit against
the gas price of the transaction. After 1559, it is more complicated
because the concept of 'transaction gas price' does not really exist.

When mining, the price limit is used to exclude transactions below a
certain effective fee amount. This change makes it apply the same check
earlier, in tx validation. Transactions below the specified fee amount
cannot enter the pool.

Fixes #23837
2021-11-08 16:25:35 +02:00
Martin Holst Swende e1c000b0dd
cmd/geth: add support for sepolia testnet (#23730)
* cmd/geth: add support for sepolia testnet

* core: last details on sepolia genesis

* params: fix sepolia hash + reduce testing code

* Update params/bootnodes.go

* cmd/geth: fix attach path for sepolia

* params: update bootnodes

* params: fix

* core: fix docstring

* params: add sepolia CHT
2021-11-08 13:06:01 +02:00
Sina Mahmoodi 8d7e6062ec
eth/tracers: support for golang tracers + add golang callTracer (#23708)
* eth/tracers: add basic native loader

* eth/tracers: add GetResult to tracer interface

* eth/tracers: add native call tracer

* eth/tracers: fix call tracer json result

* eth/tracers: minor fix

* eth/tracers: fix

* eth/tracers: fix benchTracer

* eth/tracers: test native call tracer

* eth/tracers: fix

* eth/tracers: rm extra make

Co-authored-by: Martin Holst Swende <martin@swende.se>

* eth/tracers: rm extra make

* eth/tracers: make callFrame private

* eth/tracers: clean-up and comments

* eth/tracers: add license

* eth/tracers: rework the model a bit

* eth/tracers: move tracecall tests to subpackage

* cmd/geth: load native tracers

* eth/tracers: minor fix

* eth/tracers: impl stop

* eth/tracers: add native noop tracer

* renamings

Co-authored-by: Martin Holst Swende <martin@swende.se>

* eth/tracers: more renamings

* eth/tracers: make jstracer non-exported, avoid cast

* eth/tracers, core/vm: rename vm.Tracer to vm.EVMLogger for clarity

* eth/tracers: minor comment fix

* eth/tracers/testing: lint nitpicks

* core,eth: cancel evm on nativecalltracer stop

* Revert "core,eth: cancel evm on nativecalltracer stop"

This reverts commit 01bb908790.

* eth/tracers: linter nits

* eth/tracers: fix output on err

Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-11-05 11:48:21 +01:00
Martin Holst Swende 03bc8b7858
core: more efficient nonce-update in txpool (#22231)
* Adjust pending nonce update operation

Benchmark the speed of transaction insertion under multiple accounts

core: fix rebase issues + docstring

core: make benchmark test use sync:ed method

* core: address review comments

* core: add memreport to benchmark

Co-authored-by: WeiLoy <wei_loy@163.com>
2021-11-02 18:32:23 +01:00
rjl493456442 2e8b58f076
cmd/geth: implement data import and export (#22931)
This PR offers two more database sub commands for exporting and importing data.
Two exporters are implemented: preimage and snapshot data respectively. 
The import command is generic, it can take any data export and import into leveldb. 
The data format has a 'magic' for disambiguation, and a version field for future compatibility.
2021-11-02 11:31:45 +01:00
Martin Holst Swende 551bd6e721
eth/tracers: invoke enter/exit on 0-value calls to inex accounts (#23828) 2021-11-01 19:06:33 +01:00
Ziyuan Zhong(仲梓源) c576fa153a
core: fix snapshot missing when recovery from crash (#23496)
It is because write known block only checks block and state without snapshot, which could lead to gap between newest snapshot and newest block state. However, new blocks which would cause snapshot to become fixed were ignored, since state was already known. 


Co-authored-by: Gary Rong <garyrong0905@gmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-11-01 14:09:36 +01:00
Martin Holst Swende 32150f8aa9
cmd/geth, cmd/evm, params: implement Arrow Glacier (EIP 4345) (#23810)
This PR adds support for ArrowGlacier, as defined by

    https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/arrow-glacier.md
    https://eips.ethereum.org/EIPS/eip-4345

> Starting with FORK_BLOCK_NUMBER the client will calculate the difficulty based on a fake block number suggesting to the client that the difficulty bomb is adjusting 10,700,000 blocks later than the actual block number.

This also adds support for evm t8n to return the calculated difficulty, so it can be used to construct test.
2021-10-28 22:18:14 +02:00
Marius van der Wijden bff330335b
core: fixed stale comment in txlist (#23825) 2021-10-28 14:59:51 +02:00
Martin Holst Swende eab4d898fd
core: fix benchmark tests (#23803)
Fixes crashes in various benchmarks in the core package
2021-10-27 13:08:51 +02:00
Martin Holst Swende 526c3f6b9e
core/state/snapshot: fix benchmarks (#23804) 2021-10-26 11:01:01 +02:00
meowsbits c72b16c340
core: use block difficulty for genesis (#23793)
* core: write test showing that TD is not stored properly at genesis

The ToBlock method applies a default value for an empty
difficulty value. This default is not carried over through the Commit
method because the TotalDifficulty database write writes the
original difficulty value (nil) instead of the defaulty value
present on the genesis Block.

Date: 2021-10-22 08:25:32-07:00
Signed-off-by: meows <b5c6@protonmail.com>

* core: write TD value from Block, not original genesis value

This an issue where a default TD value was not written to
the database, resulting in a 0 value TD at genesis.

A test for this issue was provided at 90e3ffd393

Date: 2021-10-22 08:28:00-07:00
Signed-off-by: meows <b5c6@protonmail.com>

* core: fix tests by adding GenesisDifficulty to expected result

See prior two commits.

Date: 2021-10-22 09:16:01-07:00
Signed-off-by: meows <b5c6@protonmail.com>

* les: fix test with genesis change

Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-10-26 08:44:43 +02:00
Martin Holst Swende 0e7efd696b
core/rawdb, ethdb: introduce batched/atomic reads from ancients (#23566)
This PR adds a new accessor method to the freezer database. This new view offers a consistent interface, guaranteeing that all individual tables (headers, bodies etc) are all on the same number, and that this number is not changes (added/truncated) while the operation is performing.
2021-10-25 16:24:27 +02:00
Ziyuan Zhong 312e02bca9
core/state/snapshot: fix BAD BLOCK error when snapshot is generating (#23635)
* core/state/snapshot: fix BAD BLOCK error when snapshot is generating

* core/state/snapshot: alternative fix for the snapshot generator

* add comments and minor update

Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-10-20 10:05:27 +02:00
Marius van der Wijden c641cff51a
core: refactored blockchain.go (#23735) 2021-10-18 10:45:59 +03:00
rjl493456442 f915f6873f
core/state/snapshot: fix data race in layer flattening (#23628)
* core/state/snapshot: fix data race in layer flattening

* core/state/snapshot: fix typo
2021-10-15 10:52:40 +03:00
jwasinger 011fe3eb5e
core: remove unused error from TxPool.Pending (#23720) 2021-10-13 23:00:45 +02:00
Iskander (Alex) Sharipov 778ff94794
all: fix some go-critic linter warnings (#23709)
This doesn't fix all go-critic warnings, just the most serious ones.

Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-10-13 17:31:02 +02:00
Ian Norden e4f570fcc6
core/types: add MarshalBinary, UnmarshalBinary for Receipt (#22806) 2021-10-13 15:16:16 +02:00
Martin Holst Swende 6289137827
consensus/clique, core: API cleanup (#23100)
This removes some code:

- The clique engine calculated the snapshot twice when verifying headers/blocks.

- The method GetBlockHashesFromHash in Header/Block/Lightchain was only used by tests. It
  is now removed from the API.
  
- The method GetTdByHash internally looked up the number before calling GetTd(hash, num).
  In many cases, callers already had the number, and used this method just because it has a
  shorter name. I have removed the method to make the API surface smaller.
2021-10-11 23:16:46 +02:00
Miro cf8a6d6173
core: tests for forked blocks retrievable by hash (#23695)
* Update tests to showcase that forked blocks can still be looked up by their hash
2021-10-11 19:25:21 +02:00
hsyodyssey 84bccd0900
core/state: fix typos in comments (#23702) 2021-10-10 15:00:00 +02:00
Martin Holst Swende edb1937cf7
core: improve shutdown synchronization in BlockChain (#22853)
This change removes misuses of sync.WaitGroup in BlockChain. Before this change,
block insertion modified the WaitGroup counter in order to ensure that Stop would wait
for pending operations to complete. This was racy and could even lead to crashes
if Stop was called at an unfortunate time. The issue is resolved by adding a specialized
'closable' mutex, which prevents chain modifications after stopping while also
synchronizing writers with each other.

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-10-07 15:47:50 +02:00
aaronbuchwald 4e599ee469
core/types: copy tx recipient address (#23376)
This resolves a long-standing TODO. The point of copying the address is
to ensure that all data referenced by types.Transaction is independent of the
data passed into the constructor.

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-10-06 13:12:52 +02:00
hsyodyssey 12f971fb2d
core/state: fix typo in comment (#23665) 2021-10-04 12:16:50 +02:00
Ikko Ashimine f2491c5ed7
core: fix typo in comment (#23658) 2021-09-29 21:28:47 +02:00
Sina Mahmoodi 783e97ef1f
core/rawdb: avoid unnecessary receipt processing for log filtering (#23147)
* core/types: rm extranous check in test

* core/rawdb: add lightweight types for block logs

* core/rawdb,eth: use lightweight accessor for log filtering

* core/rawdb: add bench for decoding into rlpLogs
2021-09-28 12:54:49 +02:00
Guillaume Ballet 443afc975c
core/state: move state account to core/types + abstracted "write account to trie" (#23567)
* core/state: abstracted "write account to trie" method

* fix appveyor build

* Apply suggestions from code review

Co-authored-by: Martin Holst Swende <martin@swende.se>

* review feedback

* core/state/accounts: move Account to core/types

* core/types: rename Account -> StateAccount

* core/state: restore EncodeRLP for stateObject

* core/types: add the missing file

* more review feedback

* more review feedback

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Felix Lange <fjl@twurst.com>
2021-09-28 10:48:07 +02:00
Marius van der Wijden b61ef24cce
consensus/clique: prevent 0 len extradata from panicing (#23538)
Closes #23522

Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-09-21 10:46:08 +02:00
Sina Mahmoodi 401354976b
core,eth: call frame tracing (#23087)
This change introduces 2 new optional methods; `enter()` and `exit()` for js tracers, and makes `step()` optiona. The two new methods are invoked when entering and exiting a call frame (but not invoked for the outermost scope, which has it's own methods). Currently these are the data fields passed to each of them:

    enter: type (opcode), from, to, input, gas, value
    exit: output, gasUsed, error

The PR also comes with a re-write of the callTracer. As a backup we keep the previous tracing script under the name `callTracerLegacy`. Behaviour of both tracers are equivalent for the most part, although there are some small differences (improvements), where the new tracer is more correct / has more information.
2021-09-17 09:31:22 +02:00
Marius van der Wijden b8d7c662cd
core, rpc: disable memory output by default in traces (#23558)
* core: cmd: invert disableMemory

* core: fix missed inversion

* cmd/evm: preserve Flags but change default value

* Apply suggestions from code review

Co-authored-by: Martin Holst Swende <martin@swende.se>

Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-09-13 18:59:52 +02:00
Felix Lange 5c1fc3bf54
core: 64bit alignment of testBlockChain.gasLimit (#23543)
This fixes crashes in txpool tests on 32bit platforms.
2021-09-07 18:54:15 +03:00
Felix Lange 51ed39c093
core: make txPricedList.stales the first field for 64bit alignment (#23542)
This fixes crashes on 32bit builds.
2021-09-07 17:15:11 +03:00
Martin Holst Swende 794c6133ef
core/rawdb: freezer batch write (#23462)
This change is a rewrite of the freezer code.

When writing ancient chain data to the freezer, the previous version first encoded each
individual item to a temporary buffer, then wrote the buffer. For small item sizes (for
example, in the block hash freezer table), this strategy causes a lot of system calls for
writing tiny chunks of data. It also allocated a lot of temporary []byte buffers.

In the new version, we instead encode multiple items into a re-useable batch buffer, which
is then written to the file all at once. This avoids performing a system call for every
inserted item.

To make the internal batching work, the ancient database API had to be changed. While
integrating this new API in BlockChain.InsertReceiptChain, additional optimizations were
also added there.

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-09-07 12:31:17 +02:00
Marius van der Wijden 067084feda
core: fix race conditions in txpool (#23474)
* core: fix race conditions in txpool

* core: fixed races in the txpool

* core: rebased on master

* core: move reheap mutex

* core: renamed mutex

* core: revert Reheap changes
2021-08-31 10:33:39 +02:00
Ikko Ashimine 31be5d41d9
core: fix typo in iterator.go (#23502)
strorage -> storage
2021-08-31 10:21:42 +02:00
Paweł Bylica 3258211f68
core/vm: rework jumpdest analysis benchmarks (#23499)
* core/vm: rework jumpdest analysis benchmarks

For BenchmarkJumpdestOpAnalysis use fixed code size of ~1.2MB
and classic benchmark loop.

* core/vm: clear bitvec in jumpdest analysis benchmark
2021-08-30 14:13:06 +02:00
Felix Lange 8a134014b4
all: add go:build lines (#23468)
Generated by go1.17 fmt ./...
2021-08-25 18:46:29 +02:00
gary rong a5a5237178
core, light, tests, trie: add state metrics (#23433) 2021-08-24 22:00:42 +03:00
Martin Holst Swende d705f5a554
core: make txpool reject too sudden changes (#23095)
* core: make txpool reject too sudden changes

* core: add some metrics to txpool
2021-08-24 21:48:36 +03:00
Martin Holst Swende a0a4a153e9
core/types: add benchmarks for rlp encoding/decoding (#23190)
Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Sina Mahmoodi <1591639+s1na@users.noreply.github.com>
2021-08-24 13:59:15 +02:00
Guillaume Ballet 0b40977480
core/vm: fix typo in comment (#23450) 2021-08-24 13:57:05 +02:00
Pedro Gomes 8e0771c218
core/bloombits: avoid crash when storing errors of different type (#23437)
This fixes a rare crash which could occur when two different errors happened
in the same bloombits.MatcherSession.
2021-08-24 13:32:19 +02:00
Martin Holst Swende 1d99573192
core/vm: faster code analysis (#23381)
* core/vm: more detailed benchmark for jumpdest analysis

* core/vm: make jd analysis benchmark alloc free

* core/vm: improve jumpdest analysis

* core/vm: improve worst-case

* core/vm: further improvements in analysis

* core/vm: improve jumpdest analysis >PUSH15

* core/vm: make jd analysis ref by value

* core/vm: fix misspell

* core/vm: improve set8 and set16 a bit

* core/vm: reduce amount of code

* core/vm: optimize byte copying
2021-08-24 10:00:10 +03:00
Marius van der Wijden d02c605367
core: only check sendernoeoa in non fake mode (#23424) 2021-08-23 12:49:39 +03:00
chuwt 5566e5d152
eth/downloader: fix typo in comment (#23413) 2021-08-18 13:03:41 +03:00
Zachinquarantine 85b9bdd641
cmd, core: remove calaveras testnet (#23366)
Removes references to the short-lived Calaveras testnet
2021-08-17 18:43:25 +02:00
Martin Holst Swende 5f98020a21
core/rawdb: implement sequential reads in freezer_table (#23117)
* core/rawdb: implement sequential reads in freezer_table

* core/rawdb, ethdb: add sequential reader to db interface

* core/rawdb: lint nitpicks

* core/rawdb: fix some nitpicks

* core/rawdb: fix flaw with deferred reads not being performed

* core/rawdb: better documentation
2021-08-13 11:51:01 +03:00
Péter Szilágyi 9d6480c3cd
core/state/snapshot: gofmt 2021-08-10 16:58:38 +03:00
Tyler Chambers 66948316f7
core/state/snapshot: clarify comment about snapshot repair (#23305)
Co-authored-by: Tyler Chambers <me@tylerchambers.net>
Co-authored-by: Felix Lange <fjl@twurst.com>
2021-08-10 11:16:53 +02:00
Ziyuan Zhong 57d9e0ac75
core/state/snapshot: fix typo in comment (#23219) 2021-08-10 11:04:29 +02:00
gary rong 6d175460df
cmd, core, eth, miner: deprecate miner.gastarget flag (#23213) 2021-08-10 11:28:33 +03:00
Péter Szilágyi 3b38a83274
core/state/pruner: fix state bloom sync permission in Windows 2021-08-10 10:40:10 +03:00
shawn d60cfd2604
core: fix london-check to avoid duplication (#23333)
Co-authored-by: lxex <liuxmzc1@163.com>
2021-08-09 16:34:20 +02:00
Shihao Xia 9e59474e46
core/rawdb: close database in test to avoid goroutine leak (#23287)
* add db close to avoid goroutine leak

* core/rawdb: move close to defer

Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-08-08 15:44:42 +02:00
Martin Holst Swende 0658712f65
core: check if sender is EOA (#23303)
This adds a check to verify that a sender-account does not have code, which means that the codehash is either `emptyCodeHash` _OR_ not present. The latter occurs IFF the sender did not previously exist, a situation which can only occur with zero cost gasprices.
2021-08-07 19:38:18 +02:00
Patrick O'Grady d3e3a460ec
core/rawdb: fix logs to print block number, not address (#23328) 2021-08-04 11:10:37 +03:00
Marius van der Wijden 28ba686cbf
core/state: add trie prefetcher tests (#23216)
* core/state: add trie prefetcher tests

* core/state: add missing license
2021-08-03 17:35:25 +02:00
Sina Mahmoodi c38fab912b
core: get header from block cache (#23299) 2021-08-03 17:29:47 +02:00
aaronbuchwald 82c5085399
cre/state: fix outdated statedb Prepare comment (#23320) 2021-08-03 09:06:58 +03:00
baptiste-b-pegasys 3526f69047
all: remove term "whitelist" in comments and log messages (#23294) 2021-07-29 17:36:15 +02:00
baptiste-b-pegasys 523866c2cc
all: change blacklist terms 2021-07-29 11:17:40 +03:00
Marius van der Wijden 2faf796d2a
internal/ethapi: fix panic in accesslist creation (#23225)
* internal/ethapi: revert + fix properly in al tracer

* internal/ethapi: use toMessage instead of creating new message

* internal/ethapi: remove ineffassign

* core: fix invalid unmarshalling, fix test

Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-07-28 14:21:35 +02:00
gary rong 370680a7a9
core/types: revert removal of legacy receipt support (#23247)
* Revert "core/types: go generate (#23177)"

This reverts commit 00b922fc5d.

* Revert "core/types: remove LogForStorage type (#23173)"

This reverts commit 7522642393.

* Revert "core/types: remove support for legacy receipt/log storage encoding (#22852)"

This reverts commit 643fd0efc6.
2021-07-22 15:43:51 +02:00