Commit Graph

152 Commits

Author SHA1 Message Date
Péter Szilágyi 0c6bbeb423
core, eth, trie: expose more detailed dirty ram tracking for diff layers (#27971) 2023-08-23 14:08:39 +03:00
rjl493456442 503f1f7ada
all: activate pbss as experimental feature (#26274)
* all: activate pbss

* core/rawdb: fix compilation error

* cma, core, eth, les, trie: address comments

* cmd, core, eth, trie: polish code

* core, cmd, eth: address comments

* cmd, core, eth, les, light, tests: address comment

* cmd/utils: shorten log message

* trie/triedb/pathdb: limit node buffer size to 1gb

* cmd/utils: fix opening non-existing db

* cmd/utils: rename flag name

* cmd, core: group chain history flags and fix tests

* core, eth, trie: fix memory leak in snapshot generation

* cmd, eth, internal: deprecate flags

* all: enable state tests for pathdb, fixes

* cmd, core: polish code

* trie/triedb/pathdb: limit the node buffer size to 256mb

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2023-08-10 22:21:36 +03:00
rjl493456442 88f3d61468
all: expose block number information to statedb (#27753)
* core/state: clean up

* all: add block number infomration to statedb

* core, trie: rename blockNumber to block
2023-07-24 13:22:09 +03:00
Darioush Jalali 83d7f426d1
core: stop chain when tests are finished (#27660)
core (blockchain_test): add chain.Stop() to tests
2023-07-06 02:56:42 -04:00
rjl493456442 5021d36d35
all: port boring changes from pbss (#27176)
* all: port boring changes from pbss

* core, trie: address comments from martin

* trie: minor fixes

* core/rawdb: update comment

* core, eth, tests, trie: address comments

* tests, trie: add extra check when update trie database

* trie/triedb/hashdb: degrade the error to warning
2023-05-09 10:11:04 +03:00
Péter Szilágyi dde2da0efb
all: remove ethash pow, only retain shims needed for consensus and tests (#27178)
* all: remove ethash pow, only retain shims needed for consensus and tests

* all: thank you linter

* all: disallow launching Geth in legacy PoW mode

* cmd/env/internal/t8ntool: remove dangling ethash flag
2023-05-03 12:58:39 +03:00
Péter Szilágyi bbc565ab05
core/types, params: add blob transaction type, RLP encoded for now (#27049)
* core/types, params: add blob transaction type, RLP encoded for now

* all: integrate Cancun (and timestamp based forks) into MakeSigner

* core/types: fix 2 back-and-forth type refactors

* core: fix review comment

* core/types: swap blob tx type id to 0x03
2023-04-21 12:52:02 +03:00
Martin Holst Swende ab1a404b01
all: remove debug-field from vm config (#27048)
This PR removes the Debug field from vmconfig, making it so that if a tracer is set, debug=true is implied.

---------

Co-authored-by: 0xTylerHolmes <tyler@ethereum.org>
Co-authored-by: Sina Mahmoodi <1591639+s1na@users.noreply.github.com>
2023-04-04 09:50:13 -04:00
Péter Szilágyi cd31f2dee2
all: change chain head markers from block to header (#26777) 2023-03-02 08:29:15 +02:00
Péter Szilágyi 095e365fac
all: remove support for Ropsten (#26644) 2023-02-09 10:03:00 +02:00
Marius van der Wijden 2a2b0419fb
all: implement withdrawals (EIP-4895) (#26484)
This change implements withdrawals as specified in EIP-4895.

Co-authored-by: lightclient@protonmail.com <lightclient@protonmail.com>
Co-authored-by: marioevz <marioevz@gmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-01-25 15:32:25 +01:00
Martin Holst Swende 2b57a27d9e
all: make timestamp-based fork checks based on uint64 (#26474)
This PR changes the API so that uint64 is used for fork timestamps.
It's a good choice because types.Header also uses uint64 for time.

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-01-25 12:12:28 +01:00
Péter Szilágyi 08481028fe
core, les, params: add timestamp based fork compatibility checks 2023-01-03 12:57:06 +02:00
Martin Holst Swende b818e73ef3
tests: update tests (#26314)
This PR builds on #26299, but also updates the tests to the most recent version, which includes tests regarding TheMerge.

This change adds checks to the beacon consensus engine, making it more strict in validating the pre- and post-headers, and not relying on the caller to have already correctly sanitized the headers/blocks.
2022-12-20 09:56:52 -05:00
Marius van der Wijden ec2ec2d08e
core: implement EIP-3651, warm coinbase (#25819)
Implements EIP-3651, "Warm Coinbase", for Shanghai hardfork. Specification: https://eips.ethereum.org/EIPS/eip-3651.
2022-11-22 22:39:52 +01:00
Mark Tyneway b4ea2bf7dd
all: implement EIP-1153 transient storage (#26003)
Implements TSTORE and TLOAD as specified by the following EIP:

https://eips.ethereum.org/EIPS/eip-1153
https://ethereum-magicians.org/t/eip-1153-transient-storage-opcodes/553


Co-authored-by: Sara Reynolds <snreynolds2506@gmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2022-11-16 10:18:52 +01:00
rjl493456442 60e30a940b
core/rawdb: refactor db inspector for extending multiple ancient store (#25896)
This PR ports a few changes from PBSS:

- Fix the snapshot generator waiter in case the generation is not even initialized
- Refactor db inspector for ancient store
2022-10-28 10:23:49 +02:00
Martin Holst Swende 067bac3f24
core, ethclient/gethclient: improve flaky tests (#25918)
* ethclient/gethclient: improve time-sensitive flaky test

* eth/catalyst: fix (?) flaky test

* core: stop blockchains in tests after use

* core: fix dangling blockchain instances

* core: rm whitespace

* eth/gasprice, eth/tracers, consensus/clique: stop dangling blockchains in tests

* all: address review concerns

* core: goimports

* eth/catalyst: fix another time-sensitive test

* consensus/clique: add snapshot test run function

* core: rename stop() to stopWithoutSaving()

Co-authored-by: Felix Lange <fjl@twurst.com>
2022-10-06 13:39:20 +02:00
rjl493456442 052c634917
core: rework tx indexer (#25723)
This PR reworks tx indexer a bit. Compared to the original version, one scenario is no longer handled - upgrading  from legacy geth without indexer support. 

The tx indexer was introduced in 2020 and have been present through hardforks, so it can be assumed that all Geth nodes have tx indexer already. So we can simplify the tx indexer logic a bit:

-    If the tail flag is not present, it means node is just initialized may or may not with an ancient store attached. In this case all blocks are regarded as unindexed
-   If the tail flag is present, it means blocks below tail are unindexed, blocks above tail are indexed

This change also address some weird cornercases that could make the indexer not work after a crash.
2022-09-29 09:50:24 +02:00
Felix Lange b628d72766
build: upgrade to go 1.19 (#25726)
This changes the CI / release builds to use the latest Go version. It also
upgrades golangci-lint to a newer version compatible with Go 1.19.

In Go 1.19, godoc has gained official support for links and lists. The
syntax for code blocks in doc comments has changed and now requires a
leading tab character. gofmt adapts comments to the new syntax
automatically, so there are a lot of comment re-formatting changes in this
PR. We need to apply the new format in order to pass the CI lint stage with
Go 1.19.

With the linter upgrade, I have decided to disable 'gosec' - it produces
too many false-positive warnings. The 'deadcode' and 'varcheck' linters
have also been removed because golangci-lint warns about them being
unmaintained. 'unused' provides similar coverage and we already have it
enabled, so we don't lose much with this change.
2022-09-10 13:25:40 +02:00
Martin Holst Swende 389021a5af
core: place a cap on reorglogs (#25711)
This PR makes the event-sending for deleted and new logs happen in batches, to prevent OOM situation due to large reorgs. 

Co-authored-by: Felix Lange <fjl@twurst.com>
2022-09-09 15:25:55 +02:00
rjl493456442 dea1fb3cfc
all: cleanup tests (#25641)
Follow-up to PR #25523 to cleanup all relevant tests.
2022-09-07 20:21:59 +02:00
rjl493456442 d10c280309
all: move genesis initialization to blockchain (#25523)
* all: move genesis initialization to blockchain

* core: add one more check

* core: fix tests
2022-08-30 18:22:28 +02:00
rjl493456442 a41ea8a97c
all: cleanup the APIs for initializing genesis (#25473)
* all: polish tests

* core: apply feedback from Guillaume

* core: fix comment
2022-08-09 12:44:39 +03:00
int88 e4b3bd6f26
core: fix uncle creation in TestFastVsFullChains (#25476)
Co-authored-by: Felix Lange <fjl@twurst.com>
2022-08-09 00:20:46 +02:00
Delweng b196ad1c16
all: add whitespace linter (#25312)
* golangci: typo

Signed-off-by: Delweng <delweng@gmail.com>

* golangci: add whietspace

Signed-off-by: Delweng <delweng@gmail.com>

* *: rm whitesapce using golangci-lint

Signed-off-by: Delweng <delweng@gmail.com>

* cmd/puppeth: revert accidental resurrection

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2022-07-25 13:14:03 +03:00
lmittmann bc013bc42e
all: prefer `new(big.Int)` over `big.NewInt(0)` (#25087)
minor performance improvement: `big.NewInt(0).Xxx` -> `new(big.Int).Xxx`
2022-06-14 15:09:48 +03:00
Martin Holst Swende a907d7e81a
all: more linters (#24783)
This enables the following linters

- typecheck
- unused
- staticcheck
- bidichk
- durationcheck
- exportloopref
- gosec

WIth a few exceptions.

- We use a deprecated protobuf in trezor. I didn't want to mess with that, since I cannot meaningfully test any changes there.
- The deprecated TypeMux is used in a few places still, so the warning for it is silenced for now.
- Using string type in context.WithValue is apparently wrong, one should use a custom type, to prevent collisions between different places in the hierarchy of callers. That should be fixed at some point, but may require some attention.
- The warnings for using weak random generator are squashed, since we use a lot of random without need for cryptographic guarantees.
2022-06-13 16:24:45 +02:00
Gustavo Silva eb94896270
Chore: Minimal gramatical errors (signleton -> singleton) (#25057)
core: fix typos
2022-06-10 18:47:06 +03:00
rjl493456442 f9806dc872
core: fix canonical hash marker update (#24996)
* core: fix reorg

* core: revert change for memory efficiency

* core: revert changes
2022-06-01 11:03:24 +02:00
rjl493456442 7f6f01d46f
core: recover state when beacon sets canonical head if it's missing (#24613)
* core: recover the state in SetChainHead if the head state is missing

* core: disable test logging

* core: address comment from martin

* core: improve log level in case state is recovered

* core, eth, les, light: rename SetChainHead to SetCanonical
2022-05-05 10:36:26 +03:00
Eng Zer Jun 8d066f1f42
all: use T.TempDir to create temporary test directories (#24633)
This commit replaces ioutil.TempDir with t.TempDir in tests. The
directory created by t.TempDir is automatically removed when the test
and all its subtests complete.

Prior to this commit, temporary directory created using ioutil.TempDir
had to be removed manually by calling os.RemoveAll, which is omitted in
some tests. The error handling boilerplate e.g.

	defer func() {
		if err := os.RemoveAll(dir); err != nil {
			t.Fatal(err)
		}
	}

is also tedious, but t.TempDir handles this for us nicely.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-04-08 15:44:55 +02:00
Di Peng c029cdc90b
core: fix typo in blockchain test (#24263) 2022-01-21 09:12:40 +01:00
Péter Szilágyi c10a0a62c3
eth: request id dispatcher and direct req/reply APIs (#23576)
* eth: request ID based message dispatcher

* eth: fix dispatcher cancellation, rework fetchers idleness tracker

* eth/downloader: drop peers who refuse to serve advertised chains
2021-11-26 13:26:03 +02:00
Marius van der Wijden 3038e480f5
all: core rework for the merge transition (#23761)
* all: work for eth1/2 transtition

* consensus/beacon, eth: change beacon difficulty to 0

* eth: updates

* all: add terminalBlockDifficulty config, fix rebasing issues

* eth: implemented merge interop spec

* internal/ethapi: update to v1.0.0.alpha.2

                                                                 This commit updates the code to the new spec, moving payloadId into
                                                                 it's own object. It also fixes an issue with finalizing an empty blockhash.
                                                                 It also properly sets the basefee

* all: sync polishes, other fixes + refactors

* core, eth: correct semantics for LeavePoW, EnterPoS

* core: fixed rebasing artifacts

* core: light: performance improvements

* core: use keyed field (f)

* core: eth: fix compilation issues + tests

* eth/catalyst: dbetter error codes

* all: move Merger to consensus/, remove reliance on it in bc

* all: renamed EnterPoS and LeavePoW to ReachTDD and FinalizePoS

* core: make mergelogs a function

* core: use InsertChain instead of InsertBlock

* les: drop merger from lightchain object

* consensus: add merger

* core: recoverAncestors in catalyst mode

* core: fix nitpick

* all: removed merger from beacon, use TTD, nitpicks

* consensus: eth: add docstring, removed unnecessary code duplication

* consensus/beacon: better comment

* all: easy to fix nitpicks by karalabe

* consensus/beacon: verify known headers to be sure

* core: comments

* core: eth: don't drop peers who advertise blocks, nitpicks

* core: never add beacon blocks to the future queue

* core: fixed nitpicks

* consensus/beacon: simplify IsTTDReached check

* consensus/beacon: correct IsTTDReached check

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2021-11-26 13:23:02 +02:00
Sina Mahmoodi 1876cb443b
all: move loggers to eth/tracers (#23892)
* all: mv loggers to eth/tracers

* core/vm: minor

* eth/tracers: tmp comment out testStoreCapture

* eth/tracers: uncomment and fix logger test

* eth/tracers: simplify test

* core/vm: re-add license

* core/vm: minor

* rename LogConfig to Config
2021-11-25 14:17:09 +02:00
Martin Holst Swende eab4d898fd
core: fix benchmark tests (#23803)
Fixes crashes in various benchmarks in the core package
2021-10-27 13:08:51 +02:00
meowsbits c72b16c340
core: use block difficulty for genesis (#23793)
* core: write test showing that TD is not stored properly at genesis

The ToBlock method applies a default value for an empty
difficulty value. This default is not carried over through the Commit
method because the TotalDifficulty database write writes the
original difficulty value (nil) instead of the defaulty value
present on the genesis Block.

Date: 2021-10-22 08:25:32-07:00
Signed-off-by: meows <b5c6@protonmail.com>

* core: write TD value from Block, not original genesis value

This an issue where a default TD value was not written to
the database, resulting in a 0 value TD at genesis.

A test for this issue was provided at 90e3ffd393

Date: 2021-10-22 08:28:00-07:00
Signed-off-by: meows <b5c6@protonmail.com>

* core: fix tests by adding GenesisDifficulty to expected result

See prior two commits.

Date: 2021-10-22 09:16:01-07:00
Signed-off-by: meows <b5c6@protonmail.com>

* les: fix test with genesis change

Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-10-26 08:44:43 +02:00
Martin Holst Swende 6289137827
consensus/clique, core: API cleanup (#23100)
This removes some code:

- The clique engine calculated the snapshot twice when verifying headers/blocks.

- The method GetBlockHashesFromHash in Header/Block/Lightchain was only used by tests. It
  is now removed from the API.
  
- The method GetTdByHash internally looked up the number before calling GetTd(hash, num).
  In many cases, callers already had the number, and used this method just because it has a
  shorter name. I have removed the method to make the API surface smaller.
2021-10-11 23:16:46 +02:00
Miro cf8a6d6173
core: tests for forked blocks retrievable by hash (#23695)
* Update tests to showcase that forked blocks can still be looked up by their hash
2021-10-11 19:25:21 +02:00
Martin Holst Swende edb1937cf7
core: improve shutdown synchronization in BlockChain (#22853)
This change removes misuses of sync.WaitGroup in BlockChain. Before this change,
block insertion modified the WaitGroup counter in order to ensure that Stop would wait
for pending operations to complete. This was racy and could even lead to crashes
if Stop was called at an unfortunate time. The issue is resolved by adding a specialized
'closable' mutex, which prevents chain modifications after stopping while also
synchronizing writers with each other.

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-10-07 15:47:50 +02:00
Martin Holst Swende 794c6133ef
core/rawdb: freezer batch write (#23462)
This change is a rewrite of the freezer code.

When writing ancient chain data to the freezer, the previous version first encoded each
individual item to a temporary buffer, then wrote the buffer. For small item sizes (for
example, in the block hash freezer table), this strategy causes a lot of system calls for
writing tiny chunks of data. It also allocated a lot of temporary []byte buffers.

In the new version, we instead encode multiple items into a re-useable batch buffer, which
is then written to the file all at once. This avoids performing a system call for every
inserted item.

To make the internal batching work, the ancient database API had to be changed. While
integrating this new API in BlockChain.InsertReceiptChain, additional optimizations were
also added there.

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-09-07 12:31:17 +02:00
baptiste-b-pegasys 523866c2cc
all: change blacklist terms 2021-07-29 11:17:40 +03:00
Péter Szilágyi 7a7abe3de8
accounts/abi/bind: fix bounded contracts and sim backend for 1559 (#23038)
* accounts/abi/bind: fix bounded contracts and sim backend for 1559

* accounts/abi/bind, ethclient: don't rely on chain config for gas prices

* all: enable London for all internal tests

* les: get receipt type info in les tests

* les: fix weird test

Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-06-15 13:56:14 +03:00
Péter Szilágyi c503f98f6d
all: rename internal 1559 gas fields, add support for graphql (#23010)
* all: rename internal 1559 gas fields, add support for graphql

* cmd/evm/testdata, core: use public 1559 gas names on API surfaces
2021-06-08 12:05:41 +02:00
Martin Holst Swende 67e7f61af7
core: fix failing tests (#22888)
This PR fixes two errors that regressed when EIP-1559 was merged.
2021-05-18 01:10:28 +02:00
Martin Holst Swende 94451c2788
all: implement EIP-1559 (#22837)
This is the initial implementation of EIP-1559 in packages core/types and core.
Mining, RPC, etc. will be added in subsequent commits.

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: lightclient@protonmail.com <lightclient@protonmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
2021-05-17 15:13:22 +02:00
Martin Holst Swende a5669ae292
core, params: implement EIP-3529 (#22733)
* core, params: implement EIP-3529

* core/vm: add london instructionset

* core/vm: add method doc for EIP enabler

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2021-05-07 09:25:32 +03:00
Martin Holst Swende cc606be74c
all: define London+baikal, undefine yolov3, add london override flag (#22822)
* all: define London+baikal, undefine yolov3, add london override flag

* cmd, core, params: add baikal genesis definition
2021-05-06 12:07:42 +03:00
Péter Szilágyi 1e207342b5
all: make logs a bit easier on the eye to digest (#22665)
* all: add thousandths separators for big numbers on log messages

* p2p/sentry: drop accidental file

* common, log: add fast number formatter

* common, eth/protocols/snap: simplifty fancy num types

* log: handle nil big ints
2021-04-15 20:35:00 +03:00