Commit Graph

497 Commits

Author SHA1 Message Date
Marius van der Wijden e6fa102eb0
core, eth, internal, rpc: implement final block (#24282)
* eth: core: implement finalized block

* eth/catalyst: fix final block

* eth/catalyst: update finalized head gauge

* internal/jsre/deps: updated web3.js to allow for finalized block

* eth/catalyst: make sure only one thread can call fcu

* eth/catalyst: nitpicks

* eth/catalyst: use plain mutex

* eth: nitpicks
2022-05-18 17:30:42 +03:00
Håvard Anda Estensen 07508ac0e9
all: replace uses of ioutil with io and os (#24869) 2022-05-16 11:59:35 +02:00
Martin Holst Swende 5157d4540a
cmd/evm: make evm t8n handle post-merge transitions (#24546)
This adds the ability to run --state.fork=Merged, and have post-merge rules apply. When doing so, it also requires the input env to contain currentRandom, and enforces the currentDifficulty to be omitted or zero.
2022-05-02 16:26:30 +02:00
John Difool a52bcccfe1
consensus/ethash: fix typos in var names (#24745) 2022-04-26 10:16:57 +02:00
ucwong d4d288e3f1
build: add imports for go generate tools (#24682)
This adds a tools.go file to import all command packages used for
go:generate. Doing so makes it possible to execute go-based code
generators using 'go run', locking in the tool version using go.mod.

Co-authored-by: Felix Lange <fjl@twurst.com>
2022-04-12 20:24:02 +02:00
rjl493456442 da16d089c0
trie, les, tests, core: implement trie tracer (#24403)
Trie tracer is an auxiliary tool to capture all deleted nodes
which can't be captured by trie.Committer. The deleted nodes
can be removed from the disk later.
2022-03-31 09:28:32 +02:00
Jonathan Chappelow 830231c1c4
crypto: use btcec/v2 for no-cgo (#24533)
This updates the no-cgo implementations in the crypto package to use
the github.com/btcsuite/btcd/btcec/v2 module instead of the older btcec
package that was part of the main github.com/btcsuite/btcd module.

name                   old time/op  new time/op  delta
EcrecoverSignature-32   198µs ± 0%   144µs ± 0%  -27.11%
VerifySignature-32      177µs ± 0%   128µs ± 0%  -27.44%
DecompressPubkey-32    20.9µs ± 0%  10.1µs ± 0%  -51.51%

Use (*ModNScalar).IsOverHalfOrder instead of math/big.Int when checking
for malleable signatures.
2022-03-16 14:23:14 +01:00
rjl493456442 7a80cf6543
les, tests: fix vflux fuzzer by removing unnecessary panic (#24537) 2022-03-16 11:13:10 +01:00
rjl493456442 8c8a9e5ca1
core, ethdb, tests, trie: introduce database snapshot (#24486) 2022-03-10 09:35:22 +01:00
rjl493456442 4d086430bd
core, ethdb, tests, trie: implement NewBatchWithSize API for batcher (#24392)
This PR adds an addtional API called `NewBatchWithSize` for db
batcher. It turns out that leveldb batch memory allocation is
super inefficient. The main reason is the allocation step of
leveldb Batch is too small when the batch size is large. It can
take a few second to build a leveldb batch with 100MB size.

Luckily, leveldb also offers another API called MakeBatch which can
pre-allocate the memory area. So if the approximate size of batch is
known in advance, this API can be used in this case.

It's needed in new state scheme PR which needs to commit a batch of
trie nodes in a single batch. Implement the feature in a seperate PR.
2022-02-15 15:15:13 +02:00
Paweł Bylica 64c53edf83
tests: external evm benchmarks (#24050)
* tests: add ipsilon/evm-benchmarks git submodule

* tests: plug-in evm-benchmarks
2022-01-28 08:47:19 +01:00
Marius van der Wijden b1e72f7ea9
core/evm: RANDOM opcode (EIP-4399) (#24141)
* core: implement eip-4399 random opcode

* core: make vmconfig threadsafe

* core: miner: pass vmConfig by value not reference

* all: enable 4399 by Rules

* core: remove diff (f)

* tests: set proper difficulty (f)

* smaller diff (f)

* eth/catalyst: nit

* core: make RANDOM a pointer which is only set post-merge

* cmd/evm/internal/t8ntool: fix t8n tracing of 4399

* tests: set difficulty

* cmd/evm/internal/t8ntool: check that baserules are london before applying the merge chainrules
2022-01-10 09:44:21 +02:00
Ikko Ashimine f5f5c0855a
tests/solidity/contracts: fix typo in OpCodes.sol (#24123) 2021-12-17 15:19:34 +01:00
Martin Holst Swende 1988b47e02
tests/fuzzzers. eth/protocols/snap: add snap protocol fuzzers (#23957) 2021-12-01 10:17:18 +01:00
Sina Mahmoodi 1876cb443b
all: move loggers to eth/tracers (#23892)
* all: mv loggers to eth/tracers

* core/vm: minor

* eth/tracers: tmp comment out testStoreCapture

* eth/tracers: uncomment and fix logger test

* eth/tracers: simplify test

* core/vm: re-add license

* core/vm: minor

* rename LogConfig to Config
2021-11-25 14:17:09 +02:00
Martin Holst Swende 32150f8aa9
cmd/geth, cmd/evm, params: implement Arrow Glacier (EIP 4345) (#23810)
This PR adds support for ArrowGlacier, as defined by

    https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/arrow-glacier.md
    https://eips.ethereum.org/EIPS/eip-4345

> Starting with FORK_BLOCK_NUMBER the client will calculate the difficulty based on a fake block number suggesting to the client that the difficulty bomb is adjusting 10,700,000 blocks later than the actual block number.

This also adds support for evm t8n to return the calculated difficulty, so it can be used to construct test.
2021-10-28 22:18:14 +02:00
Marius van der Wijden b8d7c662cd
core, rpc: disable memory output by default in traces (#23558)
* core: cmd: invert disableMemory

* core: fix missed inversion

* cmd/evm: preserve Flags but change default value

* Apply suggestions from code review

Co-authored-by: Martin Holst Swende <martin@swende.se>

Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-09-13 18:59:52 +02:00
Felix Lange 8a134014b4
all: add go:build lines (#23468)
Generated by go1.17 fmt ./...
2021-08-25 18:46:29 +02:00
Martin Holst Swende 3784e15106
tests: avoid performance tests during CI runs (#23304) 2021-08-25 15:07:58 +02:00
gary rong a5a5237178
core, light, tests, trie: add state metrics (#23433) 2021-08-24 22:00:42 +03:00
baptiste-b-pegasys 5cee33eb72
tests/fuzzers: fix go vet warning about ReadByte (#23380) 2021-08-24 14:27:15 +02:00
Marius van der Wijden d02c605367
core: only check sendernoeoa in non fake mode (#23424) 2021-08-23 12:49:39 +03:00
Martin Holst Swende fb4007bb22
tests: update, enable legacy tests, remove vm tests (#23350)
* tests: update, enable legacy tests, remove vm tests

* tests: minor fixes
2021-08-17 17:30:21 +02:00
Martin Holst Swende 0658712f65
core: check if sender is EOA (#23303)
This adds a check to verify that a sender-account does not have code, which means that the codehash is either `emptyCodeHash` _OR_ not present. The latter occurs IFF the sender did not previously exist, a situation which can only occur with zero cost gasprices.
2021-08-07 19:38:18 +02:00
baptiste-b-pegasys 85afdeef37
tests: remove whitelist feature (#23297) 2021-07-29 20:23:37 +02:00
Marius van der Wijden fb8ea5993f
tests: update tests/testdata to v9.0.4 (london) (#23279) 2021-07-29 14:05:22 +02:00
Sina Mahmoodi 5441a8fa47
all: remove noop vm config flags (#23111)
* all: rm external interpreter and ewasm config

* core/vm: rm Interpreter interface

* cmd/geth: deprecate interpreter config fields
2021-07-06 22:03:09 +02:00
Marius van der Wijden ef946a6c87
tests: fix eip1559 tx on non-eip1559 network (#23054) 2021-06-18 12:34:31 +02:00
Marius van der Wijden 58aeab77d2
tests: fix nil pointer panic on failure (#23053) 2021-06-18 12:21:34 +02:00
Martin Holst Swende 0e9c7d564d
tests: update for London (#22976)
This updates the tests submodule to the London fork tests, and
also updates the test runner to support the new EIP-1559 fields in
test JSON.
2021-06-07 14:37:56 +02:00
Marius van der Wijden 0703ef62d3
crypto/secp256k1: fix undefined behavior in BitCurve.Add (#22621)
This commit changes the behavior of BitCurve.Add to be more inline
with btcd. It fixes two different bugs:

1) When adding a point at infinity to another point, the other point
   should be returned. While this is undefined behavior, it is better
   to be more inline with the go standard library.
   Thus (0,0) + (a, b) = (a,b)

2) Adding the same point to itself produced the point at infinity.
   This is incorrect, now doubleJacobian is used to correctly calculate it.
   Thus (a,b) + (a,b) == 2* (a,b) and not (0,0) anymore.

The change also adds a differential fuzzer for Add, testing it against btcd.

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-05-27 13:30:25 +02:00
Eugene Lepeico 6c7d6cf886
tests: get test name from testing.T (#22941)
There were 2 TODOs about that fix after Golang 1.8 release.
It's here for 3 years already, so now should be the right time.
2021-05-25 22:47:14 +02:00
ucwong 017cf71fbd
rlp, tests/fuzzers/bls12381: gofmt (#22937) 2021-05-25 10:14:39 +02:00
Martin Holst Swende 94451c2788
all: implement EIP-1559 (#22837)
This is the initial implementation of EIP-1559 in packages core/types and core.
Mining, RPC, etc. will be added in subsequent commits.

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: lightclient@protonmail.com <lightclient@protonmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
2021-05-17 15:13:22 +02:00
Martin Holst Swende cc606be74c
all: define London+baikal, undefine yolov3, add london override flag (#22822)
* all: define London+baikal, undefine yolov3, add london override flag

* cmd, core, params: add baikal genesis definition
2021-05-06 12:07:42 +03:00
Péter Szilágyi fae165a5de
core, eth, ethdb, trie: simplify range proofs 2021-04-29 10:59:08 +03:00
Martin Holst Swende a81cf0d2b3
trie: remove redundant returns + use stacktrie where applicable (#22760)
* trie: add benchmark for proofless range

* trie: remove unused returns + use stacktrie
2021-04-28 22:47:48 +03:00
gary rong 6d7c9566df
les, tests: fix les clientpool (#22756)
* les, tests: fix les clientpool

* tests: disable debug mode

* les: polish code
2021-04-28 14:18:25 +02:00
Gautam Botrel 9e5bb84c0e
tests/fuzzers: crypto/bn256 and crypto/bls12381 tests against gnark-crypto (#22755)
Add more cross-fuzzers to fuzz bls with gnark versus geth's own bls12-381 library
2021-04-28 12:04:25 +02:00
Péter Szilágyi caea6c4661
eth/protocols/snap: generate storage trie from full dirty snap data (#22668)
* eth/protocols/snap: generate storage trie from full dirty snap data

* eth/protocols/snap: get rid of some more dead code

* eth/protocols/snap: less frequent logs, also log during trie generation

* eth/protocols/snap: implement dirty account range stack-hashing

* eth/protocols/snap: don't loop on account trie generation

* eth/protocols/snap: fix account format in trie

* core, eth, ethdb: glue snap packets together, but not chunks

* eth/protocols/snap: print completion log for snap phase

* eth/protocols/snap: extended tests

* eth/protocols/snap: make testcase pass

* eth/protocols/snap: fix account stacktrie commit without defer

* ethdb: fix key counts on reset

* eth/protocols: fix typos

* eth/protocols/snap: make better use of delivered data (#44)

* eth/protocols/snap: make better use of delivered data

* squashme

* eth/protocols/snap: reduce chunking

* squashme

* eth/protocols/snap: reduce chunking further

* eth/protocols/snap: break out hash range calculations

* eth/protocols/snap: use sort.Search instead of looping

* eth/protocols/snap: prevent crash on storage response with no keys

* eth/protocols/snap: nitpicks all around

* eth/protocols/snap: clear heal need on 1-chunk storage completion

* eth/protocols/snap: fix range chunker, add tests

Co-authored-by: Péter Szilágyi <peterke@gmail.com>

* trie: fix test API error

* eth/protocols/snap: fix some further liter issues

* eth/protocols/snap: fix accidental batch reuse

Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-04-27 17:19:59 +03:00
Martin Holst Swende cc33398cef
tests: disable blockchain tests based on general state tests (#22704) 2021-04-20 12:28:20 +02:00
Felföldi Zsolt 2d89fe0883
les: move client pool to les/vflux/server (#22495)
* les: move client pool to les/vflux/server

* les/vflux/server: un-expose NodeBalance, remove unused fn, fix bugs

* tests/fuzzers/vflux: add ClientPool fuzzer

* les/vflux/server: fixed balance tests

* les: rebase fix

* les/vflux/server: fixed more bugs

* les/vflux/server: unexported NodeStateMachine fields and flags

* les/vflux/server: unexport all internal components and functions

* les/vflux/server: fixed priorityPool test

* les/vflux/server: polish balance

* les/vflux/server: fixed mutex locking error

* les/vflux/server: priorityPool bug fixed

* common/prque: make Prque wrap-around priority handling optional

* les/vflux/server: rename funcs, small optimizations

* les/vflux/server: fixed timeUntil

* les/vflux/server: separated balance.posValue and negValue

* les/vflux/server: polish setup

* les/vflux/server: enforce capacity curve monotonicity

* les/vflux/server: simplified requestCapacity

* les/vflux/server: requestCapacity with target range, no iterations in SetCapacity

* les/vflux/server: minor changes

* les/vflux/server: moved default factors to balanceTracker

* les/vflux/server: set inactiveFlag in priorityPool

* les/vflux/server: moved related metrics to vfs package

* les/vflux/client: make priorityPool temp state logic cleaner

* les/vflux/server: changed log.Crit to log.Error

* add vflux fuzzer to oss-fuzz

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
2021-04-06 20:42:50 +02:00
Martin Holst Swende bc47993692
tests/fuzzers: fix goroutine leak in les fuzzer (#22455)
The oss-fuzz fuzzer has been reporting some failing testcases for les. They're all spurious, and cannot reliably be reproduced. However, running them showed that there was a goroutine leak: the tests created a lot of new clients, which started an exec queue that was never torn down.

This PR fixes the goroutine leak, and also a log message which was erroneously formatted.
2021-03-16 09:43:33 +01:00
Martin Holst Swende 72b8cacf13
tests: update reference tests with 2315 removed from Berlin 2021-03-07 20:55:01 +01:00
Marius van der Wijden cd316d7c71
tests: update to latest tests (#22290)
This updates the consensus tests to commit 31d6630 and
adds support for access list transactions in the test runner.

Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-03-03 15:50:07 +01:00
Péter Szilágyi 0928562670
all: define Berlin hard fork spec 2021-02-26 14:24:07 +02:00
lightclient bbfb1e4008
all: add support for EIP-2718, EIP-2930 transactions (#21502)
This adds support for EIP-2718 typed transactions as well as EIP-2930
access list transactions (tx type 1). These EIPs are scheduled for the
Berlin fork.

There very few changes to existing APIs in core/types, and several new APIs
to deal with access list transactions. In particular, there are two new
constructor functions for transactions: types.NewTx and types.SignNewTx.
Since the canonical encoding of typed transactions is not RLP-compatible,
Transaction now has new methods for encoding and decoding: MarshalBinary
and UnmarshalBinary.

The existing EIP-155 signer does not support the new transaction types.
All code dealing with transaction signatures should be updated to use the
newer EIP-2930 signer. To make this easier for future updates, we have
added new constructor functions for types.Signer: types.LatestSigner and
types.LatestSignerForChainID. 

This change also adds support for the YoloV3 testnet.

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Ryan Schneider <ryanleeschneider@gmail.com>
2021-02-25 15:26:57 +01:00
Felföldi Zsolt 8f03e3b107
tests/fuzzers/les: add fuzzer for les server handler (#22282)
* les: refactored server handler

* tests/fuzzers/les: add fuzzer for les server handler

* tests, les: update les fuzzer

tests: update les fuzzer

tests/fuzzer/les: release resources

tests/fuzzer/les: pre-initialize all resources

* les: refactored server handler and fuzzer

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
2021-02-20 10:40:38 +01:00
gary rong f566dd305e
all: bloom-filter based pruning mechanism (#21724)
* cmd, core, tests: initial state pruner

core: fix db inspector

cmd/geth: add verify-state

cmd/geth: add verification tool

core/rawdb: implement flatdb

cmd, core: fix rebase

core/state: use new contract code layout

core/state/pruner: avoid deleting genesis state

cmd/geth: add helper function

core, cmd: fix extract genesis

core: minor fixes

contracts: remove useless

core/state/snapshot: plugin stacktrie

core: polish

core/state/snapshot: iterate storage concurrently

core/state/snapshot: fix iteration

core: add comments

core/state/snapshot: polish code

core/state: polish

core/state/snapshot: rebase

core/rawdb: add comments

core/rawdb: fix tests

core/rawdb: improve tests

core/state/snapshot: fix concurrent iteration

core/state: run pruning during the recovery

core, trie: implement martin's idea

core, eth: delete flatdb and polish pruner

trie: fix import

core/state/pruner: add log

core/state/pruner: fix issues

core/state/pruner: don't read back

core/state/pruner: fix contract code write

core/state/pruner: check root node presence

cmd, core: polish log

core/state: use HEAD-127 as the target

core/state/snapshot: improve tests

cmd/geth: fix verification tool

cmd/geth: use HEAD as the verification default target

all: replace the bloomfilter with martin's fork

cmd, core: polish code

core, cmd: forcibly delete state root

core/state/pruner: add hash64

core/state/pruner: fix blacklist

core/state: remove blacklist

cmd, core: delete trie clean cache before pruning

cmd, core: fix lint

cmd, core: fix rebase

core/state: fix the special case for clique networks

core/state/snapshot: remove useless code

core/state/pruner: capping the snapshot after pruning

cmd, core, eth: fixes

core/rawdb: update db inspector

cmd/geth: polish code

core/state/pruner: fsync bloom filter

cmd, core: print warning log

core/state/pruner: adjust the parameters for bloom filter

cmd, core: create the bloom filter by size

core: polish

core/state/pruner: sanitize invalid bloomfilter size

cmd: address comments

cmd/geth: address comments

cmd/geth: address comment

core/state/pruner: address comments

core/state/pruner: rename homedir to datadir

cmd, core: address comments

core/state/pruner: address comment

core/state: address comments

core, cmd, tests: address comments

core: address comments

core/state/pruner: release the iterator after each commit

core/state/pruner: improve pruner

cmd, core: adjust bloom paramters

core/state/pruner: fix lint

core/state/pruner: fix tests

core: fix rebase

core/state/pruner: remove atomic rename

core/state/pruner: address comments

all: run go mod tidy

core/state/pruner: avoid false-positive for the middle state roots

core/state/pruner: add checks for middle roots

cmd/geth: replace crit with error

* core/state/pruner: fix lint

* core: drop legacy bloom filter

* core/state/snapshot: improve pruner

* core/state/snapshot: polish concurrent logs to report ETA vs. hashes

* core/state/pruner: add progress report for pruning and compaction too

* core: fix snapshot test API

* core/state: fix some pruning logs

* core/state/pruner: support recovering from bloom flush fail

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2021-02-08 13:16:30 +02:00
Marius van der Wijden 54735a6723
fuzzers: added consensys/gurvy library to bn256 differential fuzzer (#21812)
This pr adds consensys' gurvy bn256 variant into the code for differential fuzzing.
2021-02-03 15:04:28 +01:00