Commit Graph

2035 Commits

Author SHA1 Message Date
Felix Lange ee35ddc8fd
cmd/devp2p/internal/ethtest: ignore block announcement in tx test (#22957) 2021-05-27 20:53:33 +02:00
Marius van der Wijden 04cb5e2be3
cmd/puppeth: remove outdated mist support (#22940) 2021-05-27 19:45:13 +03:00
rene d836ad141e
cmd/devp2p/internal/ethtest: add block hash announcement test (#22535) 2021-05-27 11:57:49 +02:00
Martin Holst Swende 2e7714f864
cmd/utils: avoid large alloc in --dev mode (#22949)
* cmd/utils: avoid 1Gb alloc in --dev mode

* cmd/geth: avoid 512Mb alloc in genesis query tests
2021-05-27 10:13:35 +02:00
rene 49bde05a55
cmd/devp2p: refactor eth test suite (#22843)
This PR refactors the eth test suite to make it more readable and
easier to use. Some notable differences:

- A new file helpers.go stores all of the methods used between
  both eth66 and eth65 and below tests, as well as methods shared
  among many test functions.
- suite.go now contains all of the test functions for both eth65
  tests and eth66 tests.
- The utesting.T object doesn't get passed through to other helper methods,
  but is instead only used within the scope of the test function,
  whereas helper methods return errors, so only the test function
  itself can fatal out in the case of an error.
- The full test suite now only takes 13.5 seconds to run.
2021-05-25 23:09:11 +02:00
lightclient b3a1fda650
cmd/utils: expand tilde in --jspath (#22900) 2021-05-18 19:54:10 +02:00
Martin Holst Swende 94451c2788
all: implement EIP-1559 (#22837)
This is the initial implementation of EIP-1559 in packages core/types and core.
Mining, RPC, etc. will be added in subsequent commits.

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: lightclient@protonmail.com <lightclient@protonmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
2021-05-17 15:13:22 +02:00
lightclient 597ecb39cc
cmd/evm: return json error if unmarshalling from stdin fails (#22871)
* cmd/evm: return json error if unmarshalling from stdin fails

* cmd/evm: make error capitalizations uniform (all lowercase starts)

* cmd/evm: capitalize error sent directly to stderror
2021-05-17 08:52:32 +02:00
Martin Holst Swende addd8824cf
cmd/geth, eth, core: snapshot dump + unify with trie dump (#22795)
* cmd/geth, eth, core: snapshot dump + unify with trie dump

* cmd/evm: dump API fixes

* cmd/geth, core, eth: fix some remaining errors

* cmd/evm: dump - add limit, support address startkey, address review concerns

* cmd, core/state, eth: minor polishes, fix snap dump crash, unify format

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2021-05-12 11:05:39 +03:00
Ryan Schneider ca98080798
cmd/geth, eth/gasprice: add configurable threshold to gas price oracle (#22752)
This adds a cmd line parameter `--gpo.ignoreprice`, to make the gas price oracle ignore transactions below the given threshold.
2021-05-11 11:25:51 +02:00
Ceelog f19a679b09
cmd/geth: remove reference to monitor command (#22844)
'geth monitor' subcommand is no longer supported.
2021-05-10 12:19:32 +02:00
Martin Holst Swende cc606be74c
all: define London+baikal, undefine yolov3, add london override flag (#22822)
* all: define London+baikal, undefine yolov3, add london override flag

* cmd, core, params: add baikal genesis definition
2021-05-06 12:07:42 +03:00
rene 0f3a1e7f9b
cmd/devp2p/internal/ethtest: send simultaneous requests on one connection (#22801)
This changes the SimultaneousRequests test to send the requests from the same
connection, as it doesn't really make sense to test whether a node can respond
to two requests with different request IDs from separate connections.
2021-05-05 12:27:27 +02:00
Felix Lange b8040a430e
cmd/utils: use eth DNS tree for snap discovery (#22808)
This removes auto-configuration of the snap.*.ethdisco.net DNS discovery tree.
Since measurements have shown that > 75% of nodes in all.*.ethdisco.net support
snap, we have decided to retire the dedicated index for snap and just use the eth
tree instead.

The dial iterators of eth and snap now use the same DNS tree in the default configuration,
so both iterators should use the same DNS discovery client instance. This ensures that
the record cache and rate limit are shared. Records will not be requested multiple times.

While testing the change, I noticed that duplicate DNS requests do happen even
when the client instance is shared. This is because the two iterators request the tree
root, link tree root, and first levels of the tree in lockstep. To avoid this problem, the
change also adds a singleflight.Group instance in the client. When one iterator
attempts to resolve an entry which is already being resolved, the singleflight object
waits for the existing resolve call to finish and returns the entry to both places.
2021-05-04 11:29:32 +02:00
Marius van der Wijden 8f94fc26e3
cmd/utils: don't crash on nonexistent datadir (#22738) 2021-05-03 15:29:05 +03:00
rene 8ff98108e5
cmd/devp2p: fix flakey tests in CI (#22757)
This PR fixes a couple of issues in the eth test suite that caused flakiness when run in the CI.
2021-04-30 22:47:36 +02:00
rene e4270cacf4
cmd/devp2p: fix flaky SameRequestID test (#22754) 2021-04-28 21:38:38 +02:00
Martin Holst Swende 0c99868416
cmd/devp2p, eth/protocols/eth: fix tests + make sanity checks earlier (#22749) 2021-04-28 08:48:07 +02:00
lightclient ad983b300b
cmd/puppeth: add support for authentication via ssh agent (#22634) 2021-04-27 11:36:57 +02:00
rene cac1b21d39
cmd/devp2p/internal/ethtest: add more tx propagation tests (#22630)
This adds a test for large tx announcement messages, as well as a test to
check that announced tx hashes are requested by the node.
2021-04-23 18:14:39 +02:00
rene ea54c58d4f
cmd/devp2p/internal/ethtest: run test suite as Go unit test (#22698)
This change adds a Go unit test that runs the protocol test suite
against the go-ethereum implementation of the eth protocol.
2021-04-23 11:15:42 +02:00
Felix Lange 653b7e959d
cmd/devp2p: add dns nuke-route53 command (#22695) 2021-04-19 14:54:55 +02:00
Felix Lange 424656519a
cmd/devp2p: add support for -limit option in nodeset filter command (#22694)
The new -limit option makes the filter operate on top N nodes by score.
This also adds ENR attribute stats in the nodeset info command.
Node set commands are now documented in README.
2021-04-19 14:54:38 +02:00
Guillaume Ballet f79cce5de9
eth/catalyst: add catalyst API prototype (#22641)
This change adds the --catalyst flag, enabling an RPC API for eth2 integration.
In this initial version, catalyst mode also disables all peer-to-peer networking.

Co-authored-by: Mikhail Kalinin <noblesse.knight@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
2021-04-16 21:29:22 +02:00
Péter Szilágyi 1e207342b5
all: make logs a bit easier on the eye to digest (#22665)
* all: add thousandths separators for big numbers on log messages

* p2p/sentry: drop accidental file

* common, log: add fast number formatter

* common, eth/protocols/snap: simplifty fancy num types

* log: handle nil big ints
2021-04-15 20:35:00 +03:00
gary rong 7088f1e814
core, eth: faster snapshot generation (#22504)
* eth/protocols: persist received state segments

* core: initial implementation

* core/state/snapshot: add tests

* core, eth: updates

* eth/protocols/snapshot: count flat state size

* core/state: add metrics

* core/state/snapshot: skip unnecessary deletion

* core/state/snapshot: rename

* core/state/snapshot: use the global batch

* core/state/snapshot: add logs and fix wiping

* core/state/snapshot: fix

* core/state/snapshot: save generation progress even if the batch is empty

* core/state/snapshot: fixes

* core/state/snapshot: fix initial account range length

* core/state/snapshot: fix initial account range

* eth/protocols/snap: store flat states during the healing

* eth/protocols/snap: print logs

* core/state/snapshot: refactor (#4)

* core/state/snapshot: refactor

* core/state/snapshot: tiny fix and polish

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>

* core, eth: fixes

* core, eth: fix healing writer

* core, trie, eth: fix paths

* eth/protocols/snap: fix encoding

* eth, core: add debug log

* core/state/generate: release iterator asap (#5)

core/state/snapshot: less copy

core/state/snapshot: revert split loop

core/state/snapshot: handle storage becoming empty, improve test robustness

core/state: test modified codehash

core/state/snapshot: polish

* core/state/snapshot: optimize stats counter

* core, eth: add metric

* core/state/snapshot: update comments

* core/state/snapshot: improve tests

* core/state/snapshot: replace secure trie with standard trie

* core/state/snapshot: wrap return as the struct

* core/state/snapshot: skip wiping correct states

* core/state/snapshot: updates

* core/state/snapshot: fixes

* core/state/snapshot: fix panic due to reference flaw in closure

* core/state/snapshot: fix errors in state generation logic + fix log output

* core/state/snapshot: remove an error case

* core/state/snapshot: fix condition-check for exhausted snap state

* core/state/snapshot: use stackTrie for small tries

* core/state/snapshot: don't resolve small storage tries in vain

* core/state/snapshot: properly clean up storage of deleted accounts

* core/state/snapshot: avoid RLP-encoding in some cases + minor nitpicks

* core/state/snapshot: fix error (+testcase)

* core/state/snapshot: clean up tests a bit

* core/state/snapshot: work in progress on better tests

* core/state/snapshot: polish code

* core/state/snapshot: fix trie iteration abortion trigger

* core/state/snapshot: fixes flaws

* core/state/snapshot: remove panic

* core/state/snapshot: fix abort

* core/state/snapshot: more tests (plus failing testcase)

* core/state/snapshot: more testcases + fix for failing test

* core/state/snapshot: testcase for malformed data

* core/state/snapshot: some test nitpicks

* core/state/snapshot: improvements to logging

* core/state/snapshot: testcase to demo error in abortion

* core/state/snapshot: fix abortion

* cmd/geth: make verify-state report the root

* trie: fix failing test

* core/state/snapshot: add timer metrics

* core/state/snapshot: fix metrics

* core/state/snapshot: udpate tests

* eth/protocols/snap: write snapshot account even if code or state is needed

* core/state/snapshot: fix diskmore check

* core/state/snapshot: review fixes

* core/state/snapshot: improve error message

* cmd/geth: rename 'error' to 'err' in logs

* core/state/snapshot: fix some review concerns

* core/state/snapshot, eth/protocols/snap: clear snapshot marker when starting/resuming snap sync

* core: add error log

* core/state/snapshot: use proper timers for metrics collection

* core/state/snapshot: address some review concerns

* eth/protocols/snap: improved log message

* eth/protocols/snap: fix heal logs to condense infos

* core/state/snapshot: wait for generator termination before restarting

* core/state/snapshot: revert timers to counters to track total time

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2021-04-14 23:23:11 +03:00
Mudit Gupta 72e37942f3
cmd/faucet: support testnet flags in the faucet (#22545)
Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-04-13 23:51:46 +02:00
Martin Holst Swende 271e5b7fc9
cmd/geth: add db-command to inspect freezer index (#22633)
This PR makes it easier to inspect the freezer index, which could be useful to investigate things like #22111
2021-04-13 15:45:30 +02:00
Peter Simard 95219ae62d
cmd/utils: move cache sanity check to SetEthConfig (#22510)
Move the cache sanity check to the SetEthConfig function
to allow the config file to load.
2021-04-06 16:23:35 +02:00
Martin Holst Swende 59ac3c9fd3
cmd/geth: add db dumptrie command (#22563)
Adds the command "geth db dumptrie <root> <seek> <max>", to better help investigate the trie data
2021-03-30 13:57:21 +02:00
Marius van der Wijden 24588bacfd
cmd/puppeth: specify working directory for nodejs 15 (#22549) 2021-03-29 20:58:58 +02:00
Felix Lange 27056f62e5
cmd/devp2p: update to newer cloudflare API client (#22588)
This upgrades the cloudflare client dependency to v0.14.0. The new
version changes the API because all methods now require a context
parameter. This change also reduces the log level of the 'Skipping...'
message to debug, following a similar change in the AWS deployer.
2021-03-26 23:15:20 +02:00
Felix Lange cae6b5527e
cmd/geth, consensus/ethash: add support for --miner.notify.full flag (#22558)
The PR implements the --miner.notify.full flag that enables full pending block
notifications. When this flag is used, the block notifications sent to mining
endpoints contain the complete block header JSON instead of a work package
array.

Co-authored-by: AlexSSD7 <alexandersadovskyi7@protonmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-03-26 18:30:10 +01:00
Felix Lange bed74b38d9
cmd/devp2p: fix comparison of TXT record value (#22572)
* cmd/devp2p: fix comparison of TXT record value

The AWS API returns quoted DNS strings, so we must encode the new value
before comparing it against the existing record content.

* cmd/devp2p: add test

* cmd/devp2p: fix typo and rename val -> newValue
2021-03-25 13:32:32 +02:00
Felix Lange 5129cdc4f0
cmd/devp2p: skip ENR field tails properly in nodeset filter (#22565)
In Geth v1.10, we changed the structure of the "les" ENR entry. As a result, the DHT crawler that creates the DNS lists
no longer recognizes the les nodes, which is fixed in this commit. 

* cmd/devp2p: skip ENR field tails properly in nodeset filter
* cmd/devp2p: fix tail decoder for snap as well
* les: fix tail decoding in "eth" ENR entry
2021-03-24 12:32:39 +01:00
Sina Mahmoodi 8d6cc16742
cmd/geth: check block range against chain head in export cmd (#22387)
Check the input parameters against the actual head block, exit on error
2021-03-22 19:11:10 +01:00
gary rong 0c70b83e00
all: add read-only option to database (#22407)
* all: add read-only option to database

* all: fixes tests

* cmd/geth: migrate flags

* cmd/geth: fix the compact

* cmd/geth: fix the format

* cmd/geth: fix log

* cmd: add chain-readonly

* core: add readonly notion to freezer

* core/rawdb: add log

* core/rawdb: fix freezer close

* cmd: fix

* cmd, core: construct db

* core: update tests
2021-03-22 19:06:30 +01:00
Tobias Hildebrandt ec73ec092d
cmd/utils: fix compilation issue on openbsd (#22511) 2021-03-20 19:54:17 +01:00
rene 36b51b8156
cmd/devp2p: add old block announcement test to eth test suite (#22474)
Add old block announcement test to eth test suite, checks to make sure old block announcement isn't propagated
2021-03-20 19:50:44 +01:00
Martin Holst Swende 5bf6612a2e
cmd/devp2p: fix error in updating the cursor when collecting records for route53 (#22538)
This PR fixes a regression introduced in #22360, when we updated to the v2 of the AWS sdk, which causes current crawler to just get the same first 100 results over and over, and get stuck in a loop.
2021-03-20 10:35:22 +01:00
Felix Lange 9429ab1472
cmd/devp2p: add flag for AWS region (#22537) 2021-03-20 00:22:24 +01:00
rene d3040a80d7
cmd/devp2p/internal/ethtest: skip eth/66 tests when v66 not supported (#22460) 2021-03-19 15:15:39 +01:00
Quest Henkart e3a3f7cd64
cmd/devp2p: use AWS-SDK v2 (#22360)
This updates the DNS deployer to use AWS SDK v2. Migration is relatively
seamless, although there were two locations that required a slightly
different approach to achieve the same results. In particular, waiting for
DNS change propagation is very different with SDK v2. 

This change also optimizes DNS updates by publishing all changes before
waiting for propagation.
2021-03-19 13:15:57 +01:00
rene 6a528fce33
cmd/devp2p/internal/ethtest: return request ID in BlockHeaders response (#22508)
This PR fixes an issue with the eth66 test suite where, during a readAndServe when
the test is manually responding to GetBlockHeader requests, it now responds
with a BlockHeaders eth66 packet that includes the inbound request ID.
2021-03-19 10:57:23 +01:00
ligi 7cbf1d70a7
cmd/clef (docs): fix image background (#22399)
Flatten the image so we do not have dark text on dark background
2021-03-16 09:55:03 +01:00
ligi 7076e8e42f
cmd/clef: docs - link to ethereum org repo (#22400) 2021-03-16 09:53:43 +01:00
Martin Holst Swende faacc8e0fa
cmd/geth, eth/downloader: remove copydb command (#22501)
* cmd/geth: remove copydb command

* eth/downloader: remove fakepeer
2021-03-15 16:25:45 +02:00
Martin Holst Swende 3f74c8e0e5
cmd/devp2p: better testcase failure output for ethtests (#22482) 2021-03-13 21:33:30 +01:00
michael1011 6387c520b7
cmd/geth: add ancient datadir flag to snapshot subcommands (#22486) 2021-03-12 11:16:19 +02:00
gary rong 22082f9e56
cmd: extend dumpgenesis to support network flags on the cmd (#22406) 2021-03-09 11:50:25 +02:00