Commit Graph

836 Commits

Author SHA1 Message Date
rjl493456442 2c1af8b1ec
cmd, eth: implement full-sync tester (#26035)
This PR adds a parameter to startup, --synctarget. The synctarget flag is a developer-flag, that can be useful in some scenarios as a replacement for a CL node. It defines a fixed block sync target:

geth --syncmode=full --synctarget=./block_15816882.hex_rlp 

The --synctarget is only made available during syncmode=full
2022-10-28 14:48:08 +02:00
Martin Holst Swende 6069d8294e
cmd/utils: enable snapshot generation in import-mode (#25990)
This PR fixes a regression causing snapshots not to be generated in "geth --import" mode.  It also fixes the geth export command to be truly readonly, and adds a new test for geth export.
2022-10-19 08:20:39 +02:00
Paul 1c737e8b6d
cmd/geth, core: fix typo in comment (#25954)
* fix typo on comment

* typo "can't accept"
2022-10-11 09:37:33 +02:00
Felix Lange 564751668a
cmd/geth: fix help output issues (#25941)
This update resolves an issue where StringSliceFlag would not be
rendered correctly in help output + mention that -H can be used multiple times

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-10-06 19:59:23 +02:00
rjl493456442 1913b50111
eth, miner: add timeout for building sealing block (#25407)
* eth, miner: add timeout for building sealing block

* eth, cmd, miner: add newpayloadtimeout flag

* eth, miner, cmd: address comments

* eth, miner: minor fixes

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-10-03 14:10:00 +02:00
Martin Holst Swende ea26fc8a6c
cmd/geth, cmd/utils: geth attach with custom headers (#25829)
This PR makes it possible to set custom headers, in particular for two scenarios: 

- geth attach
- geth commands which can use --remotedb, e..g geth db inspect

The ability to use custom headers is typically useful for connecting to cloud-apis, e.g. providing an infura- or alchemy key, or for that matter access-keys for environments behind cloudflare.  

Co-authored-by: Felix Lange <fjl@twurst.com>
2022-09-30 13:50:25 +02:00
rjl493456442 bff84a99fe
cmd, core, eth, les, light: track deleted nodes (#25757)
* cmd, core, eth, les, light: track deleted nodes

* trie: add docs

* trie: address comments

* cmd, core, eth, les, light, trie: trie id

* trie: add tests

* trie, core: updates

* trie: fix imports

* trie: add utility print-method for nodeset

* trie: import err

* trie: fix go vet warnings

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-09-27 10:01:02 +02:00
jwasinger 7227c9ef07
cmd/geth: make dumpgenesis load genesis datadir if it exists (#25135)
`geth dumpgenesis` currently does not respect the content of the data directory. Instead, it outputs the genesis block created by command-line flags. This PR fixes it to read the genesis from the database, if the database already exists.


Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-09-26 13:55:18 +02:00
Zachinquarantine 15b4a4bf2e
cmd/puppeth, cmd/utils: finalize removal of gasTarget flag (#24370)
This PR fully removes the --miner.gastarget flag, as previously it was only hidden from the geth --help command, but could still be used.
2022-09-23 22:32:10 +02:00
rjl493456442 3da42f85d9
all: clean up the configs for pruner and snapshotter (#22396)
This PR cleans up the configurations for pruner and snapshotter by passing a config struct.

And also, this PR disables the snapshot background generation if the chain is opened in "read-only" mode. The read-only mode is necessary in some cases. For example, we have a list of commands to open the etheruem node in "read-only" mode, like export-chain. In these cases, the snapshot background generation is non expected and should be banned explicitly.
2022-09-23 20:20:36 +02:00
Felix Lange 65f3c1b46f
internal/version: use gitCommit injection in version handling code (#25851)
This changes the CI build to store the git commit and date into package
internal/version instead of package main. Doing this essentially merges our
two ways of tracking the go-ethereum version into a single place, achieving
two objectives:

- Bad block reports, which use version.Info(), will now have the git commit
  information even when geth is built in an environment such as
  launchpad.net where git access is unavailable.

- For geth builds created by `go build ./cmd/geth` (i.e. not using `go run
  build/ci.go install`), git information stored by the go tool is now used
  in the p2p node name as well as in `geth version` and `geth
  version-check`.
2022-09-23 14:08:25 +02:00
Guillaume Ballet 9d717167aa
cmd/geth: add a verkle subcommand (#25718)
* cmd/geth: add a verkle subcommand

* fix copyright year

* remove unused command parameters

* check that the output file was successfully written to

Co-authored-by: Martin Holst Swende <martin@swende.se>

* cmd/geth: goimports fix

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-09-14 12:05:03 +02:00
Sina Mahmoodi 6e6b5087f1
cmd/geth: fix legacy receipt detection for empty db (#25609) 2022-08-30 14:38:35 +02:00
Sina Mahmoodi 36874b63a1
eth/filters: add global block logs cache (#25459)
This adds a cache for block logs which is shared by all filters. The cache
size of is configurable using the `--cache.blocklogs` flag.

Co-authored-by: Felix Lange <fjl@twurst.com>
2022-08-19 11:14:59 +02:00
Justin Traglia 9762ddf8b0
cmd/geth: parse uint64 value with ParseUint instead of Atoi (#25545)
Parse uint64 value with ParseUint instead of Atoi
2022-08-19 09:03:45 +03:00
Justin Traglia 2c5648d891
all: fix some typos (#25551)
* Fix some typos

* Fix some mistakes

* Revert 4byte.json

* Fix an incorrect fix

* Change files to fails
2022-08-19 09:00:21 +03:00
rjl493456442 e44d6551c3
cmd, core, ethdb, node: move chain freezer one folder deeper (#25487)
* cmd, core, ethdb, node: create chain freezer in a sub folder

* core/rawdb: remove unused code

* core, ethdb, node: add AncientDatadir API back

* cmd, core: extend freezer info dump for sub-ancient-store

* core/rawdb: rework freezer inspector

* core/rawdb: address comments from Peter

* core/rawdb: fix build issue
2022-08-08 12:08:36 +03:00
Guillaume Ballet f67e54c92f
core: use TryGetAccount to read what TryUpdateAccount has written (#25458)
* core: use TryGetAccount to read where TryUpdateAccount has been used to write

* Gary's review feedback

* implement Gary's suggestion

* fix bug + rename NewSecure into NewStateTrie

* trie: add backwards-compatibility aliases for SecureTrie

* Update database.go

* make the linter happy

Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
2022-08-04 16:13:18 +02:00
Péter Szilágyi 6fd06ab075
cmd, core, eth, les, params: add merge-passed chain config (#24538)
* cmd, core, eth, les, params: add merge-passed chain config

* eth/catalyst, params: add various warning on malfunctioning beacons

* eth/catalyst: fix warning for beacons without transition exchanges
2022-08-01 15:13:25 +03:00
Delweng b196ad1c16
all: add whitespace linter (#25312)
* golangci: typo

Signed-off-by: Delweng <delweng@gmail.com>

* golangci: add whietspace

Signed-off-by: Delweng <delweng@gmail.com>

* *: rm whitesapce using golangci-lint

Signed-off-by: Delweng <delweng@gmail.com>

* cmd/puppeth: revert accidental resurrection

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2022-07-25 13:14:03 +03:00
rjl493456442 1ed8b7d24f
cmd: use flags.Merge for grouping flags (#25392) 2022-07-25 12:07:44 +03:00
Marius van der Wijden 3b2a6b34d9
cmd/geth: eth/catalyst: enable authrpc by default (#25152)
* cmd/geth: eth/catalyst: enable authrpc by default

* eth/catalyst: rename catalyst -> Engine API in logs

* eth/catalyst: don't panic
2022-07-23 09:56:02 +02:00
Guillaume Ballet a7d47ee77b
cmd/geth: remove redundant 0x in dbGet/dbDelete (#25315) 2022-07-18 13:22:56 +02:00
Felix Lange f6ac80c507
cmd/geth, cmd/devp2p: fix some cli parsing issues (#25234)
* cmd/geth: add some missing argument count checks

* internal/flags: skip cmds with no action func in MigrateGlobalFlags

* internal/flags: add Merge

* cmd/devp2p: re-add listener config flags in discv4 commands
2022-07-04 19:52:19 +02:00
Seungbae.yu 2697e44d81
all: change format `0x%x` to `%#x` (#25221) 2022-07-04 11:03:32 +03:00
zeim839 9a5c1000c7
cmd/geth, p2p: add support for custom discovery UDP port (#24979)
This adds a new flag to set the discovery port to be different from
the TCP listener port.

Co-authored-by: Felix Lange <fjl@twurst.com>
2022-06-28 17:25:47 +02:00
willian.eth 52ed3570c4
cmd: migrate to urfave/cli/v2 (#24751)
This change updates our urfave/cli dependency to the v2 branch of the library.
There are some Go API changes in cli v2:

- Flag values can now be accessed using the methods ctx.Bool,
  ctx.Int, ctx.String, ... regardless of whether the flag is 'local' or
  'global'.

- v2 has built-in support for flag categories. Our home-grown category
  system is removed and the categories of flags are assigned as part of
  the flag definition.

For users, there is only one observable difference with cli v2: flags must now
strictly appear before regular arguments. For example, the following command is
now invalid:

   geth account import mykey.json --password file.txt

Instead, the command must be invoked as follows:

   geth account import --password file.txt mykey.json
2022-06-27 18:22:36 +02:00
Sina Mahmoodi f20a569265
cmd/geth: drop geth js command (#25000)
* cmd/geth: drop js command

* cmd: simplify ipc path determination for attach

* Add deprecation warning for js

* rm testdata for exec

* fix account unlock test cases

* Update cmd/geth/consolecmd.go

Co-authored-by: Martin Holst Swende <martin@swende.se>

* fix

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-06-21 12:16:05 +03:00
Marius van der Wijden d8f963811d
cmd, params: implement Gray Glacier hard-fork (EIP-5133) (#25088)
* cmd/geth, params: implement Gray Glacier (EIP-5133)

* cmd/evm: add gray glacier tests

* params: nitpicks

* params: fixes
2022-06-15 14:10:38 +03:00
Martin Holst Swende a907d7e81a
all: more linters (#24783)
This enables the following linters

- typecheck
- unused
- staticcheck
- bidichk
- durationcheck
- exportloopref
- gosec

WIth a few exceptions.

- We use a deprecated protobuf in trezor. I didn't want to mess with that, since I cannot meaningfully test any changes there.
- The deprecated TypeMux is used in a few places still, so the warning for it is silenced for now.
- Using string type in context.WithValue is apparently wrong, one should use a custom type, to prevent collisions between different places in the hierarchy of callers. That should be fixed at some point, but may require some attention.
- The warnings for using weak random generator are squashed, since we use a lot of random without need for cryptographic guarantees.
2022-06-13 16:24:45 +02:00
rjl493456442 22defa5af7
all: introduce trie owner notion (#24750)
* cmd, core/state, light, trie, eth: add trie owner notion

* all: refactor

* tests: fix goimports

* core/state/snapshot: fix ineffasigns

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-06-06 17:14:55 +02:00
Martin Holst Swende c375ee91e9
cmd/geth, core/state/snapshot: rework journal loading, implement account-check (#24765)
* cmd/geth, core/state/snapshot: rework journal loading, implement account-check

* core/state/snapshot, cmd/geth: polish code (#37)

* core/state/snapshot: minor nits

* core/state/snapshot: simplify error logic

* cmd/geth: go format

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
2022-06-06 17:09:39 +02:00
rjl493456442 a10660b7f8
cmd/geth: extend traverseRawState command (#24954)
This PR adds node verification into traverseRawState command, so corrupted trie nodes can also be detected.
2022-05-30 12:37:42 +02:00
Sina Mahmoodi 0559a9a61e
cmd/geth: exit when freezer has legacy receipts (#24943)
In #24028 we flagged a warning when finding legacy receipts in the freezer. This PR nudges users a bit more strongly by preventing geth from starting in this case until receipts have been migrated.

It also adds a flag --ignore-legacy-receipts which when present allows geth to start normally.
2022-05-26 09:22:10 +02:00
Felix Lange 9244d5cd61
all: update license headers and AUTHORS from git history (#24947) 2022-05-24 20:39:40 +02:00
rjl493456442 59ac229f87
core/state/snapshot: detect and clean up dangling storage snapshot in generation (#24811)
* core/state/snapshot: check dangling storages when generating snapshot

* core/state/snapshot: polish

* core/state/snapshot: wipe the last part of the dangling storages

* core/state/snapshot: fix and add tests

* core/state/snapshot: fix comment

* README: remove mentions of fast sync (#24656)

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>

* core, cmd: expose dangling storage detector for wider usage

* core/state/snapshot: rename variable

* core, ethdb: use global iterators for snapshot generation

* core/state/snapshot: polish

* cmd, core/state/snapshot: polish

* core/state/snapshot: polish

* Update core/state/snapshot/generate.go

Co-authored-by: Martin Holst Swende <martin@swende.se>

* ethdb: extend db test suite and fix memorydb iterator

* ethdb/dbtest: rollback changes

* ethdb/memorydb: simplify iteration

* core/state/snapshot: update dangling counter

* core/state/snapshot: release iterators

* core/state/snapshot: update metrics

* core/state/snapshot: update time metrics

* metrics/influxdb: temp solution to present counter meaningfully, remove it

* add debug log, revert later

* core/state/snapshot: fix iterator panic

* all: customized snapshot iterator for backward iteration

* core, ethdb: polish

* core/state/snapshot: remove debug log

* core/state/snapshot: address comments from peter

* core/state/snapshot: reopen the iterator at the next position

* ethdb, core/state/snapshot: address comment from peter

* core/state/snapshot: reopen exhausted iterators

Co-authored-by: Tbnoapi <63448616+nuoomnoy02@users.noreply.github.com>
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-05-23 13:26:22 +03:00
Håvard Anda Estensen 4b309c7006
all: replace non-trivial uses of package ioutil with os (#24886)
Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-05-17 14:27:04 +02:00
Martin Holst Swende e0a9752b96
cmd/geth: add `db check-state-content` to verify integrity of trie nodes (#24840)
This PR adds db tooling (geth db check-state-content)  to verify the integrity of trie nodes. It iterates through the 32-byte key space in the database, which is expected to contain RLP-encoded trie nodes, addressed by hash.
2022-05-17 13:01:46 +02:00
Martin Holst Swende af806168b6
cmd/geth: update vulnerabilities.json testdata (#24856) 2022-05-16 13:41:36 +02:00
Håvard Anda Estensen 07508ac0e9
all: replace uses of ioutil with io and os (#24869) 2022-05-16 11:59:35 +02:00
Marius van der Wijden 4f80f7806e
params: set ropsten TTD for TheMerge (#24876) 2022-05-16 07:32:56 +03:00
||= nil 8a008ee0e6
cmd/geth: print info banner for --dev mode (#24759)
Co-authored-by: nedifi <nedifi@users.noreply.github.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
2022-05-11 00:29:22 +02:00
rjl493456442 1941c5e6c9
core/rawdb: untie freezer and ancient chain data (#24684)
Previously freezer has only been used for storing ancient chain data, while obviously it can be used more. This PR unties the chain data and freezer, keep the minimal freezer structure and move all other logic (like incrementally freezing block data) into a separate structure called ChainFreezer.

This PR also extends the database interface by adding a new ancient store function AncientDatadir which can return the root directory of ancient store. The ancient root directory can be used when we want to open some other ancient-stores (e.g. reverse diff freezer).
2022-05-06 13:28:42 +02:00
Péter Szilágyi ecae8e4f65
cmd, eth: fix required blocks regression 2022-05-04 19:55:17 +03:00
Sina Mahmoodi b3af0a5538
cmd/geth: fix init genesis for dev (#24693)
* cmd/geth: fix init genesis for dev

* use ancient flag for init genesis cmd
2022-05-04 16:14:14 +02:00
Martin Holst Swende 53304ff6c7
cmd/utils: double limit on free-disk monitor (#24781)
This PR doubles the limit on which to trigger automatic shutdown, and also changes the timer to run once every 30s instead of 60s.
2022-05-03 12:12:40 +02:00
rjl493456442 344d6f95cf
cmd: group network and db path flags together (#24698)
This PR groups all built-in network flags together and list them in the command as a whole.

And all database path flags(datadir, ancient) are also grouped, since usually these two are
used together.
2022-05-03 08:46:17 +02:00
s7v7nislands 5a584c2133
all: use common.FileExist for checking file existence (#24748) 2022-04-27 11:48:02 +02:00
Sina Mahmoodi 16701c5169
internal/ethapi: add db operations to api (#24739)
Adds `debug_dbGet` method to rpc api
2022-04-27 08:37:48 +02:00
Martin Holst Swende 0914234d10
cmd/geth, core/state/snapshot: fix flaw in dangling-storage check + inspect difflayers (#24677)
This PR fixes the flaw that @rjl493456442 found in https://github.com/ethereum/go-ethereum/pull/#issuecomment-1093817551 , namely, that the snapshot iterator uses the combined (disk + difflayers) 'view', wheres the raw iterator uses only the disk 'view'.

This PR instead splits up the work: one phase is iterating the disk layer data, another phase is loading the journalled difflayers and performing the same check there.
2022-04-26 09:08:43 +02:00