Commit Graph

13305 Commits

Author SHA1 Message Date
Péter Szilágyi 6ef3a16869
p2p/enode: use unix timestamp as base ENR sequence number (#19903)
This PR ensures that wiping all data associated with a node (apart from its nodekey)
will not generate already used sequence number for the ENRs, since all remote nodes
would reject them until they out-number the previously published largest one.

The big complication with this scheme is that every local update to the ENR can
potentially bump the sequence number by one. In order to ensure that local updates
do not outrun the clock, the sequence number is a millisecond-precision timestamp,
and updates are throttled to occur at most once per millisecond.

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-09-07 12:36:48 +02:00
Martin Holst Swende 794c6133ef
core/rawdb: freezer batch write (#23462)
This change is a rewrite of the freezer code.

When writing ancient chain data to the freezer, the previous version first encoded each
individual item to a temporary buffer, then wrote the buffer. For small item sizes (for
example, in the block hash freezer table), this strategy causes a lot of system calls for
writing tiny chunks of data. It also allocated a lot of temporary []byte buffers.

In the new version, we instead encode multiple items into a re-useable batch buffer, which
is then written to the file all at once. This avoids performing a system call for every
inserted item.

To make the internal batching work, the ancient database API had to be changed. While
integrating this new API in BlockChain.InsertReceiptChain, additional optimizations were
also added there.

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-09-07 12:31:17 +02:00
Zachinquarantine 9a0df80bbc
cmd/geth, cmd/utils: remove legacy rpc flags (#23358)
* remove rpc flags

* remove legacy rpc flags

* remove legacy rpc flags

* remove legacy rpc commands

* (hopefully) fix most of the build errors

* fix build errors

https://app.travis-ci.com/github/ethereum/go-ethereum/jobs/530318686

* cmd/utils: fix syntax error

* empty commit to unbreak travis ci

* fix syntax error

* syntax fixes

* syntax fixes

* fix

fixes "cmd/geth/usage.go:234:7: expected '(', found init (typecheck)"

* fix

* various fixes in usage.go

* various fixes in flags.go

* adds extra space

reverts the spacing to how it was before I resolved the merge conflict

* more fixes in usage.go

* fix

fix for cmd/geth/usage.go:243:17: expected operand, found ':=' (typecheck) in travis

* Update cmd/utils/flags.go

Co-authored-by: Martin Holst Swende <martin@swende.se>

* fix error

 fixes these errors:
cmd/utils/flags_legacy.go:21:2: "strings" imported but not used (typecheck)

	"strings"

	^

cmd/utils/flags_legacy.go:24:2: "github.com/ethereum/go-ethereum/node" imported but not used (typecheck)

	"github.com/ethereum/go-ethereum/node"

	^

* goimports

Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-09-07 10:52:58 +02:00
Zachinquarantine ca5bc676d1
internal/debug: remove deprecated flags (#23368)
* internal/debug: remove deprecated flags

The removed flags are removed in the main portion of geth, this removes it internally too.

* internal/debug: remove legacy --debug and legacy --backtrace flag

* Update flags.go

Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-09-07 10:51:33 +02:00
Martin Holst Swende 7957530225
docs: add post-mortem (#23518)
* docs: add post-mortem

* Update docs/postmortems/2021-08-22-split-postmortem.md

Co-authored-by: Sina Mahmoodi <1591639+s1na@users.noreply.github.com>

* Update docs/postmortems/2021-08-22-split-postmortem.md

Co-authored-by: Sina Mahmoodi <1591639+s1na@users.noreply.github.com>

* Update docs/postmortems/2021-08-22-split-postmortem.md

Co-authored-by: Sina Mahmoodi <1591639+s1na@users.noreply.github.com>

* Update docs/postmortems/2021-08-22-split-postmortem.md

* Update 2021-08-22-split-postmortem.md

* Update docs/postmortems/2021-08-22-split-postmortem.md

Co-authored-by: Sina Mahmoodi <1591639+s1na@users.noreply.github.com>

Co-authored-by: Sina Mahmoodi <1591639+s1na@users.noreply.github.com>
2021-09-07 09:22:40 +02:00
aaronbuchwald de2c44ab5c
trie: add missing copyright header (#23530)
* Add missing copyright header

* Update stacktrie_test.go

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2021-09-07 07:25:54 +03:00
Martin Holst Swende 4d88974864
cmd/evm: add tests for evm t8n (#23507) 2021-09-02 09:22:43 +02:00
Marius van der Wijden 067084feda
core: fix race conditions in txpool (#23474)
* core: fix race conditions in txpool

* core: fixed races in the txpool

* core: rebased on master

* core: move reheap mutex

* core: renamed mutex

* core: revert Reheap changes
2021-08-31 10:33:39 +02:00
Alexander Yastrebov d019e90162
common: fixes format verb (#23495) 2021-08-31 10:22:20 +02:00
Ikko Ashimine 31be5d41d9
core: fix typo in iterator.go (#23502)
strorage -> storage
2021-08-31 10:21:42 +02:00
Fredrik Svantes f85cf722ff
docs: add links to more audits in SECURITY.md (#23482) 2021-08-31 09:02:04 +02:00
Paweł Bylica 3258211f68
core/vm: rework jumpdest analysis benchmarks (#23499)
* core/vm: rework jumpdest analysis benchmarks

For BenchmarkJumpdestOpAnalysis use fixed code size of ~1.2MB
and classic benchmark loop.

* core/vm: clear bitvec in jumpdest analysis benchmark
2021-08-30 14:13:06 +02:00
lightclient ffae2043f0
internal/ethapi: support both input and data for personal_sendTransaction (#23476)
Currently, setDefaults overwrites the transaction input value if only input is provided. This causes personal_sendTransaction to estimate the gas based on a transaction with empty data. eth_estimateGas never calls setDefaults so it was unaffected by this.
2021-08-27 14:11:15 +02:00
Martin Holst Swende 62ad17fb00
Revert "eth, internal/ethapi: make RPC block miner field show block sealer correctly (#23312)" (#23466)
This reverts commit 57feabea66.
2021-08-26 08:35:37 +02:00
Sina Mahmoodi 108eec3fee
node: remove dependency on wallet backend packages (#23019)
* accounts: new AddBackends method in manager

* node,cmd/geth: mv accman backend init to cmd/geth

* node,cmd/geth: mv scrypt config downstreawm from node

* accounts: use static buffer size for accman sub chan

minor fix

* accounts,cmd/geth: update accman backends through its event loop

* accounts,node: add comments

* accounts: un-export newBackendEvent

* accounts: use chan instead of wg in newBlockEvent

* node: rename isKeyDirEphem

* accounts,cmd: AddBackends->AddBackend

* accounts: fix potential blocking when adding backend
2021-08-25 22:34:22 +02:00
Felix Lange d584e39862
build: stop publishing deb packages for Ubuntu 20.10 Groovy Gorilla (#23470)
Support for this Ubuntu release has ended on July 22 2021 and
Launchpad no longer accepts uploads for it.
2021-08-25 21:57:31 +02:00
Dmitry Zenovich 7c4cad064c
rpc: add BlockNumber.MarshalText (#23324)
Currently rpc.BlockNumber is marshalled to JSON as a numeric value, which is
wrong because BlockNumber.UnmarshalJSON() wants it to either be hex-encoded
or string "earliest"/"latest"/"pending". As a result, the call chain

    rpc.BlockNumberOrHashWithNumber(123) -> json.Marshal() -> json.Unmarshal()

fails with error "cannot unmarshal object into Go value of type string".
2021-08-25 19:30:29 +02:00
baptiste-b-pegasys 154b525ce8
cmd/devp2p/internal/ethtest: remove WriteTo method (#23379)
WriteTo is unused, and go vet warns about it because it
doesn't match the usual signature of the io.WriterTo method.
2021-08-25 19:06:42 +02:00
Felix Lange 32c576bd3c
rlp: minor optimizations for slice/array encoding (#23467)
As per benchmark results below, these changes speed up encoding/decoding of
consensus objects a bit.

    name                             old time/op    new time/op    delta
    EncodeRLP/legacy-header-8           384ns ± 1%     331ns ± 3%  -13.83%  (p=0.000 n=7+8)
    EncodeRLP/london-header-8           411ns ± 1%     359ns ± 2%  -12.53%  (p=0.000 n=8+8)
    EncodeRLP/receipt-for-storage-8     251ns ± 0%     239ns ± 0%   -4.97%  (p=0.000 n=8+8)
    EncodeRLP/receipt-full-8            319ns ± 0%     300ns ± 0%   -5.89%  (p=0.000 n=8+7)
    EncodeRLP/legacy-transaction-8      389ns ± 1%     387ns ± 1%     ~     (p=0.099 n=8+8)
    EncodeRLP/access-transaction-8      607ns ± 0%     581ns ± 0%   -4.26%  (p=0.000 n=8+8)
    EncodeRLP/1559-transaction-8        627ns ± 0%     606ns ± 1%   -3.44%  (p=0.000 n=8+8)
    DecodeRLP/legacy-header-8           831ns ± 1%     813ns ± 1%   -2.20%  (p=0.000 n=8+8)
    DecodeRLP/london-header-8           824ns ± 0%     804ns ± 1%   -2.44%  (p=0.000 n=8+7)

* rlp: pass length to byteArrayBytes

This makes it possible to inline byteArrayBytes. For arrays, the length is known
at encoder construction time, so the call to v.Len() can be avoided.

* rlp: avoid IsNil for pointer encoding

It's actually cheaper to use Elem first, because it performs less checks
on the value. If the pointer was nil, the result of Elem is 'invalid'.

* rlp: minor optimizations for slice/array encoding

For empty slices/arrays, we can avoid storing a list header entry in the
encoder buffer. Also avoid doing the tail check at encoding time because
it is already known at encoder construction time.
2021-08-25 19:01:10 +02:00
Felix Lange 8a134014b4
all: add go:build lines (#23468)
Generated by go1.17 fmt ./...
2021-08-25 18:46:29 +02:00
Guillaume Ballet 887902ea4d
crypto/cloudflare/bn256: fix in-place addition and unmarshalling (#23419) 2021-08-25 17:33:09 +02:00
Felix Lange d16214228f
Dockerfile: upgrade to Go 1.17 (#23465) 2021-08-25 15:18:31 +02:00
Martin Holst Swende 3784e15106
tests: avoid performance tests during CI runs (#23304) 2021-08-25 15:07:58 +02:00
Martin Holst Swende 84c51bc5ec
cmd/evm: add difficulty calculation to t8n tool (#23353)
This PR adds functionality to the evm t8n to calculate ethash difficulty. If the caller does not provide a currentDifficulty, but instead provides the parentTimestamp (well, semi-optional, will default to 0 if not given), and parentDifficulty, we can calculate it for him.

The caller can also provide a parentUncleHash. In most, but not all cases, the parent uncle hash also affects the formula. If no such hash is provided (or, if the empty all-zero hash is provided), it's assumed that there were no uncles.
2021-08-25 15:04:26 +02:00
Felix Lange efee85378e
build, .travis.yml: upgrade to Go 1.17 (#23464) 2021-08-25 14:22:33 +02:00
Felix Lange 45f34430fd
build: upgrade to golangci-lint v1.42.0 (#23461)
The new linter version is built with go 1.17 and thus includes the go vet
check for mismatched +build and go:build lines.

Fortunately, no new warnings are reported with this update.
2021-08-25 13:21:09 +02:00
gary rong 83ad92c421
les: fix data races in tests (#23457) 2021-08-25 11:56:25 +02:00
ucwong fe2f153b55
go.mod: update goleveldb (#23417) 2021-08-24 22:02:46 +03:00
gary rong a5a5237178
core, light, tests, trie: add state metrics (#23433) 2021-08-24 22:00:42 +03:00
Felix Lange a789dcc978
metrics: fix compilation for GOOS=js (#23449) 2021-08-24 21:54:55 +03:00
Ivan Bogatyy b69f5ca7d4
cmd/utils: fix typo in variable name (#23451) 2021-08-24 21:54:00 +03:00
Péter Szilágyi 0db0b27754
Revert "Revert "eth: drop eth/65, the last non-reqid protocol version" (#23426)" (#23456)
This reverts commit c368f728c1.
2021-08-24 21:52:58 +03:00
Martin Holst Swende d705f5a554
core: make txpool reject too sudden changes (#23095)
* core: make txpool reject too sudden changes

* core: add some metrics to txpool
2021-08-24 21:48:36 +03:00
baptiste-b-pegasys 5cee33eb72
tests/fuzzers: fix go vet warning about ReadByte (#23380) 2021-08-24 14:27:15 +02:00
Zachinquarantine 85126c4eb9
node: add comment about --nousb being deprecated (#23439)
Co-authored-by: Felix Lange <fjl@twurst.com>
2021-08-24 14:10:52 +02:00
Martin Holst Swende a0a4a153e9
core/types: add benchmarks for rlp encoding/decoding (#23190)
Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Sina Mahmoodi <1591639+s1na@users.noreply.github.com>
2021-08-24 13:59:15 +02:00
Guillaume Ballet 0b40977480
core/vm: fix typo in comment (#23450) 2021-08-24 13:57:05 +02:00
ligi 5c66bab3b8
tests/testdata: make submodule shallow (#23412)
Before:

316M .git/modules/tests/

After:

171M .git/modules/tests/
2021-08-24 13:56:31 +02:00
Pedro Gomes 8e0771c218
core/bloombits: avoid crash when storing errors of different type (#23437)
This fixes a rare crash which could occur when two different errors happened
in the same bloombits.MatcherSession.
2021-08-24 13:32:19 +02:00
Marius van der Wijden 8dbf261fd9
p2p, p2p/enode: fix data races (#23434)
In p2p/dial.go, conn.flags was accessed without using sync/atomic.
This race is fixed by removing the access.

In p2p/enode/iter_test.go, a similar race is resolved by writing the field atomically.

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-08-24 12:22:56 +02:00
chuwt 79bb9300c1
trie, les: add missing calls to Ticker.Stop (#23415) 2021-08-24 10:51:28 +02:00
Péter Szilágyi ea4bc2dbff
params: begin Geth v1.10.9 release cycle 2021-08-24 10:08:08 +03:00
Péter Szilágyi 26675454bf
params: release Geth v1.10.8 2021-08-24 10:04:30 +03:00
Martin Holst Swende 1d99573192
core/vm: faster code analysis (#23381)
* core/vm: more detailed benchmark for jumpdest analysis

* core/vm: make jd analysis benchmark alloc free

* core/vm: improve jumpdest analysis

* core/vm: improve worst-case

* core/vm: further improvements in analysis

* core/vm: improve jumpdest analysis >PUSH15

* core/vm: make jd analysis ref by value

* core/vm: fix misspell

* core/vm: improve set8 and set16 a bit

* core/vm: reduce amount of code

* core/vm: optimize byte copying
2021-08-24 10:00:10 +03:00
Felföldi Zsolt f38abc55f1
eth/gasprice: feeHistory improvements (#23422)
* eth/gasprice: cache feeHistory results

* eth/gasprice: changed feeHistory block count limitation

* eth/gasprice: do not use embedded struct in blockFees

* eth/gasprice: fee processing logic cleanup

* eth/gasprice: purge feeHistory cache at chain reorgs
2021-08-23 23:50:24 +02:00
ucwong dfeb2f7e80
go.mod: upgrade golang.org/x/sys for go1.17 support (#23406) 2021-08-23 13:45:16 +02:00
Felix Lange bb1f7ebf20
signer/core/apitypes: remove dependency on internal/ethapi (#23362) 2021-08-23 12:20:56 +02:00
Marius van der Wijden d02c605367
core: only check sendernoeoa in non fake mode (#23424) 2021-08-23 12:49:39 +03:00
Péter Szilágyi c368f728c1
Revert "eth: drop eth/65, the last non-reqid protocol version" (#23426) 2021-08-20 15:14:21 +03:00
chuwt 5566e5d152
eth/downloader: fix typo in comment (#23413) 2021-08-18 13:03:41 +03:00