Commit Graph

41 Commits

Author SHA1 Message Date
gary rong f566dd305e
all: bloom-filter based pruning mechanism (#21724)
* cmd, core, tests: initial state pruner

core: fix db inspector

cmd/geth: add verify-state

cmd/geth: add verification tool

core/rawdb: implement flatdb

cmd, core: fix rebase

core/state: use new contract code layout

core/state/pruner: avoid deleting genesis state

cmd/geth: add helper function

core, cmd: fix extract genesis

core: minor fixes

contracts: remove useless

core/state/snapshot: plugin stacktrie

core: polish

core/state/snapshot: iterate storage concurrently

core/state/snapshot: fix iteration

core: add comments

core/state/snapshot: polish code

core/state: polish

core/state/snapshot: rebase

core/rawdb: add comments

core/rawdb: fix tests

core/rawdb: improve tests

core/state/snapshot: fix concurrent iteration

core/state: run pruning during the recovery

core, trie: implement martin's idea

core, eth: delete flatdb and polish pruner

trie: fix import

core/state/pruner: add log

core/state/pruner: fix issues

core/state/pruner: don't read back

core/state/pruner: fix contract code write

core/state/pruner: check root node presence

cmd, core: polish log

core/state: use HEAD-127 as the target

core/state/snapshot: improve tests

cmd/geth: fix verification tool

cmd/geth: use HEAD as the verification default target

all: replace the bloomfilter with martin's fork

cmd, core: polish code

core, cmd: forcibly delete state root

core/state/pruner: add hash64

core/state/pruner: fix blacklist

core/state: remove blacklist

cmd, core: delete trie clean cache before pruning

cmd, core: fix lint

cmd, core: fix rebase

core/state: fix the special case for clique networks

core/state/snapshot: remove useless code

core/state/pruner: capping the snapshot after pruning

cmd, core, eth: fixes

core/rawdb: update db inspector

cmd/geth: polish code

core/state/pruner: fsync bloom filter

cmd, core: print warning log

core/state/pruner: adjust the parameters for bloom filter

cmd, core: create the bloom filter by size

core: polish

core/state/pruner: sanitize invalid bloomfilter size

cmd: address comments

cmd/geth: address comments

cmd/geth: address comment

core/state/pruner: address comments

core/state/pruner: rename homedir to datadir

cmd, core: address comments

core/state/pruner: address comment

core/state: address comments

core, cmd, tests: address comments

core: address comments

core/state/pruner: release the iterator after each commit

core/state/pruner: improve pruner

cmd, core: adjust bloom paramters

core/state/pruner: fix lint

core/state/pruner: fix tests

core: fix rebase

core/state/pruner: remove atomic rename

core/state/pruner: address comments

all: run go mod tidy

core/state/pruner: avoid false-positive for the middle state roots

core/state/pruner: add checks for middle roots

cmd/geth: replace crit with error

* core/state/pruner: fix lint

* core: drop legacy bloom filter

* core/state/snapshot: improve pruner

* core/state/snapshot: polish concurrent logs to report ETA vs. hashes

* core/state/pruner: add progress report for pruning and compaction too

* core: fix snapshot test API

* core/state: fix some pruning logs

* core/state/pruner: support recovering from bloom flush fail

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2021-02-08 13:16:30 +02:00
Marius van der Wijden 54735a6723
fuzzers: added consensys/gurvy library to bn256 differential fuzzer (#21812)
This pr adds consensys' gurvy bn256 variant into the code for differential fuzzing.
2021-02-03 15:04:28 +01:00
Felix Lange ad038b6289
accounts/scwallet: use go-ethereum crypto instead of go-ecdh (#22212)
* accounts/scwallet: use go-ethereum crypto instead of go-ecdh

github.com/wsddn/go-ecdh is a wrapper package for ECDH functionality
with any elliptic curve.

Since 'generic' ECDH is not required in accounts/scwallet (the curve is
always secp256k1), we can just use the standard library functionality
and our own crypto libraries to perform ECDH and save a dependency.

* Update accounts/scwallet/securechannel.go

Co-authored-by: Guillaume Ballet <gballet@gmail.com>

* Use the correct key

Co-authored-by: Guillaume Ballet <gballet@gmail.com>
2021-01-26 16:01:13 +01:00
ucwong 59a79137b9
go.mod: upgrade github.com/huin/goupnp (#22227)
This updates the goupnp dependency, fixing huin/goupnp#33
2021-01-25 12:46:09 +01:00
Marius van der Wijden 1770fe718a
go.mod: update dependencies (#22216)
This updates go module dependencies as discussed in #22050.
2021-01-25 10:42:07 +01:00
Felix Lange f26c19cbcd
common/mclock: remove dependency on github.com/aristanetworks/goarista (#22211)
It takes three lines of code to get to runtime.nanotime, no need to
pull a dependency for that.
2021-01-22 20:15:27 +01:00
ucwong 7da8f75d5b
go.mod: upgrade golang-lru (#22134) 2021-01-20 19:34:21 +01:00
Martin Holst Swende 93a89b2681
go.mod: use github.com/holiman/bloomfilter/v2 (#22044)
* deps: use improved bloom filter implementation

* eth/handler, trie: use 4 keys for syncbloom + minor fixes

* eth/protocols, trie: revert change on syncbloom method signature
2021-01-12 17:39:31 +01:00
Martin Holst Swende 0b2f1446bb
go.mod: update github.com/golang/snappy(#21934)
This updates the snappy library depency to include a fix for
a Go 1.16 incompatibility issue.
2020-12-02 16:42:38 +01:00
Guillaume Ballet fa572cd297
crypto: signing builds with signify/minisign (#21798)
* internal/build: implement signify's signing func
* Add signify to the ci utility
* fix output file format
* Add unit test for signify
* holiman's + travis' feedback
* internal/build: verify signify's output
* crypto: move signify to common dir
* use go-minisign to verify binaries
* more holiman feedback
* crypto, ci: support minisign output
* only accept one-line trusted comments
* configurable untrusted comments
* code cleanup in tests
* revert to use ed25519 from the stdlib
* bug: fix for empty untrusted comments
* write timestamp as comment if trusted comment isn't present
* rename line checker to commentHasManyLines
* crypto: added signify fuzzer (#6)
* crypto: added signify fuzzer
* stuff
* crypto: updated signify fuzzer to fuzz comments
* crypto: repro signify crashes
* rebased fuzzer on build-signify branch
* hide fuzzer behind gofuzz build flag
* extract key data inside a single function
* don't treat \r as a newline
* travis: fix signing command line
* do not use an external binary in tests
* crypto: move signify to crypto/signify
* travis: fix formatting issue
* ci: fix linter build after package move

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
2020-11-27 12:13:54 +01:00
Marius van der Wijden 420b78659b
accounts/abi: ABI explicit difference between Unpack and UnpackIntoInterface (#21091)
* accounts/abi: refactored abi.Unpack

* accounts/abi/bind: fixed error

* accounts/abi/bind: modified template

* accounts/abi/bind: added ToStruct for conversion

* accounts/abi: reenabled tests

* accounts/abi: fixed tests

* accounts/abi: fixed tests for packing/unpacking

* accounts/abi: fixed tests

* accounts/abi: added more logic to ToStruct

* accounts/abi/bind: fixed template

* accounts/abi/bind: fixed ToStruct conversion

* accounts/abi/: removed unused code

* accounts/abi: updated template

* accounts/abi: refactored unused code

* contracts/checkpointoracle: updated contracts to sol ^0.6.0

* accounts/abi: refactored reflection logic

* accounts/abi: less code duplication in Unpack*

* accounts/abi: fixed rebasing bug

* fix a few typos in comments

* rebase on master

Co-authored-by: Guillaume Ballet <gballet@gmail.com>
2020-09-28 14:10:26 +02:00
Marius van der Wijden 2c097bb7a2
mobile: better api for java users (#21580)
* (mobile): Adds string representations for types

* mobile: better interfaces add stringer to types

Co-authored-by: sarath <sarath@melvault.com>
2020-09-21 16:33:35 +02:00
Marius van der Wijden 89884dc353
tests/fuzzers/abi: add fuzzer for fuzzing package accounts/abi (#21217)
* tests/fuzzers/abi: added abi fuzzer

* accounts/abi: fixed issues found by fuzzing

* tests/fuzzers/abi: update fuzzers, added repro test

* tests/fuzzers/abi: renamed abi_fuzzer to abifuzzer

* tests/fuzzers/abi: updated abi fuzzer

* tests/fuzzers/abi: updated abi fuzzer

* accounts/abi: minor style fix

* go.mod: added go-fuzz dependency

* tests/fuzzers/abi: updated abi fuzzer

* tests/fuzzers/abi: make linter happy

* tests/fuzzers/abi: make linter happy

* tests/fuzzers/abi: comment out false positives
2020-09-16 13:15:22 +02:00
Shude Li 4ea9737de6
go.mod: remove golang.org/x/sync (#21541) 2020-09-10 14:21:51 +02:00
ucwong d90bbce954
go.mod : update goja dependency (#21432) 2020-09-01 12:56:22 +02:00
ucwong 856307d8bb
go.mod | goleveldb latest update (#21448)
* go.mod | goleveldb latest update

* go.mod update

* leveldb options

* go.mod: double check

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2020-08-26 15:53:12 +02:00
Martin Holst Swende 56434bfa89
deps: update uint256 to v1.1.1 2020-07-24 14:00:08 +02:00
ucwong 7a556abe15
go.mod: upgrade to github.com/golang/snappy with arm64 asm (#21304) 2020-07-08 14:06:53 +02:00
rene 1e635bd0bd
go.mod: updated crypto deps causing build failure (#21276) 2020-06-30 16:05:59 +02:00
Guillaume Ballet b86b1e6d43
go.mod: bump gopsutil version (#21275) 2020-06-30 15:22:21 +02:00
Guillaume Ballet 9a188c975d
common/fdlimit: build on DragonflyBSD (#21241)
* common/fdlimit: build on DragonflyBSD

* review feedback
2020-06-19 15:43:52 +02:00
ucwong e029cc6616
go.mod: update snappy dependency (#21237) 2020-06-18 14:01:49 +03:00
Martin Holst Swende c85fdb76ee
go.mod: update uint256 to 1.1.0 (#21206) 2020-06-11 07:27:43 +03:00
Paweł Bylica cf6674539c
core/vm: use uint256 in EVM implementation (#20787)
* core/vm: use fixed uint256 library instead of big

* core/vm: remove intpools

* core/vm: upgrade uint256, fixes uint256.NewFromBig

* core/vm: use uint256.Int by value in Stack

* core/vm: upgrade uint256 to v1.0.0

* core/vm: don't preallocate space for 1024 stack items (only 16)

Co-authored-by: Martin Holst Swende <martin@swende.se>
2020-06-08 15:24:40 +03:00
ucwong 4b2ff1457a
go.mod: upgrade go-duktape to hide unused function warning (#21168) 2020-06-04 17:42:05 +02:00
rene a35382de94
metrics: replace gosigar with gopsutil (#21041)
* replace gosigar with gopsutil

* removed check for whether GOOS is openbsd

* removed accidental import of runtime

* potential fix for difference in units between gosig and gopsutil

* fixed lint error

* remove multiplication factor

* uses cpu.ClocksPerSec as the multiplication factor

* changed dependency from shirou to renaynay (#20)

* updated dep

* switching back from using renaynay fork to using upstream as PRs were merged on upstream

* removed empty line

* optimized imports

* tidied go mod
2020-06-02 12:08:33 +03:00
ucwong 1264c19f11
go.mod : goupnp v1.0.0 upgrade (#20996) 2020-04-28 14:57:07 +03:00
ucwong 1893266c59
go.mod: upgrade to golang-lru v0.5.4 (#20992)
golang-lru is now a go module, and the upgrade corrects a couple
of minor issues. In particular, the library could crash if you inserted
nil into an LRU cache.
2020-04-28 10:22:23 +02:00
ucwong c60c0c97e7
go.mod : update fastcache to 1.5.7 (#20936) 2020-04-22 10:31:32 +03:00
Alex Willmer 5dd0cd12ec
go.mod: update duktape to fix sprintf warnings (#20777)
This revision of go-duktype fixes the following warning

```
duk_logging.c: In function ‘duk__logger_prototype_log_shared’:
duk_logging.c:184:64: warning: ‘Z’ directive writing 1 byte into a region of size between 0 and 9 [-Wformat-overflow=]
  184 |  sprintf((char *) date_buf, "%04d-%02d-%02dT%02d:%02d:%02d.%03dZ",
      |                                                                ^
In file included from /usr/include/stdio.h:867,
                 from duk_logging.c:5:
/usr/include/x86_64-linux-gnu/bits/stdio2.h:36:10: note: ‘__builtin___sprintf_chk’ output between 25 and 85 bytes into a destination of size 32
   36 |   return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL - 1,
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   37 |       __bos (__s), __fmt, __va_arg_pack ());
      |       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
2020-03-18 11:52:07 +02:00
Péter Szilágyi 466b009135
go.mod: update golang.org/x/crypto to fix a Go 1.14 race rejection 2020-03-14 14:09:31 +02:00
Guillaume Ballet 6df973df27
go.mod: upgrade goja to latest (#20700)
The new goja version supports the 'escape' and 'unescape' built-in functions.
This fixes #20693
2020-02-20 12:46:47 +01:00
Guillaume Ballet 594e038e75
signer/rules: use goja and remove otto (#20599)
* signer: replace otto with goja

* go.mod: remove Otto
2020-01-29 13:47:56 +01:00
Guillaume Ballet 7b68975a00 console, internal/jsre: use github.com/dop251/goja (#20470)
This replaces the JavaScript interpreter used by the console with goja,
which is actively maintained and a lot faster than otto. Clef still uses otto
and eth/tracers still uses duktape, so we are currently dependent on three
different JS interpreters. We're looking to replace the remaining uses of otto
soon though.
2020-01-27 11:50:48 +01:00
Felix Lange f51cf573b5
cmd/devp2p: implement AWS Route53 enrtree deployer (#20446) 2019-12-12 22:25:12 +01:00
Felix Lange 191364c350 p2p/dnsdisc: add enode.Iterator API (#20437)
* p2p/dnsdisc: add support for enode.Iterator

This changes the dnsdisc.Client API to support the enode.Iterator
interface.

* p2p/dnsdisc: rate-limit DNS requests

* p2p/dnsdisc: preserve linked trees across root updates

This improves the way links are handled when the link root changes.
Previously, sync would simply remove all links from the current tree and
garbage-collect all unreachable trees before syncing the new list of
links.

This behavior isn't great in certain cases: Consider a structure where
trees A, B, and C reference each other and D links to A. If D's link
root changed, the sync code would first remove trees A, B and C, only to
re-sync them later when the link to A was found again.

The fix for this problem is to track the current set of links in each
clientTree and removing old links only AFTER all links are synced.

* p2p/dnsdisc: deflake iterator test

* cmd/devp2p: adapt dnsClient to new p2p/dnsdisc API

* p2p/dnsdisc: tiny comment fix
2019-12-12 11:15:36 +02:00
Martin Holst Swende 7800ba978d
deps: update fastcache to 1.5.3 2019-11-27 13:46:07 +01:00
gary rong 9d67222f4e trie: replace bigcache with fastcache (#19971) 2019-11-25 10:58:15 +02:00
Felix Lange dd21f079e8
core/state: fix staticcheck warnings (#20357)
Also remove dependency on gopkg.in/check.v1 in tests.
2019-11-22 15:56:05 +01:00
Péter Szilágyi e1dcea8bf0
go.mod: tidy up the modules to avoid xgo writes to go.sum 2019-11-20 14:42:34 +02:00
Péter Szilágyi 15fb780de6
go.mod, vendor: switch to Go modules 2019-11-20 14:42:28 +02:00