Commit Graph

94 Commits

Author SHA1 Message Date
gary rong f566dd305e
all: bloom-filter based pruning mechanism (#21724)
* cmd, core, tests: initial state pruner

core: fix db inspector

cmd/geth: add verify-state

cmd/geth: add verification tool

core/rawdb: implement flatdb

cmd, core: fix rebase

core/state: use new contract code layout

core/state/pruner: avoid deleting genesis state

cmd/geth: add helper function

core, cmd: fix extract genesis

core: minor fixes

contracts: remove useless

core/state/snapshot: plugin stacktrie

core: polish

core/state/snapshot: iterate storage concurrently

core/state/snapshot: fix iteration

core: add comments

core/state/snapshot: polish code

core/state: polish

core/state/snapshot: rebase

core/rawdb: add comments

core/rawdb: fix tests

core/rawdb: improve tests

core/state/snapshot: fix concurrent iteration

core/state: run pruning during the recovery

core, trie: implement martin's idea

core, eth: delete flatdb and polish pruner

trie: fix import

core/state/pruner: add log

core/state/pruner: fix issues

core/state/pruner: don't read back

core/state/pruner: fix contract code write

core/state/pruner: check root node presence

cmd, core: polish log

core/state: use HEAD-127 as the target

core/state/snapshot: improve tests

cmd/geth: fix verification tool

cmd/geth: use HEAD as the verification default target

all: replace the bloomfilter with martin's fork

cmd, core: polish code

core, cmd: forcibly delete state root

core/state/pruner: add hash64

core/state/pruner: fix blacklist

core/state: remove blacklist

cmd, core: delete trie clean cache before pruning

cmd, core: fix lint

cmd, core: fix rebase

core/state: fix the special case for clique networks

core/state/snapshot: remove useless code

core/state/pruner: capping the snapshot after pruning

cmd, core, eth: fixes

core/rawdb: update db inspector

cmd/geth: polish code

core/state/pruner: fsync bloom filter

cmd, core: print warning log

core/state/pruner: adjust the parameters for bloom filter

cmd, core: create the bloom filter by size

core: polish

core/state/pruner: sanitize invalid bloomfilter size

cmd: address comments

cmd/geth: address comments

cmd/geth: address comment

core/state/pruner: address comments

core/state/pruner: rename homedir to datadir

cmd, core: address comments

core/state/pruner: address comment

core/state: address comments

core, cmd, tests: address comments

core: address comments

core/state/pruner: release the iterator after each commit

core/state/pruner: improve pruner

cmd, core: adjust bloom paramters

core/state/pruner: fix lint

core/state/pruner: fix tests

core: fix rebase

core/state/pruner: remove atomic rename

core/state/pruner: address comments

all: run go mod tidy

core/state/pruner: avoid false-positive for the middle state roots

core/state/pruner: add checks for middle roots

cmd/geth: replace crit with error

* core/state/pruner: fix lint

* core: drop legacy bloom filter

* core/state/snapshot: improve pruner

* core/state/snapshot: polish concurrent logs to report ETA vs. hashes

* core/state/pruner: add progress report for pruning and compaction too

* core: fix snapshot test API

* core/state: fix some pruning logs

* core/state/pruner: support recovering from bloom flush fail

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2021-02-08 13:16:30 +02:00
Marius van der Wijden 54735a6723
fuzzers: added consensys/gurvy library to bn256 differential fuzzer (#21812)
This pr adds consensys' gurvy bn256 variant into the code for differential fuzzing.
2021-02-03 15:04:28 +01:00
rene 4eae0c6b6f
cmd/geth, node: allow configuring JSON-RPC on custom path prefix (#22184)
This change allows users to set a custom path prefix on which to mount the http-rpc
or ws-rpc handlers via the new flags --http.rpcprefix and --ws.rpcprefix.

Fixes #21826

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-02-02 10:05:46 +01:00
Felix Lange ad038b6289
accounts/scwallet: use go-ethereum crypto instead of go-ecdh (#22212)
* accounts/scwallet: use go-ethereum crypto instead of go-ecdh

github.com/wsddn/go-ecdh is a wrapper package for ECDH functionality
with any elliptic curve.

Since 'generic' ECDH is not required in accounts/scwallet (the curve is
always secp256k1), we can just use the standard library functionality
and our own crypto libraries to perform ECDH and save a dependency.

* Update accounts/scwallet/securechannel.go

Co-authored-by: Guillaume Ballet <gballet@gmail.com>

* Use the correct key

Co-authored-by: Guillaume Ballet <gballet@gmail.com>
2021-01-26 16:01:13 +01:00
ucwong 59a79137b9
go.mod: upgrade github.com/huin/goupnp (#22227)
This updates the goupnp dependency, fixing huin/goupnp#33
2021-01-25 12:46:09 +01:00
Marius van der Wijden 1770fe718a
go.mod: update dependencies (#22216)
This updates go module dependencies as discussed in #22050.
2021-01-25 10:42:07 +01:00
Felix Lange f26c19cbcd
common/mclock: remove dependency on github.com/aristanetworks/goarista (#22211)
It takes three lines of code to get to runtime.nanotime, no need to
pull a dependency for that.
2021-01-22 20:15:27 +01:00
ucwong 7da8f75d5b
go.mod: upgrade golang-lru (#22134) 2021-01-20 19:34:21 +01:00
Martin Holst Swende 93a89b2681
go.mod: use github.com/holiman/bloomfilter/v2 (#22044)
* deps: use improved bloom filter implementation

* eth/handler, trie: use 4 keys for syncbloom + minor fixes

* eth/protocols, trie: revert change on syncbloom method signature
2021-01-12 17:39:31 +01:00
Martin Holst Swende 0b2f1446bb
go.mod: update github.com/golang/snappy(#21934)
This updates the snappy library depency to include a fix for
a Go 1.16 incompatibility issue.
2020-12-02 16:42:38 +01:00
Guillaume Ballet fa572cd297
crypto: signing builds with signify/minisign (#21798)
* internal/build: implement signify's signing func
* Add signify to the ci utility
* fix output file format
* Add unit test for signify
* holiman's + travis' feedback
* internal/build: verify signify's output
* crypto: move signify to common dir
* use go-minisign to verify binaries
* more holiman feedback
* crypto, ci: support minisign output
* only accept one-line trusted comments
* configurable untrusted comments
* code cleanup in tests
* revert to use ed25519 from the stdlib
* bug: fix for empty untrusted comments
* write timestamp as comment if trusted comment isn't present
* rename line checker to commentHasManyLines
* crypto: added signify fuzzer (#6)
* crypto: added signify fuzzer
* stuff
* crypto: updated signify fuzzer to fuzz comments
* crypto: repro signify crashes
* rebased fuzzer on build-signify branch
* hide fuzzer behind gofuzz build flag
* extract key data inside a single function
* don't treat \r as a newline
* travis: fix signing command line
* do not use an external binary in tests
* crypto: move signify to crypto/signify
* travis: fix formatting issue
* ci: fix linter build after package move

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
2020-11-27 12:13:54 +01:00
Marius van der Wijden 2c097bb7a2
mobile: better api for java users (#21580)
* (mobile): Adds string representations for types

* mobile: better interfaces add stringer to types

Co-authored-by: sarath <sarath@melvault.com>
2020-09-21 16:33:35 +02:00
Marius van der Wijden 89884dc353
tests/fuzzers/abi: add fuzzer for fuzzing package accounts/abi (#21217)
* tests/fuzzers/abi: added abi fuzzer

* accounts/abi: fixed issues found by fuzzing

* tests/fuzzers/abi: update fuzzers, added repro test

* tests/fuzzers/abi: renamed abi_fuzzer to abifuzzer

* tests/fuzzers/abi: updated abi fuzzer

* tests/fuzzers/abi: updated abi fuzzer

* accounts/abi: minor style fix

* go.mod: added go-fuzz dependency

* tests/fuzzers/abi: updated abi fuzzer

* tests/fuzzers/abi: make linter happy

* tests/fuzzers/abi: make linter happy

* tests/fuzzers/abi: comment out false positives
2020-09-16 13:15:22 +02:00
ucwong d90bbce954
go.mod : update goja dependency (#21432) 2020-09-01 12:56:22 +02:00
ucwong 856307d8bb
go.mod | goleveldb latest update (#21448)
* go.mod | goleveldb latest update

* go.mod update

* leveldb options

* go.mod: double check

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2020-08-26 15:53:12 +02:00
Martin Holst Swende 56434bfa89
deps: update uint256 to v1.1.1 2020-07-24 14:00:08 +02:00
ucwong 7a556abe15
go.mod: upgrade to github.com/golang/snappy with arm64 asm (#21304) 2020-07-08 14:06:53 +02:00
Adam Schmideg 6a48ae37b2
cmd/devp2p: add discv4 test suite (#21163)
This adds a test suite for discovery v4. The test suite is a port of the Hive suite for
discovery, and will replace the current suite on Hive soon-ish. The tests can be
run locally with this command:

    devp2p discv4 test -remote enode//...

Co-authored-by: Felix Lange <fjl@twurst.com>
2020-07-07 14:37:33 +02:00
rene 1e635bd0bd
go.mod: updated crypto deps causing build failure (#21276) 2020-06-30 16:05:59 +02:00
Guillaume Ballet b86b1e6d43
go.mod: bump gopsutil version (#21275) 2020-06-30 15:22:21 +02:00
Guillaume Ballet 9a188c975d
common/fdlimit: build on DragonflyBSD (#21241)
* common/fdlimit: build on DragonflyBSD

* review feedback
2020-06-19 15:43:52 +02:00
ucwong e029cc6616
go.mod: update snappy dependency (#21237) 2020-06-18 14:01:49 +03:00
Martin Holst Swende c85fdb76ee
go.mod: update uint256 to 1.1.0 (#21206) 2020-06-11 07:27:43 +03:00
Paweł Bylica cf6674539c
core/vm: use uint256 in EVM implementation (#20787)
* core/vm: use fixed uint256 library instead of big

* core/vm: remove intpools

* core/vm: upgrade uint256, fixes uint256.NewFromBig

* core/vm: use uint256.Int by value in Stack

* core/vm: upgrade uint256 to v1.0.0

* core/vm: don't preallocate space for 1024 stack items (only 16)

Co-authored-by: Martin Holst Swende <martin@swende.se>
2020-06-08 15:24:40 +03:00
ucwong 4b2ff1457a
go.mod: upgrade go-duktape to hide unused function warning (#21168) 2020-06-04 17:42:05 +02:00
kilic 4fc678542d
core/vm, crypto/bls12381, params: add bls12-381 elliptic curve precompiles (#21018)
* crypto: add bls12-381 elliptic curve wrapper

* params: add bls12-381 precompile gas parameters

* core/vm: add bls12-381 precompiles

* core/vm: add bls12-381 precompile tests

* go.mod, go.sum: use latest bls12381 lib

* core/vm: move point encode/decode functions to base library

* crypto/bls12381: introduce bls12-381 library init function

* crypto/bls12381: import bls12381 elliptic curve implementation

* go.mod, go.sum: remove bls12-381 library

* remove unsued frobenious coeffs

supress warning for inp that used in asm

* add mappings tests for zero inputs

fix swu g2 minus z inverse constant

* crypto/bls12381: fix typo

* crypto/bls12381: better comments for bls12381 constants

* crypto/bls12381: swu, use single conditional for e2

* crypto/bls12381: utils, delete empty line

* crypto/bls12381: utils, use FromHex for string to big

* crypto/bls12381: g1, g2, strict length check for FromBytes

* crypto/bls12381: field_element, comparision changes

* crypto/bls12381: change swu, isogeny constants with hex values

* core/vm: fix point multiplication comments

* core/vm: fix multiexp gas calculation and lookup for g1 and g2

* core/vm: simpler imput length check for multiexp and pairing precompiles

* core/vm: rm empty multiexp result declarations

* crypto/bls12381: remove modulus type definition

* crypto/bls12381: use proper init function

* crypto/bls12381: get rid of new lines at fatal desciprtions

* crypto/bls12-381: fix no-adx assembly multiplication

* crypto/bls12-381: remove old config function

* crypto/bls12381: update multiplication backend

this commit changes mul backend to 6limb eip1962 backend

mul assign operations are dropped

* core/vm/contracts_tests: externalize test vectors for precompiles

* core/vm/contracts_test: externalize failure-cases for precompiles

* core/vm: linting

* go.mod: tiny up sum file

* core/vm: fix goimports linter issues

* crypto/bls12381: build tags for plain ASM or ADX implementation

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2020-06-03 09:44:32 +03:00
rene a35382de94
metrics: replace gosigar with gopsutil (#21041)
* replace gosigar with gopsutil

* removed check for whether GOOS is openbsd

* removed accidental import of runtime

* potential fix for difference in units between gosig and gopsutil

* fixed lint error

* remove multiplication factor

* uses cpu.ClocksPerSec as the multiplication factor

* changed dependency from shirou to renaynay (#20)

* updated dep

* switching back from using renaynay fork to using upstream as PRs were merged on upstream

* removed empty line

* optimized imports

* tidied go mod
2020-06-02 12:08:33 +03:00
gary rong 65ce550b37
trie: extend range proofs with non-existence (#21000)
* trie: implement range proof with non-existent edge proof

* trie: fix cornercase

* trie: consider empty range

* trie: add singleSide test

* trie: support all-elements range proof

* trie: fix typo

* trie: tiny typos and formulations

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2020-05-20 15:45:38 +03:00
Felix Lange 7e4d1925f0
go.sum: run go mod tidy (#21014) 2020-05-01 12:51:04 +02:00
gary rong 2337aa64eb
core/state/snapshot: fix trie generator reporter (#21004) 2020-04-30 10:43:50 +03:00
ucwong 1264c19f11
go.mod : goupnp v1.0.0 upgrade (#20996) 2020-04-28 14:57:07 +03:00
ucwong 1893266c59
go.mod: upgrade to golang-lru v0.5.4 (#20992)
golang-lru is now a go module, and the upgrade corrects a couple
of minor issues. In particular, the library could crash if you inserted
nil into an LRU cache.
2020-04-28 10:22:23 +02:00
ucwong c60c0c97e7
go.mod : update fastcache to 1.5.7 (#20936) 2020-04-22 10:31:32 +03:00
Alex Willmer 5dd0cd12ec
go.mod: update duktape to fix sprintf warnings (#20777)
This revision of go-duktype fixes the following warning

```
duk_logging.c: In function ‘duk__logger_prototype_log_shared’:
duk_logging.c:184:64: warning: ‘Z’ directive writing 1 byte into a region of size between 0 and 9 [-Wformat-overflow=]
  184 |  sprintf((char *) date_buf, "%04d-%02d-%02dT%02d:%02d:%02d.%03dZ",
      |                                                                ^
In file included from /usr/include/stdio.h:867,
                 from duk_logging.c:5:
/usr/include/x86_64-linux-gnu/bits/stdio2.h:36:10: note: ‘__builtin___sprintf_chk’ output between 25 and 85 bytes into a destination of size 32
   36 |   return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL - 1,
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   37 |       __bos (__s), __fmt, __va_arg_pack ());
      |       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
2020-03-18 11:52:07 +02:00
Péter Szilágyi 466b009135
go.mod: update golang.org/x/crypto to fix a Go 1.14 race rejection 2020-03-14 14:09:31 +02:00
Guillaume Ballet 6df973df27
go.mod: upgrade goja to latest (#20700)
The new goja version supports the 'escape' and 'unescape' built-in functions.
This fixes #20693
2020-02-20 12:46:47 +01:00
Guillaume Ballet 594e038e75
signer/rules: use goja and remove otto (#20599)
* signer: replace otto with goja

* go.mod: remove Otto
2020-01-29 13:47:56 +01:00
Guillaume Ballet 7b68975a00 console, internal/jsre: use github.com/dop251/goja (#20470)
This replaces the JavaScript interpreter used by the console with goja,
which is actively maintained and a lot faster than otto. Clef still uses otto
and eth/tracers still uses duktape, so we are currently dependent on three
different JS interpreters. We're looking to replace the remaining uses of otto
soon though.
2020-01-27 11:50:48 +01:00
gary rong c6285e6437 les/checkpointoracle: move oracle into its own package (#20508)
* les: move the checkpoint oracle into its own package

It's first step of refactor LES package. LES package
basically can be divided into LES client and LES server.
However both sides will use checkpoint package for
status retrieval and verification. So this PR moves
checkpoint oracle into a separate package

* les: address comments
2020-01-07 11:24:21 +02:00
Felix Lange f51cf573b5
cmd/devp2p: implement AWS Route53 enrtree deployer (#20446) 2019-12-12 22:25:12 +01:00
Martin Holst Swende 7800ba978d
deps: update fastcache to 1.5.3 2019-11-27 13:46:07 +01:00
gary rong 9d67222f4e trie: replace bigcache with fastcache (#19971) 2019-11-25 10:58:15 +02:00
Péter Szilágyi e1dcea8bf0
go.mod: tidy up the modules to avoid xgo writes to go.sum 2019-11-20 14:42:34 +02:00
Péter Szilágyi 15fb780de6
go.mod, vendor: switch to Go modules 2019-11-20 14:42:28 +02:00