Commit Graph

115 Commits

Author SHA1 Message Date
Felix Lange 10181b57a9 core, eth/downloader: commit block data using batches (#15115)
* ethdb: add Putter interface and Has method

* ethdb: improve docs and add IdealBatchSize

* ethdb: remove memory batch lock

Batches are not safe for concurrent use.

* core: use ethdb.Putter for Write* functions

This covers the easy cases.

* core/state: simplify StateSync

* trie: optimize local node check

* ethdb: add ValueSize to Batch

* core: optimize HasHeader check

This avoids one random database read get the block number. For many uses
of HasHeader, the expectation is that it's actually there. Using Has
avoids a load + decode of the value.

* core: write fast sync block data in batches

Collect writes into batches up to the ideal size instead of issuing many
small, concurrent writes.

* eth/downloader: commit larger state batches

Collect nodes into a batch up to the ideal size instead of committing
whenever a node is received.

* core: optimize HasBlock check

This avoids a random database read to get the number.

* core: use numberCache in HasHeader

numberCache has higher capacity, increasing the odds of finding the
header without a database lookup.

* core: write imported block data using a batch

Restore batch writes of state and add blocks, tx entries, receipts to
the same batch. The change also simplifies the miner.

This commit also removes posting of logs when a forked block is imported.

* core: fix DB write error handling

* ethdb: use RLock for Has

* core: fix HasBlock comment
2017-09-09 19:03:07 +03:00
Ivan Daniluk 17ce0a37de eth/downloader: fix race in downloadTesterPeer (#14942)
* eth/downloader: fix race in downloadTesterPeer

Signed-off-by: Ivan Daniluk <ivan.daniluk@gmail.com>

* eth/downloader: minor datarace fix cleanup
2017-08-08 20:10:09 +03:00
Egon Elbre 8f06b7980d eth: fix megacheck warnings 2017-08-07 19:54:20 +03:00
Nick Johnson be8f8409bc eth/downloader, les, light: Changes in response to review 2017-07-03 15:17:12 +01:00
Nick Johnson ae11545bc5 eth, les: Refactor downloader peer to use structs 2017-06-29 12:49:18 +01:00
Nick Johnson 0550957989 eth, les, light: Refactor downloader to use blockchain interface 2017-06-28 15:58:41 +01:00
Felix Lange 9e5f03b6c4 core/state: access trie through Database interface, track errors (#14589)
With this commit, core/state's access to the underlying key/value database is
mediated through an interface. Database errors are tracked in StateDB and
returned by CommitTo or the new Error method.

Motivation for this change: We can remove the light client's duplicated copy of
core/state. The light client now supports node iteration, so tracing and storage
enumeration can work with the light client (not implemented in this commit).
2017-06-27 15:57:06 +02:00
Péter Szilágyi 9b84caf3a5 core, eth, les: support resuming fast sync on heavy rollback (#3743) 2017-03-22 01:37:24 +01:00
Jeffrey Wilcke c12f4df910 params: core, core/vm, miner: 64bit gas instructions
Reworked the EVM gas instructions to use 64bit integers rather than
arbitrary size big ints. All gas operations, be it additions,
multiplications or divisions, are checked and guarded against 64 bit
integer overflows.

In additon, most of the protocol paramaters in the params package have
been converted to uint64 and are now constants rather than variables.

* common/math: added overflow check ops
* core: vmenv, env renamed to evm
* eth, internal/ethapi, les: unmetered eth_call and cancel methods
* core/vm: implemented big.Int pool for evm instructions
* core/vm: unexported intPool methods & verification methods
* core/vm: added memoryGasCost overflow check and test
2017-02-13 21:44:25 +01:00
Jeffrey Wilcke 57f4e90257 Revert "params: core, core/vm, miner: 64bit gas instructions (#3514)"
This reverts commit 8b57c49490.
2017-02-13 15:15:12 +01:00
Jeffrey Wilcke 8b57c49490 params: core, core/vm, miner: 64bit gas instructions (#3514)
Reworked the EVM gas instructions to use 64bit integers rather than
arbitrary size big ints. All gas operations, be it additions,
multiplications or divisions, are checked and guarded against 64 bit
integer overflows.

In additon, most of the protocol paramaters in the params package have
been converted to uint64 and are now constants rather than variables.

* common/math: added overflow check ops
* core: vmenv, env renamed to evm
* eth, internal/ethapi, les: unmetered eth_call and cancel methods
* core/vm: implemented big.Int pool for evm instructions
* core/vm: unexported intPool methods & verification methods
* core/vm: added memoryGasCost overflow check and test
2017-02-02 15:25:42 +01:00
Felix Lange b9b3efb09f all: fix ineffectual assignments and remove uses of crypto.Sha3
go get github.com/gordonklaus/ineffassign
ineffassign .
2017-01-09 16:24:42 +01:00
Felix Lange e171bf74f8 core/types: remove redundant SignECDSA wrappers, rename to SignTx 2017-01-05 12:59:17 +01:00
Jeffrey Wilcke 4dca5d4db7 core/types, params: EIP#155 2016-11-13 14:55:30 +01:00
Jeffrey Wilcke 445feaeef5 core, core/state, trie: EIP158, reprice & skip empty account write
This commit implements EIP158 part 1, 2, 3 & 4

1. If an account is empty it's no longer written to the trie. An empty
  account is defined as (balance=0, nonce=0, storage=0, code=0).
2. Delete an empty account if it's touched
3. An empty account is redefined as either non-existent or empty.
4. Zero value calls and zero value suicides no longer consume the 25k
  reation costs.

params: moved core/config to params

Signed-off-by: Jeffrey Wilcke <jeffrey@ethereum.org>
2016-11-13 10:44:04 +01:00
zsfelfoldi 7db7109a5b cmd, eth: added light client and light server modes 2016-11-09 02:12:53 +01:00
Péter Szilágyi 8b10617bba
eth/downloader: reduce fast sync block requirements, fix test 2016-10-31 17:26:44 +02:00
Péter Szilágyi 90b16a3e85
core/state, eth/downloader, trie: reset fast-failure on progress 2016-10-31 14:19:14 +02:00
Felix Lange 40cdcf1183 trie, core/state: improve memory usage and performance (#3135)
* trie: store nodes as pointers

This avoids memory copies when unwrapping node interface values.

name      old time/op  new time/op  delta
Get        388ns ± 8%   215ns ± 2%  -44.56%  (p=0.000 n=15+15)
GetDB      363ns ± 3%   202ns ± 2%  -44.21%  (p=0.000 n=15+15)
UpdateBE  1.57µs ± 2%  1.29µs ± 3%  -17.80%  (p=0.000 n=13+15)
UpdateLE  1.92µs ± 2%  1.61µs ± 2%  -16.25%  (p=0.000 n=14+14)
HashBE    2.16µs ± 6%  2.18µs ± 6%     ~     (p=0.436 n=15+15)
HashLE    7.43µs ± 3%  7.21µs ± 3%   -2.96%  (p=0.000 n=15+13)

* trie: close temporary databases in GetDB benchmark

* trie: don't keep []byte from DB load around

Nodes decoded from a DB load kept hashes and values as sub-slices of
the DB value. This can be a problem because loading from leveldb often
returns []byte with a cap that's larger than necessary, increasing
memory usage.

* trie: unload old cached nodes

* trie, core/state: use cache unloading for account trie

* trie: use explicit private flags (fixes Go 1.5 reflection issue).

* trie: fixup cachegen overflow at request of nick

* core/state: rename journal size constant
2016-10-14 19:04:33 +03:00
Péter Szilágyi 2924fdfcf7 ethereum, ethclient: add SyncProgress API endpoint 2016-09-06 13:41:43 +03:00
Péter Szilágyi 1dd272080d eth, eth/downloader: better remote head tracking 2016-07-25 15:14:14 +03:00
Felix Lange 016007bd25 eth, eth/downloader, eth/fetcher: delete eth/61 code
The eth/61 protocol was disabled in #2776, this commit removes its
message handlers and hash-chain sync logic.
2016-07-22 13:17:19 +02:00
Péter Szilágyi 3291235711 accounts, core, eth: pass chain config for chain maker to test DAO 2016-07-15 16:52:55 +03:00
Péter Szilágyi 783289068a eth/downloader: fix occasional fast sync critical section test fails 2016-06-13 13:04:19 +03:00
Péter Szilágyi 88f174a014 eth/downloader: adaptive quality of service tuning 2016-06-06 14:20:57 +03:00
Péter Szilágyi 61ee9f299d eth/downloader: make fast sync resilient to critical section fails 2016-06-02 12:37:14 +03:00
Péter Szilágyi 4496a44f68 eth/downloader: ensure cancel channel is closed post sync 2016-05-30 12:01:50 +03:00
Péter Szilágyi e86619e75d eth/downloader: stream partial skeleton filling to processor 2016-05-17 10:03:34 +03:00
Péter Szilágyi b40dc8a1da eth/downloader: implement concurrent header downloads 2016-05-17 10:03:34 +03:00
Péter Szilágyi 39ce85cf5d eth/downloader: bound fork ancestry and allow heavy short forks 2016-05-13 12:12:13 +02:00
Leif Jurvetson bbbe2360d0 eth: various typos 2016-03-15 11:27:49 -07:00
Péter Szilágyi 93c0012000 eth/downloader: bump the download queue size to prevent starvation 2016-02-23 13:12:11 +02:00
Péter Szilágyi 59cd60b266 eth, eth/downloader, jsre: surface state sync progress through the API 2016-02-10 14:30:41 +02:00
Péter Szilágyi 9e011ff1cd core, eth/downloader: ensure state presence in ancestor lookup 2016-01-04 16:27:23 +02:00
Péter Szilágyi 1f39746886 eth/downloader: throttling tests are time-sensitive, don't run parallel 2015-12-30 14:42:21 +02:00
Felix Lange 9be5d5cd90 eth/downloader: fix negative balance issue in tests
The test chain generated by makeChainFork included invalid uncle
headers, crashing the generator during the state commit.

The headers were invalid because they used the iteration counter as the
block number, even though makeChainFork uses a block with number > 0 as
the parent. Fix this by introducing BlockGen.Number, which allows
accessing the actual number of the block being generated.
2015-12-18 12:09:10 +01:00
Felix Lange dd09af27af eth/downloader: run tests in parallel 2015-11-19 14:18:35 +01:00
Felix Lange 900da3d800 eth/downloader: don't hang for spurious deliveries
Unexpected deliveries could block indefinitely if they arrived at the
right time. The fix is to ensure that the cancellation channel is
always closed when the sync ends, unblocking any deliveries. Also remove
the atomic check for whether a sync is currently running because it
doesn't help and can be misleading.

Cancelling always seems to break the tests though. The downloader
spawned d.process whenever new data arrived, making it somewhat hard to
track when block processing was actually done. Fix this by running
d.process in a dedicated goroutine that is tied to the lifecycle of the
sync. d.process gets notified of new work by the queue instead of being
invoked all the time. This removes a ton of weird workaround code,
including a hairy use of atomic CAS.
2015-11-19 14:18:34 +01:00
Péter Szilágyi 5b0ee8ec30 core, eth, trie: fix data races and merge/review issues 2015-10-21 16:49:55 +03:00
Péter Szilágyi a9d8dfc8e7 core, eth: roll back uncertain headers in failed fast syncs 2015-10-19 10:03:10 +03:00
Péter Szilágyi b97e34a8e4 eth/downloader: concurrent receipt and state processing 2015-10-19 10:03:10 +03:00
Péter Szilágyi ab27bee25a core, eth, trie: direct state trie synchronization 2015-10-19 10:03:09 +03:00
Péter Szilágyi 832b37c822 core, eth: receipt chain reconstruction 2015-10-19 10:03:09 +03:00
Péter Szilágyi f186b39018 eth/downloader: add fast and light sync strategies 2015-10-19 10:03:09 +03:00
Péter Szilágyi 47f62a67aa eth/downloader: match capabilities when querying idle peers 2015-10-02 13:20:41 +03:00
Péter Szilágyi 99b62f36b6 eth/downloader: header-chain order and ancestry check 2015-09-15 14:45:53 +03:00
Péter Szilágyi 0a7d059b6a eth, rpc: standardize the chain sync progress counters 2015-09-15 14:45:53 +03:00
Péter Szilágyi cdc2662c40 core: split out TD from database and all internals 2015-09-11 17:42:25 +03:00
Péter Szilágyi 47a7fe5d22 eth: port the synchronisation algo to eth/62 2015-08-25 17:48:47 +03:00
Péter Szilágyi ca88e18f59 eth: kill off protocol eth/60 in preparation for eth/62 2015-08-24 17:57:28 +03:00