40cdcf1183
trie, core/state: improve memory usage and performance ( #3135 )
...
* trie: store nodes as pointers
This avoids memory copies when unwrapping node interface values.
name old time/op new time/op delta
Get 388ns ± 8% 215ns ± 2% -44.56% (p=0.000 n=15+15)
GetDB 363ns ± 3% 202ns ± 2% -44.21% (p=0.000 n=15+15)
UpdateBE 1.57µs ± 2% 1.29µs ± 3% -17.80% (p=0.000 n=13+15)
UpdateLE 1.92µs ± 2% 1.61µs ± 2% -16.25% (p=0.000 n=14+14)
HashBE 2.16µs ± 6% 2.18µs ± 6% ~ (p=0.436 n=15+15)
HashLE 7.43µs ± 3% 7.21µs ± 3% -2.96% (p=0.000 n=15+13)
* trie: close temporary databases in GetDB benchmark
* trie: don't keep []byte from DB load around
Nodes decoded from a DB load kept hashes and values as sub-slices of
the DB value. This can be a problem because loading from leveldb often
returns []byte with a cap that's larger than necessary, increasing
memory usage.
* trie: unload old cached nodes
* trie, core/state: use cache unloading for account trie
* trie: use explicit private flags (fixes Go 1.5 reflection issue).
* trie: fixup cachegen overflow at request of nick
* core/state: rename journal size constant
2016-10-14 19:04:33 +03:00
2924fdfcf7
ethereum, ethclient: add SyncProgress API endpoint
2016-09-06 13:41:43 +03:00
1dd272080d
eth, eth/downloader: better remote head tracking
2016-07-25 15:14:14 +03:00
016007bd25
eth, eth/downloader, eth/fetcher: delete eth/61 code
...
The eth/61 protocol was disabled in #2776 , this commit removes its
message handlers and hash-chain sync logic.
2016-07-22 13:17:19 +02:00
3291235711
accounts, core, eth: pass chain config for chain maker to test DAO
2016-07-15 16:52:55 +03:00
783289068a
eth/downloader: fix occasional fast sync critical section test fails
2016-06-13 13:04:19 +03:00
88f174a014
eth/downloader: adaptive quality of service tuning
2016-06-06 14:20:57 +03:00
61ee9f299d
eth/downloader: make fast sync resilient to critical section fails
2016-06-02 12:37:14 +03:00
4496a44f68
eth/downloader: ensure cancel channel is closed post sync
2016-05-30 12:01:50 +03:00
e86619e75d
eth/downloader: stream partial skeleton filling to processor
2016-05-17 10:03:34 +03:00
b40dc8a1da
eth/downloader: implement concurrent header downloads
2016-05-17 10:03:34 +03:00
39ce85cf5d
eth/downloader: bound fork ancestry and allow heavy short forks
2016-05-13 12:12:13 +02:00
bbbe2360d0
eth: various typos
2016-03-15 11:27:49 -07:00
93c0012000
eth/downloader: bump the download queue size to prevent starvation
2016-02-23 13:12:11 +02:00
59cd60b266
eth, eth/downloader, jsre: surface state sync progress through the API
2016-02-10 14:30:41 +02:00
9e011ff1cd
core, eth/downloader: ensure state presence in ancestor lookup
2016-01-04 16:27:23 +02:00
1f39746886
eth/downloader: throttling tests are time-sensitive, don't run parallel
2015-12-30 14:42:21 +02:00
9be5d5cd90
eth/downloader: fix negative balance issue in tests
...
The test chain generated by makeChainFork included invalid uncle
headers, crashing the generator during the state commit.
The headers were invalid because they used the iteration counter as the
block number, even though makeChainFork uses a block with number > 0 as
the parent. Fix this by introducing BlockGen.Number, which allows
accessing the actual number of the block being generated.
2015-12-18 12:09:10 +01:00
dd09af27af
eth/downloader: run tests in parallel
2015-11-19 14:18:35 +01:00
900da3d800
eth/downloader: don't hang for spurious deliveries
...
Unexpected deliveries could block indefinitely if they arrived at the
right time. The fix is to ensure that the cancellation channel is
always closed when the sync ends, unblocking any deliveries. Also remove
the atomic check for whether a sync is currently running because it
doesn't help and can be misleading.
Cancelling always seems to break the tests though. The downloader
spawned d.process whenever new data arrived, making it somewhat hard to
track when block processing was actually done. Fix this by running
d.process in a dedicated goroutine that is tied to the lifecycle of the
sync. d.process gets notified of new work by the queue instead of being
invoked all the time. This removes a ton of weird workaround code,
including a hairy use of atomic CAS.
2015-11-19 14:18:34 +01:00
5b0ee8ec30
core, eth, trie: fix data races and merge/review issues
2015-10-21 16:49:55 +03:00
a9d8dfc8e7
core, eth: roll back uncertain headers in failed fast syncs
2015-10-19 10:03:10 +03:00
b97e34a8e4
eth/downloader: concurrent receipt and state processing
2015-10-19 10:03:10 +03:00
ab27bee25a
core, eth, trie: direct state trie synchronization
2015-10-19 10:03:09 +03:00
832b37c822
core, eth: receipt chain reconstruction
2015-10-19 10:03:09 +03:00
f186b39018
eth/downloader: add fast and light sync strategies
2015-10-19 10:03:09 +03:00
47f62a67aa
eth/downloader: match capabilities when querying idle peers
2015-10-02 13:20:41 +03:00
99b62f36b6
eth/downloader: header-chain order and ancestry check
2015-09-15 14:45:53 +03:00
0a7d059b6a
eth, rpc: standardize the chain sync progress counters
2015-09-15 14:45:53 +03:00
cdc2662c40
core: split out TD from database and all internals
2015-09-11 17:42:25 +03:00
47a7fe5d22
eth: port the synchronisation algo to eth/62
2015-08-25 17:48:47 +03:00
ca88e18f59
eth: kill off protocol eth/60 in preparation for eth/62
2015-08-24 17:57:28 +03:00
6adbaabc65
eth, eth/downloader: don't report stall if fetcher filled the block
2015-07-29 15:39:08 +03:00
bfbcfbe4a9
all: fix license headers one more time
...
I forgot to update one instance of "go-ethereum" in commit 3f047be5a
2015-07-23 18:35:11 +02:00
3f047be5aa
all: update license headers to distiguish GPL/LGPL
...
All code outside of cmd/ is licensed as LGPL. The headers
now reflect this by calling the whole work "the go-ethereum library".
2015-07-22 18:51:45 +02:00
492d5454b1
eth/downloader: drop peer if advertised TD but won't delvier
2015-07-09 14:40:18 +03:00
ea54283b30
all: update license information
2015-07-07 14:12:44 +02:00
f857fb7600
eth/downloader: fix a rare test race on the OSX CI
2015-07-03 13:01:22 +03:00
f43c07cb3c
eth, eth/downloader: transition to eth 61
2015-06-30 19:05:06 +03:00
af51dc4d63
eth, eth/downloader: pass the eth protocol version through
2015-06-30 19:00:01 +03:00
e0e5f74776
eth/downloader, eth/fetcher: use core.GenerateChain in tests
...
TestMadeupParentBlockChainAttack has been deleted because it was too
hard to port and the attack that it checks the prevention of is being
averted in a different way (through a protocol change).
2015-06-29 18:51:47 +02:00
1d42888d30
core/types: make blocks immutable
2015-06-29 18:51:47 +02:00
2a1b722d04
eth/fetcher: fix timer reset bug, add initial tests
2015-06-18 15:56:07 +03:00
4365668462
eth/downloader: extend slow test to fix even slower CI server...
2015-06-18 00:42:02 +03:00
55dd8fd621
eth/downloader: always reenter processing if not exiting
2015-06-18 00:26:54 +03:00
2f4cbe22f5
eth, eth/downloader: fix processing interrupt caused by temp cancel
2015-06-18 00:04:57 +03:00
4a1e82cf3f
eth/downloader: fix #1280 , overlapping (good/bad) delivery hang
2015-06-17 12:03:16 +03:00
b240983e2b
eth, eth/downloader: do async block fetches, add dl tests
2015-06-15 12:26:05 +03:00
30a9939388
eth/downloader: sanity test for multi peer syncs
2015-06-15 09:22:37 +03:00
fc7abd9886
eth, eth/downloader: move block processing into the downlaoder
2015-06-15 09:22:37 +03:00
0fc71877a7
eth/downloader: add valid peer during attacks (check interference)
2015-06-15 09:22:37 +03:00
80833f8137
eth/downloader: instreument and test the sync peer drop
2015-06-15 09:22:37 +03:00
2dd6a62f67
eth/downloader: support individual peers in the test suite
2015-06-15 09:22:36 +03:00
2937903299
eth/downloader: remove uneeded testing functions
2015-06-15 09:22:36 +03:00
66d3dc8690
eth, eth/downloader: move peer removal into downloader
2015-06-15 09:22:36 +03:00
4ed3509a02
eth/downloader: test registration rejection on head ban
2015-06-08 15:02:52 +03:00
63c6cedb14
eth/downloader: cap the hash ban set, add test for it
2015-06-08 14:12:00 +03:00
4b2dd44711
eth/downloader: fix throttling test to be less timing dependent
2015-06-08 13:23:58 +03:00
b40c796ff7
eth/downloader: preallocate the block cache
2015-06-08 13:23:58 +03:00
9da0232eef
eth/downloader: update test for shitty travis
2015-06-08 13:23:58 +03:00
84bc93d8cb
eth/downloader: accumulating hash bans for reconnecting attackers
2015-06-08 13:23:58 +03:00
eedb25b22a
eth/downloader: clean up tests and unused variables
2015-06-08 13:23:57 +03:00
c9a546c310
eth/downloader: add a basic block download congestion control
2015-06-03 14:40:11 +03:00
eafdc1f8e3
eth, eth/downloader: surface downloaded block origin, drop on error
2015-05-26 14:00:21 +03:00
af28736bd0
Merge pull request #1064 from karalabe/downloader-attacks
...
Fix two additional download vulnerabilities
2015-05-21 09:00:12 -07:00
06a041589f
eth, eth/downloader: remove duplicate consts, bump hash fetch to 2K
2015-05-21 18:16:04 +03:00
52db6d8be5
eth/downloader: circumvent a forged block chain with known parent attack
2015-05-21 08:37:27 +03:00
e8b22b9253
eth/downloader: prevent a peer from dripping bad hashes
2015-05-21 08:07:58 +03:00
6f54eb6d9a
eth/downloader: fix test to it doesn't time out on a slow machine
2015-05-20 10:15:42 +03:00
412cf98bbc
eth/downloader: fix #992 , where tests may time out on a slow machine
2015-05-15 21:16:42 +03:00
5c1a7b965c
eth/downloader: circumvent a fake blockchain attack
2015-05-15 15:04:15 +03:00
83226762c2
eth, eth/downloader: detect and handle madeup hash attacks
2015-05-15 15:01:58 +03:00
72411eb24c
eth/downloader: circumvent hash reordering attacks
2015-05-15 15:01:58 +03:00
bcc2980179
eth/downloader: check sync after failed attacks
2015-05-15 15:01:58 +03:00
366e9627e8
eth/downloader: add a test for detecting missing blocks
2015-05-15 15:01:58 +03:00
cd2fb09051
eth, eth/downloader: prevent hash repeater attack
2015-05-15 15:01:58 +03:00
cfb2b51b74
eth/downloader: fixed test
2015-05-15 12:47:32 +02:00
fe87feccb1
eth/downloader: add a small additional check to the test
2015-05-14 15:44:54 +03:00
3eda70c64c
eth, eth/downloader: remove parent verification from the downlaoder
2015-05-14 15:38:49 +03:00
a4246c2da6
eth, eth/downloader: handle a potential unknown parent attack
2015-05-14 15:24:18 +03:00
d2d5dbc6fb
eth/downloader: fix active peer shadowing, polish func names
2015-05-13 13:13:43 +03:00
685862d2ce
eth/downloader: fix #910 , thread safe peers & polishes
2015-05-11 14:26:20 +03:00
05715f27cf
eth: added a cancel method for the downloader
...
Added a cancel method to the downloader which gracefully shuts down any
active syncing process (hash fetching or block downloading) and resets
the queue and remove any pending blocks.
Issue with the downloader which would stall because of an active ongoing
process when an invalid block was found.
2015-05-10 00:34:07 +02:00
bd5720f480
eth, eth/downloader: handle sync errors a bit more gracefully
2015-05-08 15:22:48 +03:00
9d188f73b5
eth, eth/downloader: make synchronize thread safe
2015-05-07 21:07:20 +03:00
43901c9282
eth/downloader: fix priority queue reset, add throttling test
2015-05-07 14:40:50 +03:00
4800c94392
eth/downloader: prioritize block fetch based on chain position, cap memory use
2015-05-07 10:57:34 +03:00
1470b22e90
downloader: hash downloading recovery
...
If a peer fails to respond (disconnect, etc) during hash downloading
switch to a different peer which has it's current_hash in the queue's
peer set.
2015-05-03 16:27:03 +02:00
25bc88113f
eth/downloader: added additional tests
2015-05-03 14:09:33 +02:00
016f152b36
eth, eth/downloader: Moved block processing & graceful shutdown
...
The downloader is no longer responsible for processing blocks. The
eth-protocol handler now takes care of this instead.
Added graceful shutdown during block processing. Closes #846
2015-05-01 15:58:44 +02:00
c930045834
core: fixed tetst to reflect (int, error) return by insertChain
2015-04-29 15:27:04 +02:00
a9e4b96573
eth/downloader: fixed tests
2015-04-24 15:56:18 +02:00
31f82eb334
eth, eth/downloader: don't require td on downloader. Fixed tests
2015-04-24 15:04:58 +02:00
ed07ffcde5
downloader: fixed tests with low ttl
2015-04-19 17:37:01 +02:00
c58918c84a
downloader: moved chunk ignoring. Fixes issue with catching up
2015-04-19 13:30:34 +02:00
a8a2b2a488
downloader: added missing blocks catchup functionality
...
When a parent is missing in the block list an attempt should be made to
fetch the missing parent and grandparents.
2015-04-13 16:38:32 +02:00
acf8452c33
downloader: implemented new downloader
2015-04-12 12:38:25 +02:00
Felix Lange
Péter Szilágyi
Leif Jurvetson
Jeffrey Wilcke
obscuren