interesting
/
go-ethereum
mirror of https://github.com/ethereum/go-ethereum.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
go-ethereum/core
History
Marius van der Wijden 767a2928bc core: reduce peak memory usage during reorg (#30600) 
~~Opening this as a draft to have a discussion.~~ Pressed the wrong
button
I had [a previous PR
](https://github.com/ethereum/go-ethereum/pull/24616)a long time ago
which reduced the peak memory used during reorgs by not accumulating all
transactions and logs.
This PR reduces the peak memory further by not storing the blocks in
memory.
However this means we need to pull the blocks back up from storage
multiple times during the reorg.
I collected the following numbers on peak memory usage: 

// Master: BenchmarkReorg-8 10000 899591 ns/op 820154 B/op 1440
allocs/op 1549443072 bytes of heap used
// WithoutOldChain: BenchmarkReorg-8 10000 1147281 ns/op 943163 B/op
1564 allocs/op 1163870208 bytes of heap used
// WithoutNewChain: BenchmarkReorg-8 10000 1018922 ns/op 943580 B/op
1564 allocs/op 1171890176 bytes of heap used

Each block contains a transaction with ~50k bytes and we're doing a 10k
block reorg, so the chain should be ~500MB in size

---------

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
 		2024-11-19 14:50:11 +01:00
..
asm core/vm, cmd/evm: implement eof validation (#30418) 2024-11-19 14:50:11 +01:00
bloombits all: fix typos in comments (#29873) 2024-05-29 12:24:10 +02:00
forkid all: clean up goerli flag and config (#30289) 2024-08-20 15:59:48 +02:00
rawdb core, ethdb/pebble: run pebble in non-sync mode (#30573) 2024-11-19 14:50:11 +01:00
state build: update to golangci-lint 1.61.0 (#30587) 2024-11-19 14:50:11 +01:00
stateless beacon, core, eth, miner: integrate witnesses into production Geth (#30069) 2024-09-20 16:43:42 +03:00
tracing core/tracing: add GetTransientState method to StateDB interface (#30531) 2024-11-19 14:50:11 +01:00
txpool core, eth, ethstats: simplify chain head events (#30601) 2024-11-19 14:50:11 +01:00
types build: update to golangci-lint 1.61.0 (#30587) 2024-11-19 14:50:11 +01:00
vm build: update to golangci-lint 1.61.0 (#30587) 2024-11-19 14:50:11 +01:00
.gitignore Renamed `chain` => `core` 2014-12-04 10:28:02 +01:00
bench_test.go all: remove forkchoicer and reorgNeeded (#29179) 2024-09-04 15:03:06 +02:00
block_validator.go build: update to golangci-lint 1.61.0 (#30587) 2024-11-19 14:50:11 +01:00
block_validator_test.go beacon, core, eth, miner: integrate witnesses into production Geth (#30069) 2024-09-20 16:43:42 +03:00
blockchain.go core: reduce peak memory usage during reorg (#30600) 2024-11-19 14:50:11 +01:00
blockchain_insert.go all: remove forkchoicer and reorgNeeded (#29179) 2024-09-04 15:03:06 +02:00
blockchain_reader.go core, eth, ethstats: simplify chain head events (#30601) 2024-11-19 14:50:11 +01:00
blockchain_repair_test.go core, ethdb/pebble: run pebble in non-sync mode (#30573) 2024-11-19 14:50:11 +01:00
blockchain_sethead_test.go core, ethdb/pebble: run pebble in non-sync mode (#30573) 2024-11-19 14:50:11 +01:00
blockchain_snapshot_test.go core, ethdb/pebble: run pebble in non-sync mode (#30573) 2024-11-19 14:50:11 +01:00
blockchain_test.go core: reduce peak memory usage during reorg (#30600) 2024-11-19 14:50:11 +01:00
bloom_indexer.go core: preallocate batch size in bloomIndexer (#25289) 2022-08-03 17:02:09 +02:00
chain_indexer.go core, eth, ethstats: simplify chain head events (#30601) 2024-11-19 14:50:11 +01:00
chain_indexer_test.go all: improve some error strings (#29842) 2024-05-28 13:44:40 +02:00
chain_makers.go core: enable EIP-2935 in chain maker (#30575) 2024-11-19 14:50:11 +01:00
chain_makers_test.go all: remove forkchoicer and reorgNeeded (#29179) 2024-09-04 15:03:06 +02:00
dao_test.go all: remove forkchoicer and reorgNeeded (#29179) 2024-09-04 15:03:06 +02:00
error.go cmd, core, params, trie: add verkle access witness gas charging (#29338) 2024-05-10 20:13:11 +02:00
events.go core, eth, ethstats: simplify chain head events (#30601) 2024-11-19 14:50:11 +01:00
evm.go all: use big.Sign to compare with zero (#29490) 2024-04-09 12:14:30 +02:00
gaspool.go core, miner: revert block gas counter in case of invalid transaction (#26799) 2023-03-07 05:23:52 -05:00
gen_genesis.go core: go fmt (#29544) 2024-04-16 15:42:16 +03:00
genesis.go all: implement EIP-7002 & EIP-7251 (#30571) 2024-11-19 14:50:11 +01:00
genesis_alloc.go all: clean up goerli flag and config (#30289) 2024-08-20 15:59:48 +02:00
genesis_test.go all: remove forkchoicer and reorgNeeded (#29179) 2024-09-04 15:03:06 +02:00
headerchain.go all: remove forkchoicer and reorgNeeded (#29179) 2024-09-04 15:03:06 +02:00
headerchain_test.go all: remove forkchoicer and reorgNeeded (#29179) 2024-09-04 15:03:06 +02:00
mkalloc.go accounts, cmd/geth, core: close opened files (#29598) 2024-04-30 15:47:21 +02:00
rlp_test.go core: move genesis alloc types to core/types (#29003) 2024-02-16 19:05:33 +01:00
sender_cacher.go all: refactor txpool into it's own package in prep for 4844 2022-10-24 16:35:53 +03:00
state_prefetcher.go all: stateless witness builder and (self-)cross validator (#29719) 2024-06-25 14:48:08 +03:00
state_processor.go all: implement EIP-7002 & EIP-7251 (#30571) 2024-11-19 14:50:11 +01:00
state_processor_test.go core/state: state reader abstraction (#29761) 2024-09-05 13:10:47 +03:00
state_transition.go internal/ethapi: eth_multicall (#27720) 2024-09-06 11:31:00 +02:00
stateless.go beacon, core, eth, miner: integrate witnesses into production Geth (#30069) 2024-09-20 16:43:42 +03:00
txindexer.go core, eth, ethstats: simplify chain head events (#30601) 2024-11-19 14:50:11 +01:00
txindexer_test.go core: use in-memory freezer for tests (#29720) 2024-05-08 09:43:33 +03:00
types.go all: implement flat deposit requests encoding (#30425) 2024-11-19 14:50:11 +01:00