go-ethereum/core/rawdb
Martin Holst Swende 794c6133ef
core/rawdb: freezer batch write (#23462)
This change is a rewrite of the freezer code.

When writing ancient chain data to the freezer, the previous version first encoded each
individual item to a temporary buffer, then wrote the buffer. For small item sizes (for
example, in the block hash freezer table), this strategy causes a lot of system calls for
writing tiny chunks of data. It also allocated a lot of temporary []byte buffers.

In the new version, we instead encode multiple items into a re-useable batch buffer, which
is then written to the file all at once. This avoids performing a system call for every
inserted item.

To make the internal batching work, the ancient database API had to be changed. While
integrating this new API in BlockChain.InsertReceiptChain, additional optimizations were
also added there.

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-09-07 12:31:17 +02:00
..
accessors_chain.go core/rawdb: freezer batch write (#23462) 2021-09-07 12:31:17 +02:00
accessors_chain_test.go core/rawdb: freezer batch write (#23462) 2021-09-07 12:31:17 +02:00
accessors_indexes.go core/rawdb: fix logs to print block number, not address (#23328) 2021-08-04 11:10:37 +03:00
accessors_indexes_test.go core/rawdb: single point of maintenance for writing and deleting tx lookup indexes (#21480) 2020-09-15 10:37:01 +02:00
accessors_metadata.go core, eth, les: implement unclean-shutdown marker (#21893) 2020-12-11 15:56:00 +01:00
accessors_snapshot.go core, eth: abort snapshot generation on snap sync and resume later 2021-04-30 17:03:10 +03:00
accessors_state.go core, eth, les, trie: add a prefix to contract code (#21080) 2020-08-21 15:10:40 +03:00
chain_iterator.go core/rawdb: fix transaction indexing/unindexing hashing error (#22457) 2021-03-16 12:15:14 +02:00
chain_iterator_test.go core/rawdb: fix transaction indexing/unindexing hashing error (#22457) 2021-03-16 12:15:14 +02:00
database.go core/rawdb: freezer batch write (#23462) 2021-09-07 12:31:17 +02:00
database_test.go eth/protocols/snap: generate storage trie from full dirty snap data (#22668) 2021-04-27 17:19:59 +03:00
freezer.go core/rawdb: freezer batch write (#23462) 2021-09-07 12:31:17 +02:00
freezer_batch.go core/rawdb: freezer batch write (#23462) 2021-09-07 12:31:17 +02:00
freezer_table.go core/rawdb: freezer batch write (#23462) 2021-09-07 12:31:17 +02:00
freezer_table_test.go core/rawdb: freezer batch write (#23462) 2021-09-07 12:31:17 +02:00
freezer_test.go core/rawdb: freezer batch write (#23462) 2021-09-07 12:31:17 +02:00
schema.go core, eth: abort snapshot generation on snap sync and resume later 2021-04-30 17:03:10 +03:00
table.go core/rawdb: freezer batch write (#23462) 2021-09-07 12:31:17 +02:00
table_test.go all: simplify and fix database iteration with prefix/start (#20808) 2020-04-15 14:08:53 +03:00