go-ethereum/core/rawdb
rjl493456442 0ad0966cec
core/rawdb: introduce flush offset in freezer (#30392)
This is a follow-up PR to #29792 to get rid of the data file sync.

**This is a non-backward compatible change, which increments the
database version from 8 to 9**.

We introduce a flushOffset for each freezer table, which tracks the position
of the most recently fsync’d item in the index file. When this offset moves
forward, it indicates that all index entries below it, along with their corresponding
data items, have been properly persisted to disk. The offset can also be moved
backward when truncating from either the head or tail of the file.

Previously, the data file required an explicit fsync after every mutation, which
was highly inefficient. With the introduction of the flush offset, the synchronization
strategy becomes more flexible, allowing the freezer to sync every 30 seconds
instead.

The data items above the flush offset are regarded volatile and callers must ensure
they are recoverable after the unclean shutdown, or explicitly sync the freezer
before any proceeding operations.

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
2025-02-04 11:45:45 +01:00
..
ancienttest core/rawdb: implement in-memory freezer (#29135) 2024-04-30 11:33:22 +02:00
testdata core/rawdb: avoid unnecessary receipt processing for log filtering (#23147) 2021-09-28 12:54:49 +02:00
accessors_chain.go all: nuke total difficulty (#30744) 2025-01-28 18:55:41 +01:00
accessors_chain_test.go core/rawdb: introduce flush offset in freezer (#30392) 2025-02-04 11:45:45 +01:00
accessors_indexes.go core/types, params: add blob transaction type, RLP encoded for now (#27049) 2023-04-21 12:52:02 +03:00
accessors_indexes_test.go ethdb: refactor Database interface (#30693) 2024-10-29 10:32:40 +02:00
accessors_metadata.go Revert "core/rawdb: remove unused transition status state accessors" (#30449) 2024-09-18 11:53:50 +03:00
accessors_snapshot.go all: fix various typos (#29542) 2024-04-16 15:44:00 +03:00
accessors_state.go all: implement path-based state scheme (#25963) 2023-08-01 15:17:32 +03:00
accessors_sync.go core, accounts, eth, trie: handle genesis state missing (#28171) 2023-09-28 10:00:53 +03:00
accessors_trie.go core/rawdb: make sure specified state scheme is valid (#30499) 2024-09-24 09:26:29 +02:00
ancient_scheme.go core/rawdb: introduce flush offset in freezer (#30392) 2025-02-04 11:45:45 +01:00
ancient_utils.go core/rawdb, triedb, cmd: create an isolated disk namespace for verkle (#30105) 2024-07-16 16:17:58 +03:00
chain_freezer.go all: nuke total difficulty (#30744) 2025-01-28 18:55:41 +01:00
chain_iterator.go core, core/rawdb, eth/sync: no tx indexing during snap sync (#28703) 2024-01-22 21:05:18 +01:00
chain_iterator_test.go all: refactor so `NewBlock`, `WithBody` take `types.Body` (#29482) 2024-04-30 14:55:08 +02:00
database.go all: nuke total difficulty (#30744) 2025-01-28 18:55:41 +01:00
database_test.go all: update license headers and AUTHORS from git history (#24947) 2022-05-24 20:39:40 +02:00
freezer.go core/rawdb: fix panic in freezer (#30973) 2025-01-06 14:52:01 +08:00
freezer_batch.go core/rawdb: introduce flush offset in freezer (#30392) 2025-02-04 11:45:45 +01:00
freezer_memory.go ethdb: refactor Database interface (#30693) 2024-10-29 10:32:40 +02:00
freezer_memory_test.go core/rawdb: implement in-memory freezer (#29135) 2024-04-30 11:33:22 +02:00
freezer_meta.go core/rawdb: introduce flush offset in freezer (#30392) 2025-02-04 11:45:45 +01:00
freezer_meta_test.go core/rawdb: introduce flush offset in freezer (#30392) 2025-02-04 11:45:45 +01:00
freezer_resettable.go ethdb: refactor Database interface (#30693) 2024-10-29 10:32:40 +02:00
freezer_resettable_test.go core/rawdb: implement in-memory freezer (#29135) 2024-04-30 11:33:22 +02:00
freezer_table.go core/rawdb: introduce flush offset in freezer (#30392) 2025-02-04 11:45:45 +01:00
freezer_table_test.go core/rawdb: introduce flush offset in freezer (#30392) 2025-02-04 11:45:45 +01:00
freezer_test.go core/rawdb: drop MigrateTable (#30331) 2024-08-20 14:06:00 +02:00
freezer_utils.go core/rawdb: fsync the index file after each freezer write (#28483) 2023-11-10 12:56:39 +02:00
freezer_utils_test.go all: more linters (#24783) 2022-06-13 16:24:45 +02:00
key_length_iterator.go core/rawdb: add specified key length iterator (#24535) 2022-03-15 10:28:26 +02:00
key_length_iterator_test.go core/rawdb: add specified key length iterator (#24535) 2022-03-15 10:28:26 +02:00
schema.go all: nuke total difficulty (#30744) 2025-01-28 18:55:41 +01:00
table.go ethdb: add DeleteRange feature (#30668) 2024-10-25 17:33:46 +02:00
table_test.go all: simplify and fix database iteration with prefix/start (#20808) 2020-04-15 14:08:53 +03:00