Commit Graph

301 Commits

Author SHA1 Message Date
Guillaume Ballet 45a3ab42aa
core/state: move slot RLP encoding into the MPT implementation (#27000)
Continuing with a series of PRs to make the Trie interface more generic, this PR moves
the RLP encoding of storage slots inside the StateTrie and light.Trie implementations,
as other types of tries don't use RLP.
2023-06-01 10:29:41 +02:00
rjl493456442 a14301823e
all: new empty trie with types.EmptyRootHash instead of null (#27230) 2023-05-11 10:19:42 +03:00
rjl493456442 5021d36d35
all: port boring changes from pbss (#27176)
* all: port boring changes from pbss

* core, trie: address comments from martin

* trie: minor fixes

* core/rawdb: update comment

* core, eth, tests, trie: address comments

* tests, trie: add extra check when update trie database

* trie/triedb/hashdb: degrade the error to warning
2023-05-09 10:11:04 +03:00
rjl493456442 5d3f5805d5
trie: add node type common package (#27160)
* trie: add node type common package

In trie/types package, a few node wrappers are defined, which will be used
in both trie package, trie/snap package, etc. Therefore, a standalone common
package is created to put these stuffs.

* trie: rename trie/types to trie/trienode
2023-04-26 09:01:54 +03:00
rjl493456442 bbcb5ea37b
core, trie: rework trie database (#26813)
* core, trie: rework trie database

* trie: fix comment
2023-04-24 10:38:52 +03:00
rjl493456442 99f81d2724
all: refactor trie API (#26995)
In this PR, all TryXXX(e.g. TryGet) APIs of trie are renamed to XXX(e.g. Get) with an error returned.

The original XXX(e.g. Get) APIs are renamed to MustXXX(e.g. MustGet) and does not return any error -- they print a log output. A future PR will change the behaviour to panic on errorrs.
2023-04-20 06:57:24 -04:00
Guillaume Ballet 41f89ca944
core/state, trie: remove Try prefix in Trie accessors (#26975)
This change renames StateTrie methods to remove the Try* prefix. 

We added the Trie methods with prefix 'Try' a long time ago, working
around the problem that most existing methods of Trie did not return the
database error. This weird naming convention has persisted until now.

Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2023-03-27 10:48:46 +02:00
Guillaume Ballet 8990c92aea
core/state: add account address to Trie slot accessors (#26934)
This changes the Trie interface to add the plain account address as a
parameter to all storage-related methods.

After the introduction of the TryAccount* functions, TryGet, TryUpdate and
TryDelete are now only meant to read an account's storage. In their current
form, they assume that an account storage is stored in a separate trie, and
that the hashing of the slot is independent of its account's address.

The proposed structure for a stateless storage breaks these two
assumptions: the hashing of a slot key requires the address and all slots
and accounts are stored in a single trie.

This PR therefore adds an address parameter to the interface. It is ignored
in the MPT version, so this change has no functional impact, however it
will reduce the diff size when merging verkle trees.
2023-03-23 11:52:22 +01:00
Marius van der Wijden 81b0aa0cc7
trie: reduce unit test time (#26918) 2023-03-20 04:09:35 -04:00
Darioush Jalali b7bfbc1e64
trie, accounts/abi: add error-checks (#26914) 2023-03-17 06:19:51 -04:00
rjl493456442 c8a6b7100c
core/state, trie: port changes from PBSS (#26763) 2023-03-14 04:50:53 -04:00
rjl493456442 fe01a2f63b
all: use unified emptyRootHash and emptyCodeHash (#26718)
The EmptyRootHash and EmptyCodeHash are defined everywhere in the codebase, this PR replaces all of them with unified one defined in core/types package, and also defines constants for TxRoot, WithdrawalsRoot and UncleRoot
2023-02-21 06:12:27 -05:00
rjl493456442 13ef21d467
Revert "core/trie: remove trie tracer (#26665)" (#26732)
This reverts commit 7c749c947a.
2023-02-20 09:54:52 -05:00
Martin Holst Swende 7c749c947a
core/trie: remove trie tracer (#26665)
This PR contains a small portion of the full pbss PR, namely

    Remove the tracer from trie (and comitter), and instead using an accessList.
    Related changes to the Nodeset.


---------

Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2023-02-19 14:10:19 -05:00
Martin Holst Swende 4d3525610e
all: remove deprecated uses of math.rand (#26710)
This PR is a (superior) alternative to https://github.com/ethereum/go-ethereum/pull/26708, it handles deprecation, primarily two specific cases. 

`rand.Seed` is typically used in two ways
- `rand.Seed(time.Now().UnixNano())` -- we seed it, just to be sure to get some random, and not always get the same thing on every run. This is not needed, with global seeding, so those are just removed. 
- `rand.Seed(1)` this is typically done to ensure we have a stable test. If we rely on this, we need to fix up the tests to use a deterministic prng-source. A few occurrences like this has been replaced with a proper custom source. 

`rand.Read` has been replaced by `crypto/rand`.`Read` in this PR.
2023-02-16 14:36:58 -05:00
Martin Holst Swende 22c3ad1d12
core/state, trie: remove unused error-return from trie Commit operation (#26641) 2023-02-09 08:56:59 -05:00
Péter Szilágyi bf1798e04e
common/prque: generic priority queue (#26290)
* common, core, eth, les, trie: make prque generic

* les/vflux/server: fixed issues in priorityPool

* common, core, eth, les, trie: make priority also generic in prque

* les/flowcontrol: add test case for priority accumulator overflow

* les/flowcontrol: avoid priority value overflow

* common/prque: use int priority in some tests

No need to convert to int64 when we can just change the type used by the
queue.

* common/prque: remove comment about int64 range

---------

Co-authored-by: Zsolt Felfoldi <zsfelfoldi@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-02-09 13:03:54 +02:00
rjl493456442 9842301376
all: remove database commit callback, rework noderesolver (#26637)
This change ports some changes from the main PBSS PR:

  - get rid of callback function in `trie.Database.Commit` which is not required anymore
  - rework the `nodeResolver` in `trie.Iterator` to make it compatible with multiple state scheme
  - some other shallow changes in tests and typo-fixes
2023-02-08 06:14:34 -05:00
Martin Holst Swende 8860b39754
all: prepare for path-based trie storage (#26603)
This PR moves some trie-related db accessor methods to a different file, and also removes the schema type. Instead of the schema type, a string is used to distinguish between hashbased/pathbased db accessors.
This also moves some code from trie package to rawdb package.

This PR is intended to be a no-functionality-change prep PR for #25963 .

---------

Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2023-02-06 10:28:40 -05:00
rjl493456442 6c149fd4ad
core, eth, trie, light: clean up trie interface (#26388)
* all: cleanup trie interface

* eth, trie: address comments
2023-01-03 15:41:40 +02:00
rjl493456442 d3411b9f67
trie: wrap deletion in case trie.root is nil (#26365)
This PR fixes an error in trie commit. If the trie.root is nil, it can be two possible scenarios:

-  The trie was empty, and no change happens
-  The trie was non-empty and all nodes are dropped

For the latter one, we should collect the deletions and apply them into database(e.g. in PBSS).
2022-12-19 04:56:13 -05:00
rjl493456442 743e404906
core, eth, les, tests, trie: abstract node scheme (#25532)
This PR introduces a node scheme abstraction. The interface is only implemented by `hashScheme` at the moment, but will be extended by `pathScheme` very soon.

Apart from that, a few changes are also included which is worth mentioning:

-  port the changes in the stacktrie, tracking the path prefix of nodes during commit
-  use ethdb.Database for constructing trie.Database. This is not necessary right now, but it is required for path-based used to open reverse diff freezer
2022-11-28 14:31:28 +01:00
rjl493456442 60e30a940b
core/rawdb: refactor db inspector for extending multiple ancient store (#25896)
This PR ports a few changes from PBSS:

- Fix the snapshot generator waiter in case the generation is not even initialized
- Refactor db inspector for ancient store
2022-10-28 10:23:49 +02:00
Martin Holst Swende 5a02b2d6d0
all: fix spelling mistakes (#25961) 2022-10-11 09:37:00 +02:00
Martin Holst Swende 456b187892
trie, eth/protocols/snap: less tiny db-writes during trie-healing (#25681)
This PR prevent making tiny writes during state healing, by only performing the batch-write if the accumulated data is large enough to be meaningful.
2022-09-28 08:08:18 +02:00
rjl493456442 bff84a99fe
cmd, core, eth, les, light: track deleted nodes (#25757)
* cmd, core, eth, les, light: track deleted nodes

* trie: add docs

* trie: address comments

* cmd, core, eth, les, light, trie: trie id

* trie: add tests

* trie, core: updates

* trie: fix imports

* trie: add utility print-method for nodeset

* trie: import err

* trie: fix go vet warnings

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-09-27 10:01:02 +02:00
Amir Hossein 57a65f00c9
trie: handle more batch commit errors in Database (#25674) 2022-09-15 23:35:53 +02:00
Felix Lange b628d72766
build: upgrade to go 1.19 (#25726)
This changes the CI / release builds to use the latest Go version. It also
upgrades golangci-lint to a newer version compatible with Go 1.19.

In Go 1.19, godoc has gained official support for links and lists. The
syntax for code blocks in doc comments has changed and now requires a
leading tab character. gofmt adapts comments to the new syntax
automatically, so there are a lot of comment re-formatting changes in this
PR. We need to apply the new format in order to pass the CI lint stage with
Go 1.19.

With the linter upgrade, I have decided to disable 'gosec' - it produces
too many false-positive warnings. The 'deadcode' and 'varcheck' linters
have also been removed because golangci-lint warns about them being
unmaintained. 'unused' provides similar coverage and we already have it
enabled, so we don't lose much with this change.
2022-09-10 13:25:40 +02:00
Darioush Jalali 8363f79f8f
trie: update comments + err check for preimages (#25672)
This PR includes minor updates to comments in trie/committer that reference insertion to the db, and adds an err != nil check for the return value of preimages.commit.
2022-09-08 12:36:07 +02:00
rjl493456442 5ddedd2f83
core, light, trie: remove DiskDB function from trie database (#25690) 2022-09-07 09:08:56 +02:00
Péter Szilágyi a9ec2ab2e6
trie: check childrens' existence concurrently for snap heal 2022-09-06 12:57:03 +03:00
Amir Hossein 4b9c307d26
trie: fix unhandled error in test (#25628) 2022-09-02 18:07:55 +02:00
Seungbae Yu 0d68b6bf10
trie: fix typo in comment (#25667) 2022-09-02 17:48:26 +02:00
Martin Holst Swende d79bd2f0f6
trie: better error reporting (#25645) 2022-09-01 08:41:10 +02:00
rjl493456442 3f79afb599
trie: cleanup stateTrie (#25640)
It's a trivial PR to hide the error log when the trie node is not found in the database. The idea for this change is for all TryXXX functions, the error is already returned and we don't need to fire a log explicitly.

Recently there are a few tickets #25613 #25589 reporting that the trie nodes are missing because of debug.SetHead. The root cause is after resetting, the chain rewinds to a historical point and re-imports the blocks on top.

Since the node is already synced and started to accept transactions previously, these transactions are still kept in the txpool and verified by txpool with a live state. This live state is constructed based on the live trie database, which is changed fast by node referencing and de-referencing.

Unfortunately, when we construct a live state(like the state in txpool), we don't reference the state we have. The blockchain will garbage collect the intermediate version nodes in another thread which leads the broken live state.

The best solution for this is to forcibly obtain a reference for all live states we create and call release function once it's used up. But it might end up with more junks persisted into disk. Will try to find an elegant solution later in the following PR.
2022-08-31 18:18:18 +02:00
aaronbuchwald 3d68bb03c3
trie: fix grammar in comment (#25648) 2022-08-31 16:26:39 +02:00
Péter Szilágyi 5758d1fb11
core/state, trie: fix trie flush order for proper pruning 2022-08-23 21:17:12 +03:00
Justin Traglia 2c5648d891
all: fix some typos (#25551)
* Fix some typos

* Fix some mistakes

* Revert 4byte.json

* Fix an incorrect fix

* Change files to fails
2022-08-19 09:00:21 +03:00
rjl493456442 a1b8892384
trie: improve node rlp decoding performance (#25357)
This avoids copying the input []byte while decoding trie nodes. In most
cases, particularly when the input slice is provided by the underlying
database, this optimization is safe to use.

For cases where the origin of the input slice is unclear, the copying version
is retained. The new code performs better even when the input must be
copied, because it is now only copied once in decodeNode.
2022-08-19 00:39:47 +02:00
Guillaume Ballet 6da5c1644d
core/state, trie, light: add a TryDeleteAccount method (#25531)
* core/state, trie, light: Add a DeleteAccount method

* review feedback

* Update database.go

* pr triage feedback

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
2022-08-17 14:14:49 +03:00
Guillaume Ballet 12185e40e0
core, trie: flush preimages to db on blockchain close (#25533)
* core, trie: flush preimages to db on database close

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>

* rename Close to CommitPreimages for clarity

* core, trie: nitpick fixes

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2022-08-17 14:12:10 +03:00
Guillaume Ballet f67e54c92f
core: use TryGetAccount to read what TryUpdateAccount has written (#25458)
* core: use TryGetAccount to read where TryUpdateAccount has been used to write

* Gary's review feedback

* implement Gary's suggestion

* fix bug + rename NewSecure into NewStateTrie

* trie: add backwards-compatibility aliases for SecureTrie

* Update database.go

* make the linter happy

Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
2022-08-04 16:13:18 +02:00
rjl493456442 8b53b92eb4
core, trie: rework trie committer (#25320)
* all: rework trie and trie committer

* all: get rid of internal cache in trie

* all: fixes

* trie: polish

* core, trie: address comments

* trie: fix imports

* core/state: address comments

* core/state/snapshot: polish

* trie: remove unused code

* trie: update tests

* trie: don't set db as nil

* trie: address comments

* trie: unskip test
2022-08-04 11:03:20 +03:00
rjl493456442 9d76a9b94f
core, trie, eth, cmd: rework preimage store (#25287)
* core, trie, eth, cmd: rework preimage store

* trie: address comment
2022-07-27 20:37:04 +02:00
Delweng b196ad1c16
all: add whitespace linter (#25312)
* golangci: typo

Signed-off-by: Delweng <delweng@gmail.com>

* golangci: add whietspace

Signed-off-by: Delweng <delweng@gmail.com>

* *: rm whitesapce using golangci-lint

Signed-off-by: Delweng <delweng@gmail.com>

* cmd/puppeth: revert accidental resurrection

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2022-07-25 13:14:03 +03:00
Felix Lange e3df3d34cf
trie: fix 'gosimple' lint issue (#25309) 2022-07-15 18:36:05 +02:00
rjl493456442 1657e43931
core, les, eth: port snap sync changes (#24898)
core, eth, les, trie: rework snap sync
2022-07-15 14:55:51 +03:00
aaronbuchwald 926b3e08ba
trie: fix typo in comment (#25241)
paralallel -> parallel
2022-07-06 08:49:09 +02:00
Seungbae.yu 2697e44d81
all: change format `0x%x` to `%#x` (#25221) 2022-07-04 11:03:32 +03:00
aaronbuchwald 241dd27300
trie: fix size accounting in cleaner (#25007)
Decrease children size instead of dirties size when marking dirties as cleaned up in trie cleaner
2022-06-21 12:00:37 +03:00