Commit Graph

15512 Commits

Author SHA1 Message Date
Marius van der Wijden 236147bf70
ethdb: refactor Database interface (#30693) 2024-10-29 10:32:40 +02:00
Péter Szilágyi 7180d26530
core, eth, node: break rawdb -> {leveldb, pebble} dependency (#30689) 2024-10-29 10:31:04 +02:00
Delweng 98056e1ef2
eth/tracers: add disableCode/Storage options for prestateTracer (#30648)
When using the prestateTracer, in some cases users are only concerned
with balances or nonce information, and are not interested in the lengthy
contract code or storage data.

Therefore, this PR introduces two new configuration options in the
`prestateTracerConfig` structure:
- `disableCode`
- `disableStorage`

These options allow users to control whether the tracer returns contract
code and storage data during execution tracing. By setting these
options, users can more flexibly customize their needs and focus on
obtaining information that is more critical and relevant to their
specific use cases.

These options work with the default mode as well as `diffMode: true`.

---------

Signed-off-by: jsvisa <delweng@gmail.com>
Co-authored-by: Sina M <1591639+s1na@users.noreply.github.com>
2024-10-29 07:35:06 +01:00
Péter Szilágyi bce420b99f
cmd/geth: avoid hard coding the IPC name (#30687) 2024-10-28 22:29:25 +02:00
jwasinger c3919f9bda
build: document doGoModTidy function in ci.go (#30685) 2024-10-28 11:26:36 +02:00
Felföldi Zsolt 80bdab757d
ethdb: add DeleteRange feature (#30668)
This PR adds `DeleteRange` to `ethdb.KeyValueWriter`. While range
deletion using an iterator can be really slow, `DeleteRange` is natively
supported by pebble and apparently runs in O(1) time (typically 20-30ms
in my tests for removing hundreds of millions of keys and gigabytes of
data). For leveldb and memorydb an iterator based fallback is
implemented. Note that since the iterator method can be slow and a
database function should not unexpectedly block for a very long time,
the number of deleted keys is limited at 10000 which should ensure that
it does not block for more than a second. ErrTooManyKeys is returned if
the range has only been partially deleted. In this case the caller can
repeat the call until it finally succeeds.
2024-10-25 17:33:46 +02:00
Felföldi Zsolt 6c6bf6fe64
beacon/blsync: add holesky config and update checkpoints (#30671)
This PR adds the beacon chain config for the holesky testnet. It also
updates beacon checkpoints for Mainnet and Sepolia.
2024-10-25 13:20:18 +02:00
jwasinger 24c5493bec
core/vm: remove debug printout in eof test (#30665) 2024-10-24 09:13:01 +02:00
Sina M 461afdf665
core: fix tracing of system calls (#30666)
This change makes it so that the wrapped statedb with tracing-hooks is passed to the system call processing

Fixes #30658
2024-10-24 09:11:47 +02:00
Fredrik Svantes 3e567b8b29
docs: update security policy (#30606)
previous key expired 2023-07-27, the new one expires 2026-02-22:

pub   rsa4096 2016-11-11 [SC] [expires: 2026-02-22]
      AE96ED969E479B0084F3E17FE88D3334FA5F6A0A
uid Ethereum Foundation Security Team <security@ethereum.org>
uid Ethereum Foundation Bug Bounty <bounty@ethereum.org>
sub   rsa4096 2016-11-11 [E] [expires: 2026-02-22]
2024-10-23 15:12:56 +02:00
Shude Li f8f5609b8e
eth/tracers/internal/tracertest: add missing Random to call context (#30652)
Fixes a configuration issue in a test-helper, so that we can do call tracing-tests post-merge
2024-10-23 08:33:14 +02:00
jwasinger 478012ab23
all: remove TerminalTotalDifficultyPassed (#30609)
rebased https://github.com/ethereum/go-ethereum/pull/29766 . The
downstream branch appears to have been deleted and I don't have perms to
push to that fork.

`TerminalTotalDifficultyPassed` is removed. `TerminalTotalDifficulty`
must now be non-nil, and it is expected that networks are already
merged: we can only import PoW/Clique chains, not produce blocks on
them.

---------

Co-authored-by: stevemilk <wangpeculiar@gmail.com>
2024-10-23 08:26:18 +02:00
kevaundray 74461aecf6
crypto, tests/fuzzers: add gnark bn254 precompile methods for fuzzing (#30585)
Makes the gnark precompile methods more amenable to fuzzing
2024-10-23 08:11:25 +02:00
Martin HS 459bb4a647
core/state: move state log mechanism to a separate layer (#30569)
This PR moves the logging/tracing-facilities out of `*state.StateDB`,
in to a wrapping struct which implements `vm.StateDB` instead.

In most places, it is a pretty straight-forward change: 
- First, hoisting the invocations from state objects up to the statedb. 
- Then making the mutation-methods simply return the previous value, so
that the external logging layer could log everything.

Some internal code uses the direct object-accessors to mutate the state,
particularly in testing and in setting up state overrides, which means
that these changes are unobservable for the hooked layer. Thus, configuring
the overrides are not necessarily part of the API we want to publish.

The trickiest part about the layering is that when the selfdestructs are
finally deleted during `Finalise`, there's the possibility that someone
sent some ether to it, which is burnt at that point, and thus needs to
be logged. The hooked layer reaches into the inner layer to figure out
these events.

In package `vm`, the conversion from `state.StateDB + hooks` into a
hooked `vm.StateDB` is performed where needed.

---------

Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2024-10-23 08:03:36 +02:00
Péter Szilágyi a5fe7353cf
common: drop BigMin and BigMax, they pollute our dep graph (#30645)
Way back we've added `common.math.BigMin` and `common.math.BigMax`.
These were kind of cute helpers, but unfortunate ones, because package
all over out codebase added dependencies to this package just to avoid
having to write out 3 lines of code.

Because of this, we've also started having package name clashes with the
stdlib `math`, which got solves even more badly by moving some helpers
over ***from*** the stdlib into our custom lib (e.g. MaxUint64). The
latter ones were nuked out in a previous PR and this PR nukes out BigMin
and BigMax, inlining them at all call sites.

As we're transitioning to uint256, if need be, we can add a min and max
to that.
2024-10-21 12:45:33 +03:00
Péter Szilágyi 31a6418d77
consensus/clique, miner: remove clique -> accounts dependency (#30642)
Clique currently depends on the `accounts` package. This was a bit of a
big cannon even in the past, just to pass a signer "account" to the
Clique block producer. Either way, nowadays Geth does not support clique
mining any more, so by removing that bit of functionality from our code,
we can also break this dependency.

Clique should ideally be further torn out, but this at least gets us one
step closer to cleanups.
2024-10-21 09:24:28 +03:00
Martin HS e4dbd5f685
eth/tracers/js: avoid compiling js bigint when not needed (#30640)
While looking at some mem profiles from `evm` runs, I noticed that
`goja` compilation of the bigint library was present. The bigint library
compilation happens in a package `init`, whenever the package
`eth/tracers/js` is loaded. This PR changes it to load lazily when
needed.

It becomes slightly faster with this change, and slightly less alloc:y. 

Non-scientific benchmark with 100 executions: 
```
time for i in {1..100}; do ./evm --code 6040 run; done;
 ```

current `master`:

```
real    0m6.634s
user    0m5.213s
sys     0m2.277s
```
Without compiling bigint
```
real    0m5.802s
user    0m4.191s
sys     0m1.965s
```
2024-10-20 19:36:51 +03:00
Péter Szilágyi dac54e31a7
build, internal, version: break ci.go/version->common dependency (#30638)
This PR tries to break the ci.go to common dependency by moving the
version number out of params.
2024-10-20 19:28:39 +03:00
Péter Szilágyi 5c3b792e61 common/math: sigh, keep deleting dead code 2024-10-20 15:43:39 +03:00
Péter Szilágyi 9015a05f31
common/math: delete some further dead code (#30639) 2024-10-20 15:38:31 +03:00
Péter Szilágyi bb527b949a
build: get rid of ci.go -> common direct dependency (#30637) 2024-10-20 14:54:06 +03:00
Péter Szilágyi 48d05c43c9
all: get rid of custom MaxUint64 and MaxUint64 (#30636) 2024-10-20 14:41:51 +03:00
Péter Szilágyi babd5d8026
core/state: fix runaway alloc caused by prefetcher heap escape (#30629)
Co-authored-by: lightclient <lightclient@protonmail.com>
2024-10-20 13:25:15 +03:00
rjl493456442 b6c62d5887
core, trie, triedb: minor changes from snapshot integration (#30599)
This change ports some non-important changes from https://github.com/ethereum/go-ethereum/pull/30159, including interface renaming and some trivial refactorings.
2024-10-18 17:06:31 +02:00
Péter Szilágyi 3ff73d46b3
build: reenable building arm64 concurrently (#30626) 2024-10-18 14:49:27 +03:00
Péter Szilágyi 9891f02d48
gitignore: get rid of some relics (#30623)
Clean out some ancient stuff from git ignore.
2024-10-18 12:02:32 +03:00
Péter Szilágyi f32f8686cd
swarm: nuke this leftover (#30622)
Swarm moved out more than 5 years ago, time to let it go.
2024-10-18 11:34:46 +03:00
Péter Szilágyi afea3bd49c
beacon/engine, core/txpool, eth/catalyst: add engine_getBlobsV1 API (#30537) 2024-10-17 19:27:35 +03:00
lightclient e26468f6f6
beacon/engine,eth/catalyst: hex marshal requests in engine api (#30603)
Co-authored-by: Felix Lange <fjl@twurst.com>
2024-10-17 10:05:17 +02:00
lightclient 1da34a37ec
miner: send full request when resolving full payload (#30615)
Fixes an issue missed in #30576 where we send empty requests for a full
payload being resolved, causing hash mismatch later on when we get the
payload back via `NewPayload`.
2024-10-17 10:04:20 +02:00
Sina M 978ca5fc5e
eth/tracers: various fixes (#30540)
Breaking changes:

- The ChainConfig was exposed to tracers via VMContext passed in
`OnTxStart`. This is unnecessary specially looking through the lens of
live tracers as chain config remains the same throughout the lifetime of
the program. It was there so that native API-invoked tracers could
access it. So instead we moved it to the constructor of API tracers.

Non-breaking:

- Change the default config of the tracers to be `{}` instead of nil.
This way an extra nil check can be avoided.

Refactoring:

- Rename `supply` struct to `supplyTracer`.
- Un-export some hook definitions.
2024-10-17 06:51:47 +02:00
Marius van der Wijden 18a591811f
core: reduce peak memory usage during reorg (#30600)
~~Opening this as a draft to have a discussion.~~ Pressed the wrong
button
I had [a previous PR
](https://github.com/ethereum/go-ethereum/pull/24616)a long time ago
which reduced the peak memory used during reorgs by not accumulating all
transactions and logs.
This PR reduces the peak memory further by not storing the blocks in
memory.
However this means we need to pull the blocks back up from storage
multiple times during the reorg.
I collected the following numbers on peak memory usage: 

// Master: BenchmarkReorg-8 10000 899591 ns/op 820154 B/op 1440
allocs/op 1549443072 bytes of heap used
// WithoutOldChain: BenchmarkReorg-8 10000 1147281 ns/op 943163 B/op
1564 allocs/op 1163870208 bytes of heap used
// WithoutNewChain: BenchmarkReorg-8 10000 1018922 ns/op 943580 B/op
1564 allocs/op 1171890176 bytes of heap used

Each block contains a transaction with ~50k bytes and we're doing a 10k
block reorg, so the chain should be ~500MB in size

---------

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2024-10-16 19:46:40 +03:00
Péter Szilágyi 368e16f39d
core, eth, ethstats: simplify chain head events (#30601) 2024-10-16 10:32:58 +03:00
rjl493456442 15bf90ebc5
core, ethdb/pebble: run pebble in non-sync mode (#30573)
Implements https://github.com/ethereum/go-ethereum/issues/29819
2024-10-15 18:10:03 +03:00
Péter Szilágyi a44905763e
ethdb/pebble: switch to increasing level sizes (#30602) 2024-10-15 17:00:14 +03:00
Roman Krasiuk 4c4219e405
beacon/engine: omit null witness field from payload envelope (#30597)
## Description

Omit null `witness` field from payload envelope.

## Motivation

Currently, JSON encoded payload types always include `"witness": null`,
which, I believe, is not intentional.
2024-10-15 11:51:20 +03:00
Alex Gartner 30ce17386b
crypto: use decred secp256k1 directly (#30595)
Use `github.com/decred/dcrd/dcrec/secp256k1/v4` directly rather than
`github.com/btcsuite/btcd/btcec/v2` which is just a wrapper around the
underlying decred library. Inspired by
https://github.com/cosmos/cosmos-sdk/pull/15018

`github.com/btcsuite/btcd/btcec/v2` has a very annoying breaking change
when upgrading from `v2.3.3` to `v2.3.4`. The easiest way to workaround
this is to just remove the wrapper.

Would be very nice if you could backport this to the release branches.

References:
- https://github.com/btcsuite/btcd/issues/2221
- https://github.com/cometbft/cometbft/pull/4294
- https://github.com/cometbft/cometbft/pull/3728
- https://github.com/zeta-chain/node/pull/2934
2024-10-15 11:49:08 +03:00
jwasinger 4b9c7821b9
internal/ethapi: refactor `TxArgs.setCancunFeeDefaults` (#30541)
calculating a reasonable tx blob fee cap (`max_blob_fee_per_gas *
total_blob_gas`) only depends on the excess blob gas of the parent
header. The parent header is assumed to be correct, so the method should
not be able to fail and return an error.
2024-10-15 10:02:02 +02:00
Felix Lange add5709cb5
beacon/engine: strip type byte in requests (#30576)
This change brings geth into compliance with the current engine API
specification for the Prague fork. I have moved the assignment of
ExecutionPayloadEnvelope.Requests into BlockToExecutableData to ensure
there is a single place where the type is removed.

While doing so, I noticed that handling of requests in the miner was not
quite correct for the empty payload. It would return `nil` requests for
the empty payload even for blocks after the Prague fork. To fix this, I
have added the emptyRequests field in miner.Payload.
2024-10-14 21:43:35 +02:00
Martin HS 5adc314817
build: update to golangci-lint 1.61.0 (#30587)
Changelog: https://golangci-lint.run/product/changelog/#1610 

Removes `exportloopref` (no longer needed), replaces it with
`copyloopvar` which is basically the opposite.

Also adds: 
- `durationcheck`
- `gocheckcompilerdirectives`
- `reassign`
- `mirror`
- `tenv`

---------

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
2024-10-14 19:25:22 +02:00
Martin HS f4dc7530b1
trie: concurrent commit (#30545)
This change makes the trie commit operation concurrent, if the number of changes exceed 100. 

Co-authored-by: stevemilk <wangpeculiar@gmail.com>
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2024-10-14 13:32:15 +02:00
Felix Lange 16f64098b9
core: enable EIP-2935 in chain maker (#30575) 2024-10-13 18:51:51 +02:00
lightclient 2246d66135
cmd/evm: fixup issues with requests in t8n (#30584)
This fixes a few issues missed in #29052:

* `requests` must be hex encoded, so added a helper to marshal.
* The statedb was committed too early and so the result of the system
calls was lost.
* For devnet-4 we need to pull off the type byte prefix from the request
data.
2024-10-13 18:47:51 +02:00
Hteev Oli fad7e74a1b
build: add support for ubuntu 24.10 (#30580) 2024-10-12 12:25:58 +03:00
Felix Lange 3a5313f3f3
all: implement EIP-7002 & EIP-7251 (#30571)
This is a redo of #29052 based on newer specs. Here we implement EIPs
scheduled for the Prague fork:

- EIP-7002: Execution layer triggerable withdrawals
- EIP-7251: Increase the MAX_EFFECTIVE_BALANCE

Co-authored-by: lightclient <lightclient@protonmail.com>
2024-10-11 21:36:13 +02:00
Karol Chojnowski 16bf471151
core/tracing: add GetTransientState method to StateDB interface (#30531)
Allows live custom tracers to access contract transient storage through the StateDB interface.
2024-10-10 13:03:03 +02:00
Shude Li 5b393ac85a
eth/protocols/eth: remove Requests in block body (#30562)
Block no longer has Requests. This PR just removes some code that wasn't removed in #30425.
2024-10-10 10:47:40 +02:00
Martin HS 58cf152e98
eth/catalyst, core/txpool/blobpool: make tests output less logs (#30563)
A couple of tests set the debug level to `TRACE` on stdout,
and all subsequent tests in the same package are also affected
by that, resulting in outputs of tens of megabytes. 

This PR removes such calls from two packages where it was prevalent.
This makes getting a summary of failing tests simpler, and possibly
reduces some strain from the CI pipeline.
2024-10-10 07:54:07 +02:00
easyfold 7942a6b5ad
eth/tracers: do system contract processing prior to parallel-tracing (#30520)
This fixes `debug_traceBlock` methods for JS tracers in that it correctly
applies the beacon block root processing to the state.
2024-10-09 14:45:14 +02:00
Felix Lange 2936b41514
all: implement flat deposit requests encoding (#30425)
This implements recent changes to EIP-7685, EIP-6110, and
execution-apis.

---------

Co-authored-by: lightclient <lightclient@protonmail.com>
Co-authored-by: Shude Li <islishude@gmail.com>
2024-10-09 12:24:58 +02:00