Commit Graph

15537 Commits

Author SHA1 Message Date
dependabot[bot] 31bc9e65bc build(deps): bump github.com/golang-jwt/jwt/v4 from 4.5.0 to 4.5.1 (#30728)
Bumps [github.com/golang-jwt/jwt/v4](https://github.com/golang-jwt/jwt) from 4.5.0 to 4.5.1.


Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-19 14:50:11 +01:00
Marius van der Wijden d01591ccb5 cmd/utils: change blssync.JWTSecretFlag to DirectoryFlag (#30729)
closes https://github.com/ethereum/go-ethereum/issues/30304

We already use `DirectoryFlag` for `authrpc.jwtsecret` which expands the
tilde, so this should work out of the box
2024-11-19 14:50:11 +01:00
Martin HS b04d6c4f42 core/state: small fix in hooked statedb (#30732)
fixes a very tiny bug
2024-11-19 14:50:11 +01:00
Delweng 8a3a697722 eth/tracers: fill the creationMethod in flatCall (#30539)
`flatCallTracer` will now specify the type of a create in the action
via the `creationMethod` field.

---------

Signed-off-by: jsvisa <delweng@gmail.com>
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
2024-11-19 14:50:11 +01:00
Martin HS 6578b9aac5 all: fix issues with benchmarks (#30667)
This PR fixes some issues with benchmarks

- [x] Removes log output from a log-test
- [x] Avoids a `nil`-defer in `triedb/pathdb`
- [x] Fixes some crashes re tracers
- [x] Refactors a very resource-expensive benchmark for blobpol.
**NOTE**: this rewrite touches live production code (a little bit), as
it makes the validator-function used by the blobpool configurable.
- [x] Switch some benches over to use pebble over leveldb
- [x] reduce mem overhead in the setup-phase of some tests
- [x] Marks some tests with a long setup-phase to be skipped if `-short`
is specified (where long is on the order of tens of seconds). Ideally,
in my opinion, one should be able to run with `-benchtime 10ms -short`
and sanity-check all tests very quickly.
- [x]  Drops some metrics-bechmark which times the speed of `copy`.

---------

Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
2024-11-19 14:50:11 +01:00
Guillaume Ballet ed01e6fb2e core, trie: verkle state processor tests (#30672)
Tests that are crucial to for verifying the verkle testnet functions properly.

---------

Signed-off-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>
Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com>
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
Co-authored-by: Martin HS <martin@swende.se>
2024-11-19 14:50:11 +01:00
Martin HS b45ba07040 core/vm/runtime: invoke tx-end hook (#30711)
When using the `core/vm/runtime` helpers to execute code, callbacks for the tx end were not invoked. This change fixes it by invoking them.
2024-11-19 14:50:11 +01:00
Péter Szilágyi 9bb5194166 eth/catalyst: make engine api test time independent (#30713)
This test depends on a 100ms timer, which fails quite often, messing up
our pipelines. Hook directly into the internal version of getPayload
which has the capacity to wait for the full payload before returning.
This might not be absolutely correct from a test perspective, but it
beats failing ci. The alternative would be to expose the full build hook
into the outside, but it might be a bit overkill for this scenario.
2024-11-19 14:50:11 +01:00
piersy 5de5f3344a core/txpool: improve error responses with wrapped errors (#30715) 2024-11-19 14:50:11 +01:00
Martin HS 3adb076a2f tests/fuzzers/bls12381: more verbose fuzzing-output (#30724)
This PR updates the fuzzing verbosity a bit, in case of mismatches
2024-11-19 14:50:11 +01:00
zhiqiangxu 1fe607bf4f build: use slices.Clone for copying slice (#30716) 2024-11-19 14:50:11 +01:00
Delweng b92e17f5a8 eth/tracers: flatCallTracer error compatible with parity (#30497)
Compatible error message in the flat call tracer with parity-style endpoints.

Signed-off-by: jsvisa <delweng@gmail.com>
2024-11-19 14:50:11 +01:00
Martin HS 21b3400ae5 all: remove `personal` RPC namespace (#30704)
This PR is a first step towards removing account management from geth,
and contains a lot of the user-facing changes.

With this PR, the `personal` namespace disappears. **Note**: `personal`
namespace has been deprecated for quite some time (since
https://github.com/ethereum/go-ethereum/pull/26390 1 year and 8 months
ago), and users who have wanted to use it has been forced to used the
flag `--rpc.enabledeprecatedpersonal`. So I think it's fairly
non-controversial to drop it at this point.

Specifically, this means: 

- Account/wallet listing
  -`personal.getListAccounts`  
  -`personal.listAccounts`     
  -`personal.getListWallets`   
  -`personal.listWallets`      
- Lock/unlock
  -`personal.lockAccount`      
  -`personal.openWallet`       
  -`personal.unlockAccount`
- Sign ops
  -`personal.sign`             
  -`personal.sendTransaction`  
  -`personal.signTransaction`  
- Imports / inits
  -`personal.deriveAccount`    
  -`personal.importRawKey`     
  -`personal.initializeWallet` 
  -`personal.newAccount`       
  -`personal.unpair` 
- Other: 
  -`personal.ecRecover`        


The underlying keystores and account managent code is still in place,
which means that `geth --dev` still works as expected, so that e.g. the
example below still works:

```
> eth.sendTransaction({data:"0x6060", value: 1, from:eth.accounts[0]})
```	

Also, `ethkey` and `clef` are untouched. 

With the removal of `personal`, as far as I know we have no more API
methods which contain credentials, and if we want to implement
logging-capabilities of RPC ingress payload, it would be possible after
this.

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
2024-11-19 14:50:11 +01:00
Péter Szilágyi 750250387e internal/flags: remove low-use type TextMarshalerFlag (#30707)
Currently we have a custom TextMarshalerFlag. It's a nice idea, allowing
anything implementing text marshaller to be used as a flag. That said,
we only ever used it in one place because it's not that obvious how to
use and it needs some boilerplate on the type itself too, apart of the
heavy boilerplate got the custom flag.

All in all there's no *need* to drop this feature just now, but while
porting the cmds over to cli @v3, all other custom flags worker
perfectly, whereas this one started crashing deep inside the cli
package. The flag handling in v3 got rebuild on generics and there are a
number of new methods needed; and my guess is that maybe one of them
doesn't work like this flag currently is designed too.

We could definitely try and redesign this flag for cli v3... but all
that effort and boilerplate just to use it for 1 flag in 1 location,
seems not worth it. So for now I'm suggesting removing it and maybe
reconsider a similar feature in cli v3 with however it will work.
2024-11-19 14:50:11 +01:00
Péter Szilágyi 9f58dc0c32 internal/flags: remove Merge, it's identical to slices.Concat (#30706)
This is a noop change to not have custom code for stdlib functionality.
2024-11-19 14:50:11 +01:00
Péter Szilágyi 83def9b8f5 cmd/utils, eth/ethconfig: remove some ancient leftover flag (#30705)
This is a flag leftover from the swarm era. No need to deprecate it,
it's been useless/dead forever now.
2024-11-19 14:50:11 +01:00
lightclient 0d6d79fd4e core: add code to witness when state object is accessed (#30698)
I think the core code should generally be agnostic about the witness and
the statedb layer should determine what elements need to be included in
the witness. Because code is accessed via `GetCode`, and
`GetCodeLength`, the statedb will always know when it needs to add that
code into the witness.

The edge case is block hashes, so we continue to add them manually in
the implementation of `BLOCKHASH`.

It probably makes sense to refactor statedb so we have a wrapped
implementation that accumulates the witness, but this is a simpler
change that makes #30078 less aggressive.
2024-11-19 14:50:11 +01:00
Martin HS b1defd4433 core/vm: speed up push and interpreter loop (#30662)
Looking at the cpu profile of a burntpix benchmark, I noticed that a lot
of time was spent in gas-used, in the interpreter loop. It's an actual
call (not inlined), which explicitly wants to be ignored by tracing
("tracing.GasChangeIgnored"), so it can be safely and simply inlined.

The other change is in `pushX`. These also do a call to
`common.RightPadBytes`. I replaced that by a doing a corresponding `Lsh`
on the `u256` if needed. Note: it's needed only to make the stack output
look right, for fuzzers. It technically doesn't matter what we put
there: if code ends on a pushdata immediate, nothing will consume the
stack element. We could just as well just ignore it, if we didn't care
about fuzzers (which I do).

Seems quite a lot faster on burntpix, according to my runs. 

This PR:
```
EVM gas used:    5642735088
execution time:  34.84609475s
allocations:     915683
allocated bytes: 175334088
```
```
EVM gas used:    5642735088
execution time:  36.671958278s
allocations:     915701
allocated bytes: 175340528
```

Master
```
EVM gas used:    5642735088
execution time:  49.349209526s
allocations:     915684
allocated bytes: 175333368
```
```
EVM gas used:    5642735088
execution time:  46.581006598s
allocations:     915681
allocated bytes: 175330728
```

---------

Co-authored-by: Sina M <1591639+s1na@users.noreply.github.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
2024-11-19 14:50:11 +01:00
zhiqiangxu f124ce3ff9 beacon/light: remove unused CommitteeChain.signerThreshold (#30484)
This field is a duplicate of UpdateScore.SignerCount and never referenced.
2024-11-19 14:50:11 +01:00
Péter Szilágyi 8de45c1820 appveyor, build, internal: ci.go cleanups, add package dep checker (#30696) 2024-11-19 14:50:11 +01:00
Marius van der Wijden 3017efedc2 ethdb: refactor Database interface (#30693) 2024-11-19 14:50:11 +01:00
Péter Szilágyi b7767efed0 core, eth, node: break rawdb -> {leveldb, pebble} dependency (#30689) 2024-11-19 14:50:11 +01:00
Delweng f3cbacf373 eth/tracers: add disableCode/Storage options for prestateTracer (#30648)
When using the prestateTracer, in some cases users are only concerned
with balances or nonce information, and are not interested in the lengthy
contract code or storage data.

Therefore, this PR introduces two new configuration options in the
`prestateTracerConfig` structure:
- `disableCode`
- `disableStorage`

These options allow users to control whether the tracer returns contract
code and storage data during execution tracing. By setting these
options, users can more flexibly customize their needs and focus on
obtaining information that is more critical and relevant to their
specific use cases.

These options work with the default mode as well as `diffMode: true`.

---------

Signed-off-by: jsvisa <delweng@gmail.com>
Co-authored-by: Sina M <1591639+s1na@users.noreply.github.com>
2024-11-19 14:50:11 +01:00
Péter Szilágyi 856f807b9b cmd/geth: avoid hard coding the IPC name (#30687) 2024-11-19 14:50:11 +01:00
jwasinger 571628a114 build: document doGoModTidy function in ci.go (#30685) 2024-11-19 14:50:11 +01:00
Felföldi Zsolt a7e2b8be63 ethdb: add DeleteRange feature (#30668)
This PR adds `DeleteRange` to `ethdb.KeyValueWriter`. While range
deletion using an iterator can be really slow, `DeleteRange` is natively
supported by pebble and apparently runs in O(1) time (typically 20-30ms
in my tests for removing hundreds of millions of keys and gigabytes of
data). For leveldb and memorydb an iterator based fallback is
implemented. Note that since the iterator method can be slow and a
database function should not unexpectedly block for a very long time,
the number of deleted keys is limited at 10000 which should ensure that
it does not block for more than a second. ErrTooManyKeys is returned if
the range has only been partially deleted. In this case the caller can
repeat the call until it finally succeeds.
2024-11-19 14:50:11 +01:00
Felföldi Zsolt a7c01a235b beacon/blsync: add holesky config and update checkpoints (#30671)
This PR adds the beacon chain config for the holesky testnet. It also
updates beacon checkpoints for Mainnet and Sepolia.
2024-11-19 14:50:11 +01:00
jwasinger 6a3f575fa7 core/vm: remove debug printout in eof test (#30665) 2024-11-19 14:50:11 +01:00
Sina M 0d1d689c37 core: fix tracing of system calls (#30666)
This change makes it so that the wrapped statedb with tracing-hooks is passed to the system call processing

Fixes #30658
2024-11-19 14:50:11 +01:00
Fredrik Svantes 99ec8e4545 docs: update security policy (#30606)
previous key expired 2023-07-27, the new one expires 2026-02-22:

pub   rsa4096 2016-11-11 [SC] [expires: 2026-02-22]
      AE96ED969E479B0084F3E17FE88D3334FA5F6A0A
uid Ethereum Foundation Security Team <security@ethereum.org>
uid Ethereum Foundation Bug Bounty <bounty@ethereum.org>
sub   rsa4096 2016-11-11 [E] [expires: 2026-02-22]
2024-11-19 14:50:11 +01:00
Shude Li e9e1cb0784 eth/tracers/internal/tracertest: add missing Random to call context (#30652)
Fixes a configuration issue in a test-helper, so that we can do call tracing-tests post-merge
2024-11-19 14:50:11 +01:00
jwasinger 53e3f445ed all: remove TerminalTotalDifficultyPassed (#30609)
rebased https://github.com/ethereum/go-ethereum/pull/29766 . The
downstream branch appears to have been deleted and I don't have perms to
push to that fork.

`TerminalTotalDifficultyPassed` is removed. `TerminalTotalDifficulty`
must now be non-nil, and it is expected that networks are already
merged: we can only import PoW/Clique chains, not produce blocks on
them.

---------

Co-authored-by: stevemilk <wangpeculiar@gmail.com>
2024-11-19 14:50:11 +01:00
kevaundray cba2e77fe2 crypto, tests/fuzzers: add gnark bn254 precompile methods for fuzzing (#30585)
Makes the gnark precompile methods more amenable to fuzzing
2024-11-19 14:50:11 +01:00
Martin HS d7434facba core/state: move state log mechanism to a separate layer (#30569)
This PR moves the logging/tracing-facilities out of `*state.StateDB`,
in to a wrapping struct which implements `vm.StateDB` instead.

In most places, it is a pretty straight-forward change: 
- First, hoisting the invocations from state objects up to the statedb. 
- Then making the mutation-methods simply return the previous value, so
that the external logging layer could log everything.

Some internal code uses the direct object-accessors to mutate the state,
particularly in testing and in setting up state overrides, which means
that these changes are unobservable for the hooked layer. Thus, configuring
the overrides are not necessarily part of the API we want to publish.

The trickiest part about the layering is that when the selfdestructs are
finally deleted during `Finalise`, there's the possibility that someone
sent some ether to it, which is burnt at that point, and thus needs to
be logged. The hooked layer reaches into the inner layer to figure out
these events.

In package `vm`, the conversion from `state.StateDB + hooks` into a
hooked `vm.StateDB` is performed where needed.

---------

Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2024-11-19 14:50:11 +01:00
Péter Szilágyi e5fb824e67 common: drop BigMin and BigMax, they pollute our dep graph (#30645)
Way back we've added `common.math.BigMin` and `common.math.BigMax`.
These were kind of cute helpers, but unfortunate ones, because package
all over out codebase added dependencies to this package just to avoid
having to write out 3 lines of code.

Because of this, we've also started having package name clashes with the
stdlib `math`, which got solves even more badly by moving some helpers
over ***from*** the stdlib into our custom lib (e.g. MaxUint64). The
latter ones were nuked out in a previous PR and this PR nukes out BigMin
and BigMax, inlining them at all call sites.

As we're transitioning to uint256, if need be, we can add a min and max
to that.
2024-11-19 14:50:11 +01:00
Péter Szilágyi c6823c77a3 consensus/clique, miner: remove clique -> accounts dependency (#30642)
Clique currently depends on the `accounts` package. This was a bit of a
big cannon even in the past, just to pass a signer "account" to the
Clique block producer. Either way, nowadays Geth does not support clique
mining any more, so by removing that bit of functionality from our code,
we can also break this dependency.

Clique should ideally be further torn out, but this at least gets us one
step closer to cleanups.
2024-11-19 14:50:11 +01:00
Martin HS 26e1470393 eth/tracers/js: avoid compiling js bigint when not needed (#30640)
While looking at some mem profiles from `evm` runs, I noticed that
`goja` compilation of the bigint library was present. The bigint library
compilation happens in a package `init`, whenever the package
`eth/tracers/js` is loaded. This PR changes it to load lazily when
needed.

It becomes slightly faster with this change, and slightly less alloc:y. 

Non-scientific benchmark with 100 executions: 
```
time for i in {1..100}; do ./evm --code 6040 run; done;
 ```

current `master`:

```
real    0m6.634s
user    0m5.213s
sys     0m2.277s
```
Without compiling bigint
```
real    0m5.802s
user    0m4.191s
sys     0m1.965s
```
2024-11-19 14:50:11 +01:00
Péter Szilágyi db03e01577 build, internal, version: break ci.go/version->common dependency (#30638)
This PR tries to break the ci.go to common dependency by moving the
version number out of params.
2024-11-19 14:50:11 +01:00
Péter Szilágyi b7d8f477db common/math: sigh, keep deleting dead code 2024-11-19 14:50:11 +01:00
Péter Szilágyi 997d122cbf common/math: delete some further dead code (#30639) 2024-11-19 14:50:11 +01:00
Péter Szilágyi 1c88e42ea9 build: get rid of ci.go -> common direct dependency (#30637) 2024-11-19 14:50:11 +01:00
Péter Szilágyi 7a79f2f89f all: get rid of custom MaxUint64 and MaxUint64 (#30636) 2024-11-19 14:50:11 +01:00
Péter Szilágyi d03ed72022 core/state: fix runaway alloc caused by prefetcher heap escape (#30629)
Co-authored-by: lightclient <lightclient@protonmail.com>
2024-11-19 14:50:11 +01:00
rjl493456442 6412beedc4 core, trie, triedb: minor changes from snapshot integration (#30599)
This change ports some non-important changes from https://github.com/ethereum/go-ethereum/pull/30159, including interface renaming and some trivial refactorings.
2024-11-19 14:50:11 +01:00
Péter Szilágyi 56d14b228c build: reenable building arm64 concurrently (#30626) 2024-11-19 14:50:11 +01:00
Péter Szilágyi ac3ebe240d gitignore: get rid of some relics (#30623)
Clean out some ancient stuff from git ignore.
2024-11-19 14:50:11 +01:00
Péter Szilágyi 44c44c7534 swarm: nuke this leftover (#30622)
Swarm moved out more than 5 years ago, time to let it go.
2024-11-19 14:50:11 +01:00
Péter Szilágyi cfd1489b92 beacon/engine, core/txpool, eth/catalyst: add engine_getBlobsV1 API (#30537) 2024-11-19 14:50:11 +01:00
lightclient 1660698efc beacon/engine,eth/catalyst: hex marshal requests in engine api (#30603)
Co-authored-by: Felix Lange <fjl@twurst.com>
2024-11-19 14:50:11 +01:00
lightclient 5d41ac58a8 miner: send full request when resolving full payload (#30615)
Fixes an issue missed in #30576 where we send empty requests for a full
payload being resolved, causing hash mismatch later on when we get the
payload back via `NewPayload`.
2024-11-19 14:50:11 +01:00