This PR refactors the structlog a bit, making it so that it can be used
in a streaming mode.
-------------
OBS: this PR makes a change in the input `config` config, the third
input-parem field to `debug.traceCall`. Previously, seteting it to e.g.
` {"enableMemory": true, "limit": 1024}` would mean that the response
was limited to `1024` items. Since an 'item' may include both memory and
storage, the actual size of the response was undertermined.
After this change, the response will be limited to `1024` __`bytes`__
(or thereabouts).
-----------
The commandline usage of structlog now uses the streaming mode, leaving
the non-streaming mode of operation for the eth_Call.
There are two benefits of streaming mode
1. Not have to maintain a long list of operations,
2. Not have to duplicate / n-plicate data, e.g. memory / stack /
returndata so that each entry has their own private slice.
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
Lots of packages depend on eth/downloader just for the SyncMode type.
Since we have a dedicated package for eth protocol configuration, it
makes more sense to define SyncMode there, turning eth/downloader into
more of a leaf package.
* unify `staterunner` and `blockrunner` CLI flags, especially around
tracing
* added support for struct logger or json logging (although having issue
#30658)
* new --cross-check flag to validate the stateless witness collection
/ execution matches stateful
* adds support for tracing the stateless execution when a tracer is set
(to more easily debug differences)
* --human for more readable test summary
* directory or file input, so if you pass tests/spec-tests/fixtures/blockchain_tests it will execute all
blockchain tests
This change relocates the EVM tx context switching to the ApplyMessage function.
With this change, we can remove a lot of EVM.SetTxContext calls before
message execution.
### Tracing API changes
- This PR replaces the `GasPrice` field of the `VMContext` struct with
`BaseFee`. Users may instead take the effective gas price from
`tx.EffectiveGasTipValue(env.BaseFee)`.
---------
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
Since we don't really support custom networks anymore, we don't need the
bootnode utility. In case a discovery-only node is wanted, it can still be run using cmd/devp2p.
Reusing state between benchmark iterations resulted in inconsistent
results across runs, which surfaced in https://github.com/ethereum/go-ethereum/issues/30778 .
If these errors are triggered again, they will now trigger panic.
---------
Co-authored-by: Martin HS <martin@swende.se>
This pull request refactors the EVM constructor by removing the
TxContext parameter.
The EVM object is frequently overused. Ideally, only a single EVM
instance should be created and reused throughout the entire state
transition of a block, with the transaction context switched as needed
by calling evm.SetTxContext.
Unfortunately, in some parts of the code, the EVM object is repeatedly
created, resulting in unnecessary complexity. This pull request is the
first step towards gradually improving and simplifying this setup.
---------
Co-authored-by: Martin Holst Swende <martin@swende.se>
This is one further step towards removing account management from
`geth`. This PR deprecates the flag `unlock`, and makes the flag moot:
unlock via geth is no longer possible.
This PR moves chain config related code (config file processing, fork
logic, network defaults) from `beacon/types` and `beacon/blsync` into
`beacon/params` while the command line flag logic of the chain config is
moved into `cmd/utils`, thereby removing the cli dependencies from
package `beacon` and its sub-packages.
When `evm statetest --bench` is specified, benchmark the execution
similarly to `evm run`.
Also adds the ability to filter tests by name, index and fork.
---------
Co-authored-by: Martin Holst Swende <martin@swende.se>
This PR is a first step towards removing account management from geth,
and contains a lot of the user-facing changes.
With this PR, the `personal` namespace disappears. **Note**: `personal`
namespace has been deprecated for quite some time (since
https://github.com/ethereum/go-ethereum/pull/26390 1 year and 8 months
ago), and users who have wanted to use it has been forced to used the
flag `--rpc.enabledeprecatedpersonal`. So I think it's fairly
non-controversial to drop it at this point.
Specifically, this means:
- Account/wallet listing
-`personal.getListAccounts`
-`personal.listAccounts`
-`personal.getListWallets`
-`personal.listWallets`
- Lock/unlock
-`personal.lockAccount`
-`personal.openWallet`
-`personal.unlockAccount`
- Sign ops
-`personal.sign`
-`personal.sendTransaction`
-`personal.signTransaction`
- Imports / inits
-`personal.deriveAccount`
-`personal.importRawKey`
-`personal.initializeWallet`
-`personal.newAccount`
-`personal.unpair`
- Other:
-`personal.ecRecover`
The underlying keystores and account managent code is still in place,
which means that `geth --dev` still works as expected, so that e.g. the
example below still works:
```
> eth.sendTransaction({data:"0x6060", value: 1, from:eth.accounts[0]})
```
Also, `ethkey` and `clef` are untouched.
With the removal of `personal`, as far as I know we have no more API
methods which contain credentials, and if we want to implement
logging-capabilities of RPC ingress payload, it would be possible after
this.
---------
Co-authored-by: Felix Lange <fjl@twurst.com>
Currently we have a custom TextMarshalerFlag. It's a nice idea, allowing
anything implementing text marshaller to be used as a flag. That said,
we only ever used it in one place because it's not that obvious how to
use and it needs some boilerplate on the type itself too, apart of the
heavy boilerplate got the custom flag.
All in all there's no *need* to drop this feature just now, but while
porting the cmds over to cli @v3, all other custom flags worker
perfectly, whereas this one started crashing deep inside the cli
package. The flag handling in v3 got rebuild on generics and there are a
number of new methods needed; and my guess is that maybe one of them
doesn't work like this flag currently is designed too.
We could definitely try and redesign this flag for cli v3... but all
that effort and boilerplate just to use it for 1 flag in 1 location,
seems not worth it. So for now I'm suggesting removing it and maybe
reconsider a similar feature in cli v3 with however it will work.
rebased https://github.com/ethereum/go-ethereum/pull/29766 . The
downstream branch appears to have been deleted and I don't have perms to
push to that fork.
`TerminalTotalDifficultyPassed` is removed. `TerminalTotalDifficulty`
must now be non-nil, and it is expected that networks are already
merged: we can only import PoW/Clique chains, not produce blocks on
them.
---------
Co-authored-by: stevemilk <wangpeculiar@gmail.com>
Breaking changes:
- The ChainConfig was exposed to tracers via VMContext passed in
`OnTxStart`. This is unnecessary specially looking through the lens of
live tracers as chain config remains the same throughout the lifetime of
the program. It was there so that native API-invoked tracers could
access it. So instead we moved it to the constructor of API tracers.
Non-breaking:
- Change the default config of the tracers to be `{}` instead of nil.
This way an extra nil check can be avoided.
Refactoring:
- Rename `supply` struct to `supplyTracer`.
- Un-export some hook definitions.
Changelog: https://golangci-lint.run/product/changelog/#1610
Removes `exportloopref` (no longer needed), replaces it with
`copyloopvar` which is basically the opposite.
Also adds:
- `durationcheck`
- `gocheckcompilerdirectives`
- `reassign`
- `mirror`
- `tenv`
---------
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
This fixes a few issues missed in #29052:
* `requests` must be hex encoded, so added a helper to marshal.
* The statedb was committed too early and so the result of the system
calls was lost.
* For devnet-4 we need to pull off the type byte prefix from the request
data.
This is a redo of #29052 based on newer specs. Here we implement EIPs
scheduled for the Prague fork:
- EIP-7002: Execution layer triggerable withdrawals
- EIP-7251: Increase the MAX_EFFECTIVE_BALANCE
Co-authored-by: lightclient <lightclient@protonmail.com>
This implements recent changes to EIP-7685, EIP-6110, and
execution-apis.
---------
Co-authored-by: lightclient <lightclient@protonmail.com>
Co-authored-by: Shude Li <islishude@gmail.com>
The bulk of this PR is authored by @lightclient , in the original
EOF-work. More recently, the code has been picked up and reworked for the new EOF
specification, by @MariusVanDerWijden , in https://github.com/ethereum/go-ethereum/pull/29518, and also @shemnon has contributed with fixes.
This PR is an attempt to start eating the elephant one small bite at a
time, by selecting only the eof-validation as a standalone piece which
can be merged without interfering too much in the core stuff.
In this PR:
- [x] Validation of eof containers, lifted from #29518, along with
test-vectors from consensus-tests and fuzzing, to ensure that the move
did not lose any functionality.
- [x] Definition of eof opcodes, which is a prerequisite for validation
- [x] Addition of `undefined` to a jumptable entry item. I'm not
super-happy with this, but for the moment it seems the least invasive
way to do it. A better way might be to go back and allowing nil-items or
nil execute-functions to denote "undefined".
- [x] benchmarks of eof validation speed
---------
Co-authored-by: lightclient <lightclient@protonmail.com>
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Danno Ferrin <danno.ferrin@shemnon.com>
This PR integrates witness-enabled block production, witness-creating
payload execution and stateless cross-validation into the `engine` API.
The purpose of the PR is to enable the following use-cases (for API
details, please see next section):
- Cross validating locally created blocks:
- Call `forkchoiceUpdatedWithWitness` instead of `forkchoiceUpdated` to
trigger witness creation too.
- Call `getPayload` as before to retrieve the new block and also the
above created witness.
- Call `executeStatelessPayload` against another client to
cross-validate the block.
- Cross validating locally processed blocks:
- Call `newPayloadWithWitness` instead of `newPayload` to trigger
witness creation too.
- Call `executeStatelessPayload` against another client to
cross-validate the block.
- Block production for stateless clients (local or MEV builders):
- Call `forkchoiceUpdatedWithWitness` instead of `forkchoiceUpdated` to
trigger witness creation too.
- Call `getPayload` as before to retrieve the new block and also the
above created witness.
- Propagate witnesses across the consensus libp2p network for stateless
Ethereum.
- Stateless validator validation:
- Call `executeStatelessPayload` with the propagated witness to
statelessly validate the block.
*Note, the various `WithWitness` methods could also *just be* an
additional boolean flag on the base methods, but this PR wanted to keep
the methods separate until a final consensus is reached on how to
integrate in production.*
---
The following `engine` API types are introduced:
```go
// StatelessPayloadStatusV1 is the result of a stateless payload execution.
type StatelessPayloadStatusV1 struct {
Status string `json:"status"`
StateRoot common.Hash `json:"stateRoot"`
ReceiptsRoot common.Hash `json:"receiptsRoot"`
ValidationError *string `json:"validationError"`
}
```
- Add `forkchoiceUpdatedWithWitnessV1,2,3` with same params and returns
as `forkchoiceUpdatedV1,2,3`, but triggering a stateless witness
building if block production is requested.
- Extend `getPayloadV2,3` to return `executionPayloadEnvelope` with an
additional `witness` field of type `bytes` iff created via
`forkchoiceUpdatedWithWitnessV2,3`.
- Add `newPayloadWithWitnessV1,2,3,4` with same params and returns as
`newPayloadV1,2,3,4`, but triggering a stateless witness creation during
payload execution to allow cross validating it.
- Extend `payloadStatusV1` with a `witness` field of type `bytes` if
returned by `newPayloadWithWitnessV1,2,3,4`.
- Add `executeStatelessPayloadV1,2,3,4` with same base params as
`newPayloadV1,2,3,4` and one more additional param (`witness`) of type
`bytes`. The method returns `statelessPayloadStatusV1`, which mirrors
`payloadStatusV1` but replaces `latestValidHash` with `stateRoot` and
`receiptRoot`.
This PR fixes an issue with blob transaction propagation due to the blob
transation txpool rejecting transactions with gapped nonces. The
specific changes are:
- fetch transactions from a peer in the order they were announced to
minimize nonce-gaps (which cause blob txs to be rejected
- don't wait on fetching blob transactions after announcement is
received, since they are not broadcast
Testing:
- unit tests updated to reflect that fetch order should always match tx
announcement order
- unit test added to confirm blob transactions are scheduled immediately
for fetching
- running the PR on an eth mainnet full node without incident so far
---------
Signed-off-by: Roberto Bayardo <bayardo@alum.mit.edu>
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This pull request introduces a state.Reader interface for state
accessing.
The interface could be implemented in various ways. It can be pure trie
only reader, or the combination of trie and state snapshot. What's more,
this interface allows us to have more flexibility in the future, e.g.
the
archive reader (for accessing archive state).
Additionally, this pull request removes the following metrics
- `chain/snapshot/account/reads`
- `chain/snapshot/storage/reads`
This PR changes how sidechains are handled.
Before the merge, it was possible to import a chain with lower td and not set it as canonical. After the merge, we expect every chain that we get via InsertChain to be canonical. Non-canonical blocks can still be inserted
with InsertBlockWIthoutSetHead.
If during the InsertChain, the existing chain is not canonical anymore, we mark it as a sidechain and send the SideChainEvents normally.
This PR adds the `dns:read` and `dns:edit` permissions to the required
set of permissions checked before deploying an ENR tree to Cloudflare.
These permissions are necessary for a successful publish.
**Background**:
The current logic for `devp2p dns to-cloudflare` checks for `zone:edit`
and `zone:read` permissions. However, when running the command with only
these two permissions, the following error occurs:
```
wrong permissions on zone REMOVED-ZONE: map[#zone:edit:false #zone:read:true]
```
Adding `zone:read` and `zone:edit` to the API token led to a different
error:
```
INFO [08-19|14:06:16.782] Retrieving existing TXT records on pos-nodes.hardfork.dev
Authentication error (10000)
```
This suggested that additional permissions were required. I added
`dns:read`, but encountered another error:
```
INFO [08-19|14:11:42.342] Retrieving existing TXT records on pos-nodes.hardfork.dev
INFO [08-19|14:11:42.851] Updating DNS entries
failed to publish REMOVED.pos-nodes.hardfork.dev: Authentication error (10000)
```
Finally, after adding both `dns:read` and `dns:edit` permissions, the
command executed successfully with the following output:
```
INFO [08-19|14:13:07.677] Checking Permissions on zone REMOVED-ZONE
INFO [08-19|14:13:08.014] Retrieving existing TXT records on pos-nodes.hardfork.dev
INFO [08-19|14:13:08.440] Updating DNS entries
INFO [08-19|14:13:08.440] "Updating pos-nodes.hardfork.dev from \"enrtree-root:v1 e=FSED3EDKEKRDDFMCLP746QY6CY l=FDXN3SN67NA5DKA4J2GOK7BVQI seq=1 sig=Glja2c9RviRqOpaaHR0MnHsQwU76nJXadJwFeiXpp8MRTVIhvL0LIireT0yE3ETZArGEmY5Ywz3FVHZ3LR5JTAE\" to \"enrtree-root:v1 e=AB66M4ULYD5OYN4XFFCPVZRLUM l=FDXN3SN67NA5DKA4J2GOK7BVQI seq=1 sig=H8cqDzu0FAzBplK4g3yudhSaNtszIebc2aj4oDm5a5ZE5PAg-xpCnQgVE_53CsgsqQpalD9byafx_FrUT61sagA\""
INFO [08-19|14:13:16.932] Updated DNS entries new=32 updated=1 untouched=100
INFO [08-19|14:13:16.932] Deleting stale DNS entries
INFO [08-19|14:13:24.663] Deleted stale DNS entries count=31
```
With this PR, the required permissions for deploying an ENR tree to
Cloudflare now include `zone:read`, `zone:edit`, `dns:read`, and
`dns:edit`. The initial check now includes all of the necessary
permissions and indicates in the error message which permissions are
missing:
```
INFO [08-19|14:17:20.339] Checking Permissions on zone REMOVED-ZONE
wrong permissions on zone REMOVED-ZONE: map[#dns_records:edit:false #dns_records:read:false #zone:edit:false #zone:read:true]
```
Fixes#30254
It seems like the removed CreateAccount call is very old and not needed anymore.
After removing it, setting a sender that does not exist in the state doesn't seem to cause
an issue.
Currently, we have 3 flags to configure blob pool. However, we don't
read these flags and set the blob pool configuration in eth config
accordingly. This commit adds a function to check if these flags are
provided and set blob pool configuration based on them.
Martin HS
Felix Lange
lightclient
rjl493456442
Marius van der Wijden
jwasinger
Felföldi Zsolt
Péter Szilágyi
Sina M
Ng Wei Han
zhiqiangxu
Roberto Bayardo
stevemilk
John Hilliard
yukionfire
minh-bq
Jeremy Schlatter