* accounts/abi/bind: support for multi-dim arrays
Also:
- reduce usage of regexes a bit.
- fix minor Java syntax problems
Fixes#15648
* accounts/abi/bind: Add some more documentation
* accounts/abi/bind: Improve code readability
* accounts/abi: bugfix for unpacking nested arrays
The code previously assumed the arrays/slices were always 1 level
deep. While the packing supports nested arrays (!!!).
The current code for unpacking doesn't return the "consumed" length, so
this fix had to work around that by calculating it (i.e. packing and
getting resulting length) after the unpacking of the array element.
It's far from ideal, but unpacking behaviour is fixed now.
* accounts/abi: Fix unpacking of nested arrays
Removed the temporary workaround of packing to calculate size, which was
incorrect for slice-like types anyway.
Full size of nested arrays is used now.
* accounts/abi: deeply nested array unpack test
Test unpacking of an array nested more than one level.
* accounts/abi: Add deeply nested array pack test
Same as the deep nested array unpack test, but the other way around.
* accounts/abi/bind: deeply nested arrays bind test
Test the usage of bindings that were generated
for methods with multi-dimensional (and not
just a single extra dimension, like foo[2][3])
array arguments and returns.
edit: trigger rebuild, CI failed to fetch linter module.
* accounts/abi/bind: improve array binding
wrapArray uses a regex now, and arrayBindingJava is improved.
* accounts/abi: Improve naming of element size func
The full step size for unpacking an array
is now retrieved with "getFullElemSize".
* accounts/abi: support nested nested array args
Previously, the code only considered the outer-size of the array,
ignoring the size of the contents. This was fine for most types,
but nested arrays are packed directly into it, and count towards
the total size. This resulted in arguments following a nested
array to replicate some of the binary contents of the array.
The fix: for arrays, calculate their complete contents size:
count the arg.Type.Elem.Size when Elem is an Array, and
repeat when their child is an array too, etc.
The count is the number of 32 byte elements, similar to how it
previously counted, but nested.
* accounts/abi: Test deep nested arr multi-arguments
Arguments with a deeply nested array should not cause the next arguments
to be read from the wrong position.
* core/types, core/vm, eth, tests: regenerate gencodec files
* Makefile: update devtools target
Install protoc-gen-go and print reminders about npm, solc and protoc.
Also switch to github.com/kevinburke/go-bindata because it's more
maintained.
* contracts/ens: update contracts and regenerate with solidity v0.4.19
The newer upstream version of the FIFSRegistrar contract doesn't set the
resolver anymore. The resolver is now deployed separately.
* contracts/release: regenerate with solidity v0.4.19
* contracts/chequebook: fix fallback and regenerate with solidity v0.4.19
The contract didn't have a fallback function, payments would be rejected
when compiled with newer solidity. References to 'mortal' and 'owned'
use the local file system so we can compile without network access.
* p2p/discv5: regenerate with recent stringer
* cmd/faucet: regenerate
* dashboard: regenerate
* eth/tracers: regenerate
* internal/jsre/deps: regenerate
* dashboard: avoid sed -i because it's not portable
* accounts/usbwallet/internal/trezor: fix go generate warnings
+ The event slice unpacker doesn't correctly extract element from the
slice. The indexed arguments are not ignored as they should be
(the data offset should not include the indexed arguments).
+ The `Elem()` call in the slice unpack doesn't work.
The Slice related tests fails because of that.
+ the check in the loop are suboptimal and have been extracted
out of the loop.
+ extracted common code from event and method tupleUnpack
This change inlines the logic of bytesAreProper at its sole
callsite, ABI.Unpack, and applies the multiple-of-32 test only in
the case of unpacking methods. Event data is not required to be a
multiple of 32 bytes long.
* cmd, consensus, core, miner: instatx clique for --dev
* cmd, consensus, clique: support configurable --dev block times
* cmd, core: allow --dev to use persistent storage too
The accountCache contains a file cache, and remembers from
scan to scan what files were present earlier. Thus, whenever
there's a change, the scan phase only bothers processing new
and removed files.
* core: remove redundant storage of transactions and receipts
* core, eth, internal: new transaction schema usage polishes
* eth: implement upgrade mechanism for db deduplication
* core, eth: drop old sequential key db upgrader
* eth: close last iterator on successful db upgrage
* core: prefix the lookup entries to make their purpose clearer
Tests are now included as a submodule. This should make updating easier
and removes ~60MB of JSON data from the working copy.
State tests are replaced by General State Tests, which run the same test
with multiple fork configurations.
With the new test runner, consensus tests are run as subtests by walking
json files. Many hex issues have been fixed upstream since the last
update and most custom parsing code is replaced by existing JSON hex
types. Tests can now be marked as 'expected failures', ensuring that
fixes for those tests will trigger an update to test configuration. The
new test runner also supports parallel execution and the -short flag.
With this commit, core/state's access to the underlying key/value database is
mediated through an interface. Database errors are tracked in StateDB and
returned by CommitTo or the new Error method.
Motivation for this change: We can remove the light client's duplicated copy of
core/state. The light client now supports node iteration, so tracing and storage
enumeration can work with the light client (not implemented in this commit).
* accounts/abi: reorganizing package and some notes and a quick correction of name.
Signed-off-by: RJ Catalano <rj@monax.io>
get rid of some imports
Signed-off-by: RJ Catalano <rj@monax.io>
* accounts/abi: move file names
Signed-off-by: RJ Catalano <rj@monax.io>
* accounts/abi: fix boolean decode function
Signed-off-by: RJ Catalano <rj@monax.io>
* accounts/abi: fix for the array set and for creating a bool
Signed-off-by: RJ Catalano <rj@monax.io>
* accounts/abi: be very very very correct
Signed-off-by: RJ Catalano <rj@monax.io>
* accounts/abi: fix up error message and variable names
Signed-off-by: RJ Catalano <rj@monax.io>
* accounts/abi: take out unnecessary argument in pack method
Signed-off-by: RJ Catalano <rj@monax.io>
* accounts/abi: add bool unpack test and add a panic to readBool function
Signed-off-by: RJ Catalano <rj@monax.io>
* accounts/abi: fix panic message
Signed-off-by: RJ Catalano <rj@monax.io>
* accounts/abi: change from panic to basic error
Signed-off-by: RJ Catalano <rj@monax.io>
* accounts/abi: fix nil to false
Signed-off-by: RJ Catalano <rj@monax.io>
* accounts/abi: fill out type regex tests and fill with the correct type for integers
Signed-off-by: RJ Catalano <rj@monax.io>
* accounts/abi: move packNumbers into pack.go.
Signed-off-by: RJ Catalano <rj@monax.io>
* accounts/abi: separation of the testing suite into appropriately named files.
Signed-off-by: RJ Catalano <rj@monax.io>
* account/abi: change to hex string tests.
Signed-off-by: RJ Catalano <rj@monax.io>
* account/abi: fix up rest of tests to hex
Signed-off-by: RJ Catalano <rj@monax.io>
* accounts/abi: declare bool at the package level
Signed-off-by: RJ Catalano <rj@monax.io>
* accounts/abi: use errors package in the error file.
Signed-off-by: RJ Catalano <rj@monax.io>
* accounts/abi: fix ugly hack and fix error type declaration.
Signed-off-by: RJ Catalano <rj@monax.io>
This commit adds pluggable consensus engines to go-ethereum. In short, it
introduces a generic consensus interface, and refactors the entire codebase to
use this interface.
This commit solves several issues concerning the genesis block:
* Genesis/ChainConfig loading was handled by cmd/geth code. This left
library users in the cold. They could specify a JSON-encoded
string and overwrite the config, but didn't get any of the additional
checks performed by geth.
* Decoding and writing of genesis JSON was conflated in
WriteGenesisBlock. This made it a lot harder to embed the genesis
block into the forthcoming config file loader. This commit changes
things so there is a single Genesis type that represents genesis
blocks. All uses of Write*Genesis* are changed to use the new type
instead.
* If the chain config supplied by the user was incompatible with the
current chain (i.e. the chain had already advanced beyond a scheduled
fork), it got overwritten. This is not an issue in practice because
previous forks have always had the highest total difficulty. It might
matter in the future though. The new code reverts the local chain to
the point of the fork when upgrading configuration.
The change to genesis block data removes compression library
dependencies from package core.
There is no need to depend on the old context package now that the
minimum Go version is 1.7. The move to "context" eliminates our weird
vendoring setup. Some vendored code still uses golang.org/x/net/context
and it is now vendored in the normal way.
This change triggered new vet checks around context.WithTimeout which
didn't fire with golang.org/x/net/context.
* accounts, cmd, eth, ethdb: port logs over to new system
* ethdb: drop concept of cache distribution between dbs
* eth: fix some log nitpicks to make them nicer
* common/math: optimize PaddedBigBytes, use it more
name old time/op new time/op delta
PaddedBigBytes-8 71.1ns ± 5% 46.1ns ± 1% -35.15% (p=0.000 n=20+19)
name old alloc/op new alloc/op delta
PaddedBigBytes-8 48.0B ± 0% 32.0B ± 0% -33.33% (p=0.000 n=20+20)
* all: unify big.Int zero checks
Various checks were in use. This commit replaces them all with Int.Sign,
which is cheaper and less code.
eg templates:
func before(x *big.Int) bool { return x.BitLen() == 0 }
func after(x *big.Int) bool { return x.Sign() == 0 }
func before(x *big.Int) bool { return x.BitLen() > 0 }
func after(x *big.Int) bool { return x.Sign() != 0 }
func before(x *big.Int) int { return x.Cmp(common.Big0) }
func after(x *big.Int) int { return x.Sign() }
* common/math, crypto/secp256k1: make ReadBits public in package math
* common: remove CurrencyToString
Move denomination values to params instead.
* common: delete dead code
* common: move big integer operations to common/math
This commit consolidates all big integer operations into common/math and
adds tests and documentation.
There should be no change in semantics for BigPow, BigMin, BigMax, S256,
U256, Exp and their behaviour is now locked in by tests.
The BigD, BytesToBig and Bytes2Big functions don't provide additional
value, all uses are replaced by new(big.Int).SetBytes().
BigToBytes is now called PaddedBigBytes, its minimum output size
parameter is now specified as the number of bytes instead of bits. The
single use of this function is in the EVM's MSTORE instruction.
Big and String2Big are replaced by ParseBig, which is slightly stricter.
It previously accepted leading zeros for hexadecimal inputs but treated
decimal inputs as octal if a leading zero digit was present.
ParseUint64 is used in places where String2Big was used to decode a
uint64.
The new functions MustParseBig and MustParseUint64 are now used in many
places where parsing errors were previously ignored.
* common: delete unused big integer variables
* accounts/abi: replace uses of BytesToBig with use of encoding/binary
* common: remove BytesToBig
* common: remove Bytes2Big
* common: remove BigTrue
* cmd/utils: add BigFlag and use it for error-checked integer flags
While here, remove environment variable processing for DirectoryFlag
because we don't use it.
* core: add missing error checks in genesis block parser
* common: remove String2Big
* cmd/evm: use utils.BigFlag
* common/math: check for 256 bit overflow in ParseBig
This is supposed to prevent silent overflow/truncation of values in the
genesis block JSON. Without this check, a genesis block that set a
balance larger than 256 bits would lead to weird behaviour in the VM.
* cmd/utils: fixup import
- Handle a data race where a Ledger drops between list and open
- Prolong Ledger tx confirmation window to 30 days from 1 minute
- Simplify Ledger chainid-signature calculation and validation
- Simplify Ledger USB APDU request chunking algorithm
- Silence keystore account cache notifications for manual actions
- Only enable self derivations if wallet open succeeds
Gas estimation currently mostly works, but can underestimate for more funky
refunds. This is because various ops (e.g. CALL) need more gas to run than they
actually consume (e.g. 2300 stipend that is refunded if not used). With more
intricate contract interplays, it becomes almost impossible to return a proper
value to the user.
This commit swaps out the simplistic gas estimation to a binary search approach,
honing in on the correct gas use. This does mean that gas estimation needs to
rerun the transaction log(max-price) times to measure whether it fails or not,
but it's a price paid by the transaction issuer, and it should be worth it to
support proper estimates.
On solidity contract I have "uint32 []" type, when abigen creates Go
bindings - they are also "[]uint32" type on Go side. Even though it
looks like it should work - the actual type of the data coming from
the chain is of type " []*big.Int".
When executing contract function from Go side - getting unmarshal error:
abi: cannot unmarshal []*big.Int in to []uint32
The fix is to create array with the correct type
This fixed the issue reported in: https://github.com/ethereum/go-ethereum/issues/2802
This significantly reduces the dependency closure of ethclient, which no
longer depends on core/vm as of this change.
All uses of vm.Logs are replaced by []*types.Log. NewLog is gone too,
the constructor simply returned a literal.
The run loop, which previously contained custom opcode executes have been
removed and has been simplified to a few checks.
Each operation consists of 4 elements: execution function, gas cost function,
stack validation function and memory size function. The execution function
implements the operation's runtime behaviour, the gas cost function implements
the operation gas costs function and greatly depends on the memory and stack,
the stack validation function validates the stack and makes sure that enough
items can be popped off and pushed on and the memory size function calculates
the memory required for the operation and returns it.
This commit also allows the EVM to go unmetered. This is helpful for offline
operations such as contract calls.
To address increasing complexity in code that handles signatures, this PR
discards all notion of "different" signature types at the library level. Both
the crypto and accounts package is reduced to only be able to produce plain
canonical secp256k1 signatures. This makes the crpyto APIs much cleaner,
simpler and harder to abuse.
Environment is now a struct (not an interface). This
reduces a lot of tech-debt throughout the codebase where a virtual
machine environment had to be implemented in order to test or run it.
The new environment is suitable to be used en the json tests, core
consensus and light client.
This commit implements EIP158 part 1, 2, 3 & 4
1. If an account is empty it's no longer written to the trie. An empty
account is defined as (balance=0, nonce=0, storage=0, code=0).
2. Delete an empty account if it's touched
3. An empty account is redefined as either non-existent or empty.
4. Zero value calls and zero value suicides no longer consume the 25k
reation costs.
params: moved core/config to params
Signed-off-by: Jeffrey Wilcke <jeffrey@ethereum.org>
This commit includes several API changes:
- The behavior of eth_sign is changed. It now accepts an arbitrary
message, prepends the well-known string
\x19Ethereum Signed Message:\n<length of message>
hashes the result using keccak256 and calculates the signature of
the hash. This breaks backwards compatability!
- personal_sign(hash, address [, password]) is added. It has the same
semantics as eth_sign but also accepts a password. The private key
used to sign the hash is temporarily unlocked in the scope of the
request.
- personal_recover(message, signature) is added and returns the
address for the account that created a signature.
This commit replaces the deep-copy based state revert mechanism with a
linear complexity journal. This commit also hides several internal
StateDB methods to limit the number of ways in which calling code can
use the journal incorrectly.
As usual consultation and bug fixes to the initial implementation were
provided by @karalabe, @obscuren and @Arachnid. Thank you!
The need for these functions comes up in code that actually deploys and
uses contracts. As of this commit, they can be used with both
SimulatedBackend and ethclient.
SimulatedBackend gains some additional methods in the process and is now
safe for concurrent use.
In this commit, contract bindings and their backend start using the
Ethereum Go API interfaces offered by ethclient. This makes ethclient a
suitable replacement for the old remote backend and gets us one step
closer to the final stable Go API that is planned for go-ethereum 1.5.
The changes in detail:
* Pending state is optional for read only contract bindings.
BoundContract attempts to discover the Pending* methods via an
interface assertion. There are a couple of advantages to this:
ContractCaller is just two methods and can be implemented on top of
pretty much anything that provides Ethereum data. Since the backend
interfaces are now disjoint, ContractBackend can simply be declared as
a union of the reader and writer side.
* Caching of HasCode is removed. The caching could go wrong in case of
chain reorganisations and removing it simplifies the code a lot.
We'll figure out a performant way of providing ErrNoCode before the
1.5 release.
* BoundContract now ensures that the backend receives a non-nil context
with every call.
The remote backend is superseded by ethclient.
The nil backend's stated purpose was to enable testing of
accounts/abi/bind. None of its methods actually worked. A much simpler
way to get a crashing backend is to simply pass nil as the backend. With
a one-line change to the generator (removing two explicit interface
assertions), passing nil actually works.
Removing these backends means that less changes are required later.
The account manager was previously created by packge cmd/utils as part
of flag processing and then passed down into eth.Ethereum through its
config struct. Since we are starting to create nodes which do not have
eth.Ethereum as a registered service, the code was rearranged to
register the account manager as its own service. Making it a service is
ugly though and it doesn't really fix the root cause: creating nodes
without eth.Ethereum requires duplicating lots of code.
This commit splits utils.MakeSystemNode into three functions, making
creation of other node/service configurations easier. It also moves the
account manager into Node so it can be used by those configurations
without requiring package eth.
When converting a negative number e.g., -2, the resulting ABI encoding
should look as follows:
fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe.
However, since the check of the type is for an uint instead of an
int, it results in the following ABI encoding:
0101010101010101010101010101010101010101010101010101010101010102. The
Ethereum ABI
(https://github.com/ethereum/wiki/wiki/Ethereum-Contract-ABI) says,
that signed integers are stored in two's complement which should be
of the form ffffff.... and not 01010101..... for e.g. -1. Thus, I
removed the type check in numbers.go as well as the function S256
as I don't think they are correct. Or maybe I'm missing something?
Previously it was assumed that wheneven type `[]interface{}` was given
that the interface was empty. The abigen rightfully assumed that
interface slices which already have pre-allocated variable sets to be
assigned.
This PR fixes that by checking that the given `[]interface{}` is larger
than zero and assigns each value using the generic `set` function (this
function has also been moved to abi/reflect.go) and checks whether the
assignment was possible.
The generic assignment function `set` now also deals with pointers
(useful for interface slice mentioned above) by dereferencing the
pointer until it finds a setable type.
Refactored the abi package parsing and type handling. Relying mostly on
package reflect as opposed to most of our own type reflection. Our own
type reflection is still used however for cases such as Bytes and
FixedBytes (abi: bytes•).
This also inclused several fixes for slice handling of arbitrary and
fixed size for all supported types.
This also further removes implicit type casting such as assigning,
for example `[2]T{} = []T{1}` will fail, however `[2]T{} == []T{1, 2}`
(notice assigning *slice* to fixed size *array*). Assigning arrays to
slices will always succeed if they are of the same element type.
Incidentally also fixes#2379
In order to avoid disk thrashing for Accounts and HasAccount,
address->key file mappings are now cached in memory. This makes it no
longer necessary to keep the key address in the file name. The address
of each key is derived from file content instead.
There are minor user-visible changes:
- "geth account list" now reports key file paths alongside the address.
- If multiple keys are present for an address, unlocking by address is
not possible. Users are directed to remove the duplicate files
instead. Unlocking by index is still possible.
- Key files are overwritten written in place when updating the password.
- Manager.Accounts no longer returns an error.
- Manager methods take Account instead of common.Address.
- All uses of Account with unkeyed fields are converted.
The account management API was originally implemented as a thin layer
around crypto.KeyStore, on the grounds that several kinds of key stores
would be implemented later on. It turns out that this won't happen so
KeyStore is a superflous abstraction.
In this commit crypto.KeyStore and everything related to it moves to
package accounts and is unexported.
The chain maker and the simulated backend now run with a homestead phase
beginning at block 0 (i.e. there's no frontier).
This commit also fixes up #2388
Added chain configuration options and write out during genesis database
insertion. If no "config" was found, nothing is written to the database.
Configurations are written on a per genesis base. This means
that any chain (which is identified by it's genesis hash) can have their
own chain settings.
Fixed up `[]byte` slice support such that `function print(bytes input)`
accepts `[]byte` as input and treats it as 1 element rather than a slice
of multiple elements.
Added support for variable length input parameters like `bytes` and
`strings`.
Removed old unmarshalling of return types: `abi.Call(...).([]byte)`.
This is now replaced by a new syntax:
```
var a []byte
err := abi.Call(&a, ...)
```
It also addresses a few issues with Bytes and Strings and can also
handle both fixed and arbitrary sized byte slices, including strings.
* multiple passwords allowed in password file
* split on "\n", sideeffect: chop trailing slashes. fixes common mistake <(echo 'pass')
* remove accounts.Primary method
* do not fall back to primary account for mining
- cli: add passwordfile flag
- cli: change unlock flag only takes account
- cli: with unlock you are prompted for password or use passfile with password flag
- cli: unlockAccount used in normal client start (run) and accountExport
- cli: getPassword used in accountCreate and accountImport
- accounts: Manager.Import, Manager.Export
- crypto: SaveECDSA (to complement LoadECDSA) to save to file
- crypto: NewKeyFromECDSA added (used in accountImport and New = generated constructor)
There is no point to using time.Duration if the value is interpreted as
milliseconds. Callers should use the standard multiplication idiom to
choose the unit. In fact, the only caller outside of the tests already
does so.
Account is now always a non-pointer. This will be important once
the manager starts remembering accounts.
AccountManager is now always a pointer because it contains locks
and locks cannot be copied.
* Change account signing API to two sign functions;
Sign without passphrase - works if account is unlocked
Sign with passphrase - always works and unlocks the account
* Account stays unlocked for X ms and is then automatically locked
* Use crypto.Sign instead of directly calling secp256k1 lib
* Rename UserAccount to Account and Addr to Address (for consistency)
* Change AccountManager.Sign to take ptr to Account instead of
address byte array
* Simplify copying of Accounts in Accounts()
* PubkeyToAddress and GetEntropyCSPRNG now exported
* Add initial UserAccount and AccountManager structs
* Add NewAccount, Sign and Accounts functions
* Refactor key stores to use key address as main identifier
while keeping the UUID.
* Use key address as file/dir names instead of UUID
Ricardo Domingos
Martin Holst Swende
Mark
Javier Peletier
protolambda
Péter Szilágyi
Fynn
Felix Lange
mark.lin
Miguel Mota
Furkan KAMACI
gary rong
Robert Zaremba
Bob Glickstein
Dmitry Shulyak
Yondon Fu
Armin
Zach
Anton Evangelatov
ferhat elmas
RJ Catalano
Dave Appleton
nkbai
Miya Chen
Egon Elbre
Leo Shklovskii
FaceHo
Felix Lange
bas-vk
Nick Johnson
Diego Siqueira
Thomas Bocek
Jeffrey Wilcke
RJ
Gustav Simonsson
zsfelfoldi
Bas van Kervel
Jeffrey Wilcke
Ales Katona
Ricardo Catalinas Jiménez
zelig
Bas van Kervel
Maran