Commit Graph

451 Commits

Author SHA1 Message Date
Felix Lange 10181b57a9 core, eth/downloader: commit block data using batches (#15115)
* ethdb: add Putter interface and Has method

* ethdb: improve docs and add IdealBatchSize

* ethdb: remove memory batch lock

Batches are not safe for concurrent use.

* core: use ethdb.Putter for Write* functions

This covers the easy cases.

* core/state: simplify StateSync

* trie: optimize local node check

* ethdb: add ValueSize to Batch

* core: optimize HasHeader check

This avoids one random database read get the block number. For many uses
of HasHeader, the expectation is that it's actually there. Using Has
avoids a load + decode of the value.

* core: write fast sync block data in batches

Collect writes into batches up to the ideal size instead of issuing many
small, concurrent writes.

* eth/downloader: commit larger state batches

Collect nodes into a batch up to the ideal size instead of committing
whenever a node is received.

* core: optimize HasBlock check

This avoids a random database read to get the number.

* core: use numberCache in HasHeader

numberCache has higher capacity, increasing the odds of finding the
header without a database lookup.

* core: write imported block data using a batch

Restore batch writes of state and add blocks, tx entries, receipts to
the same batch. The change also simplifies the miner.

This commit also removes posting of logs when a forked block is imported.

* core: fix DB write error handling

* ethdb: use RLock for Has

* core: fix HasBlock comment
2017-09-09 19:03:07 +03:00
Péter Szilágyi b0ca1b67ce
eth: use maxpeers from p2p layer instead of extra config 2017-09-05 19:18:28 +03:00
Oli Bye ebf41d16a0 cmd/geth: fix --nousb typo (#15040) 2017-08-25 16:54:36 +03:00
Miya Chen bf1e263128 core, light: send chain events using event.Feed (#14865) 2017-08-18 12:58:36 +02:00
Péter Szilágyi 4a260dc1f2
cmd: add makecache cmd, use caches during import cmd 2017-08-03 18:39:34 +02:00
Péter Szilágyi 3d32690b54 cmd, core, eth: journal local transactions to disk (#14784)
* core: reduce txpool event loop goroutines and sync structs

* cmd, core, eth: journal local transactions to disk

* core: journal replacement pending transactions too

* core: separate transaction journal from pool
2017-07-28 15:09:39 +02:00
Péter Szilágyi 5e38f7a664
cmd, core: add --txpool.nolocals to disable local price exemptions 2017-07-05 17:06:05 +03:00
Bas van Kervel 7a11e86442 whisper: move flags from whisper package to utils 2017-06-21 10:49:14 +02:00
Bas van Kervel b58a501673 whisperv5: integrate whisper and add whisper RPC simulator 2017-06-15 11:53:15 +02:00
Péter Szilágyi 08959bbc70
cmd, core, eth: configurable txpool parameters 2017-05-29 11:29:46 +03:00
Bas van Kervel a346aedb90
cmd/geth: reorganise help section for new cli flag handling 2017-05-25 09:15:51 +02:00
Péter Szilágyi a2f23ca9b1 cmd, core, eth, miner: remove txpool gas price limits (#14442) 2017-05-16 21:07:27 +02:00
Péter Szilágyi 3b3989de6a
cmd, node: support different bootnodes, fix default light port 2017-05-10 17:51:52 +03:00
Péter Szilágyi ff2c966e7f Merge pull request #14418 from karalabe/rinkeby-flag
cmd, core, params: add --rinkeby flag for fast connectivity
2017-05-04 13:54:02 +03:00
Péter Szilágyi d51a9fd6b7
cmd, core, params: add --rinkeby flag for fast connectivity 2017-05-04 12:36:20 +03:00
Bas van Kervel 502a2bd69f
cmd/geth: reorganise console/attach commands/flags 2017-05-03 14:33:07 +02:00
Péter Szilágyi 181a3309df
cmd/geth, cmd/utils: init/removedb on light/full dbs simultaneously 2017-05-03 13:35:47 +03:00
Bas van Kervel f6c0f76cc5
cmd/geth: reorganise account/wallet command/flags 2017-04-28 14:01:54 +02:00
gluk256 f9be9a2302 whisper: switching to v5 + minor refactoring (#14387) 2017-04-28 11:57:15 +02:00
Péter Szilágyi e61035c5a3
cmd, eth, les, mobile: make networkid uint64 everywhere 2017-04-25 14:53:50 +03:00
Péter Szilágyi 6430e672c9
cmd, node: add --nosub and node.Config.NoUSB to disable hw wallets 2017-04-20 14:01:51 +03:00
bas-vk 5e29f4be93 cmd/utils, node: remove unused solc references and improve RPC config (#14324)
Currently http cors and websocket origins are a comma separated string in the
config object. These are replaced with string arrays that are more expressive in
case of a config file.
2017-04-12 23:04:14 +02:00
Felix Lange 30d706c35e cmd/geth: add --config file flag (#13875)
* p2p/discover, p2p/discv5: add marshaling methods to Node

* p2p/netutil: make Netlist decodable from TOML

* common/math: encode nil HexOrDecimal256 as 0x0

* cmd/geth: add --config file flag

* cmd/geth: add missing license header

* eth: prettify Config again, fix tests

* eth: use gasprice.Config instead of duplicating its fields

* eth/gasprice: hide nil default from dumpconfig output

* cmd/geth: hide genesis block in dumpconfig output

* node: make tests compile

* console: fix tests

* cmd/geth: make TOML keys look exactly like Go struct fields

* p2p: use discovery by default

This makes the zero Config slightly more useful. It also fixes package
node tests because Node detects reuse of the datadir through the
NodeDatabase.

* cmd/geth: make ethstats URL settable through config file

* cmd/faucet: fix configuration

* cmd/geth: dedup attach tests

* eth: add comment for DefaultConfig

* eth: pass downloader.SyncMode in Config

This removes the FastSync, LightSync flags in favour of a more
general SyncMode flag.

* cmd/utils: remove jitvm flags

* cmd/utils: make mutually exclusive flag error prettier

It now reads:

   Fatal: flags --dev, --testnet can't be used at the same time

* p2p: fix typo

* node: add DefaultConfig, use it for geth

* mobile: add missing NoDiscovery option

* cmd/utils: drop MakeNode

This exposed a couple of places that needed to be updated to use
node.DefaultConfig.

* node: fix typo

* eth: make fast sync the default mode

* cmd/utils: remove IPCApiFlag (unused)

* node: remove default IPC path

Set it in the frontends instead.

* cmd/geth: add --syncmode

* cmd/utils: make --ipcdisable and --ipcpath mutually exclusive

* cmd/utils: don't enable WS, HTTP when setting addr

* cmd/utils: fix --identity
2017-04-12 17:27:23 +03:00
Felföldi Zsolt 9aca9e6deb cmd, les, eth, eth/gasprice: using new gas price oracle (#13853)
* cmd, les, eth, eth/gasprice: using new gas price oracle

* eth/gasprice: renamed source file

* eth/gasprice: added security checks for gpo params

* eth/gasprice: fixed naming issues

* eth/gasprice: max limit, maxEmpty
2017-04-06 17:20:42 +03:00
Péter Szilágyi 09777952ee core, consensus: pluggable consensus engines (#3817)
This commit adds pluggable consensus engines to go-ethereum. In short, it
introduces a generic consensus interface, and refactors the entire codebase to
use this interface.
2017-04-05 00:16:29 +02:00
Felix Lange 37dd9086ec core: refactor genesis handling
This commit solves several issues concerning the genesis block:

* Genesis/ChainConfig loading was handled by cmd/geth code. This left
  library users in the cold. They could specify a JSON-encoded
  string and overwrite the config, but didn't get any of the additional
  checks performed by geth.
* Decoding and writing of genesis JSON was conflated in
  WriteGenesisBlock. This made it a lot harder to embed the genesis
  block into the forthcoming config file loader. This commit changes
  things so there is a single Genesis type that represents genesis
  blocks. All uses of Write*Genesis* are changed to use the new type
  instead.
* If the chain config supplied by the user was incompatible with the
  current chain (i.e. the chain had already advanced beyond a scheduled
  fork), it got overwritten. This is not an issue in practice because
  previous forks have always had the highest total difficulty. It might
  matter in the future though. The new code reverts the local chain to
  the point of the fork when upgrading configuration.

The change to genesis block data removes compression library
dependencies from package core.
2017-03-23 15:58:43 +01:00
Péter Szilágyi b7d93500f1 all: finish integrating Go ethash, delete C++ vendor 2017-03-09 15:50:14 +01:00
Péter Szilágyi 023670f6ba cmd, eth, les, node, pow: disk caching and progress reports 2017-03-09 15:50:14 +01:00
Péter Szilágyi 567d41d936 all: swap out the C++ ethash to the pure Go one (mining todo) 2017-03-09 15:50:14 +01:00
Martin Holst Swende 544247c918 * cmd/geth: add --nocompaction flag + multiple import files
* main,import: Add --nocompaction flag + multiple import files

* geth/import: documentation

* import: Added more info to err message

* fix :P
2017-03-08 13:26:19 +02:00
Péter Szilágyi 9184249b39 Logger updates 3 (#3730)
* accounts, cmd, eth, ethdb: port logs over to new system

* ethdb: drop concept of cache distribution between dbs

* eth: fix some log nitpicks to make them nicer
2017-03-02 14:06:16 +01:00
Jeffrey Wilcke 7ff75ac2f2 cmd/utils, core, params: fork all teh things for dev mode (#3697) 2017-02-28 23:18:13 +01:00
Felix Lange 5c8fe28b72 common: move big integer math to common/math (#3699)
* common: remove CurrencyToString

Move denomination values to params instead.

* common: delete dead code

* common: move big integer operations to common/math

This commit consolidates all big integer operations into common/math and
adds tests and documentation.

There should be no change in semantics for BigPow, BigMin, BigMax, S256,
U256, Exp and their behaviour is now locked in by tests.

The BigD, BytesToBig and Bytes2Big functions don't provide additional
value, all uses are replaced by new(big.Int).SetBytes().

BigToBytes is now called PaddedBigBytes, its minimum output size
parameter is now specified as the number of bytes instead of bits. The
single use of this function is in the EVM's MSTORE instruction.

Big and String2Big are replaced by ParseBig, which is slightly stricter.
It previously accepted leading zeros for hexadecimal inputs but treated
decimal inputs as octal if a leading zero digit was present.

ParseUint64 is used in places where String2Big was used to decode a
uint64.

The new functions MustParseBig and MustParseUint64 are now used in many
places where parsing errors were previously ignored.

* common: delete unused big integer variables

* accounts/abi: replace uses of BytesToBig with use of encoding/binary

* common: remove BytesToBig

* common: remove Bytes2Big

* common: remove BigTrue

* cmd/utils: add BigFlag and use it for error-checked integer flags

While here, remove environment variable processing for DirectoryFlag
because we don't use it.

* core: add missing error checks in genesis block parser

* common: remove String2Big

* cmd/evm: use utils.BigFlag

* common/math: check for 256 bit overflow in ParseBig

This is supposed to prevent silent overflow/truncation of values in the
genesis block JSON. Without this check, a genesis block that set a
balance larger than 256 bits would lead to weird behaviour in the VM.

* cmd/utils: fixup import
2017-02-26 22:21:51 +01:00
Péter Szilágyi 1ca20a2697
cmd, whisper/mailserver: revert to utils.Fatalf 2017-02-23 12:16:46 +02:00
Péter Szilágyi 23a5d64fd0
accounts, cmd: port packages over to the new logging system 2017-02-23 12:16:46 +02:00
Péter Szilágyi d4fd06c3dc
all: blidly swap out glog to our log15, logs need rework 2017-02-23 12:16:44 +02:00
Péter Szilágyi fad5eb0a87
accounts, cmd, eth, internal, miner, node: wallets and HD APIs 2017-02-13 14:00:07 +02:00
Péter Szilágyi 833e4d1319
accounts, cmd, eth, internal, mobile, node: split account backends 2017-02-13 14:00:02 +02:00
Zsolt Felfoldi a390ca5f30 les, cmd/util: disable topic discovery with --nodiscover 2017-01-27 02:52:45 +01:00
Péter Szilágyi d63752ef4d Merge pull request #3579 from bas-vk/natspec
cmd,eth,les,internal: remove natspec support
2017-01-17 14:38:57 +02:00
Nick Johnson 17d92233d9 cmd/geth, core: add support for recording SHA3 preimages (#3543) 2017-01-17 12:19:50 +01:00
Bas van Kervel 54a65e6d87
cmd,eth,les,internal: remove natspec support 2017-01-17 12:13:50 +01:00
Bas van Kervel 745a3adebd
core: remove support for Olympic network 2017-01-12 09:50:54 +01:00
Péter Szilágyi b52fde7cf7 Merge pull request #3546 from fjl/deps-update
vendor: update dependencies
2017-01-11 12:14:08 +02:00
Felix Lange d78f9b834a vendor: update all dependencies except Azure SDK
The Azure SDK doesn't support Go 1.5 anymore. We can't upgrade it until
Go 1.8 comes out.
2017-01-10 22:33:24 +01:00
Felix Lange 445deb7470 cmd/utils: fix comma-separated --bootnodes 2017-01-10 21:13:43 +01:00
Felix Lange b9b3efb09f all: fix ineffectual assignments and remove uses of crypto.Sha3
go get github.com/gordonklaus/ineffassign
ineffassign .
2017-01-09 16:24:42 +01:00
Felix Lange 13e3b2f433 logger, pow/dagger, pow/ezp: delete dead code 2017-01-06 18:18:07 +01:00
Felix Lange f2da6581ba all: fix issues reported by honnef.co/go/simple/cmd/gosimple 2017-01-06 18:18:07 +01:00
Péter Szilágyi 8f9daaa3ba Merge pull request #3518 from fjl/ethclient-dependency-cleanup
core/types: dependency cleanup
2017-01-06 15:42:03 +02:00
Felix Lange b9683d3748 params: avoid importing p2p/discover for bootnodes
params is imported by leaf-ish library packages and should not pull in
the p2p stack.
2017-01-06 14:15:21 +01:00
Péter Szilágyi bbce726c8a Merge pull request #3515 from bas-vk/exportropsten
core,cmd/utils: bugfix for ropsten dump imports
2017-01-05 12:54:45 +02:00
Péter Szilágyi 06b381d1c9 cmd/utils: disallow `--fakepow` for live operation (#3512) 2017-01-05 11:40:56 +01:00
Bas van Kervel a1798a8188
core,cmd/utils: bugfix for ropsten dump imports 2017-01-05 11:31:57 +01:00
Péter Szilágyi 2ce30382d9
cmd/utils, eth, les: bubble --fakepow flag into eth/les too 2017-01-04 10:42:41 +02:00
Nick Johnson 9ba9fe818d cmd/utils, eth: Add gzip support for chain dump and restore 2016-12-14 08:59:55 +00:00
Luca Zeug 3363a1c227 cmd/utils: update helpscreen's testnet description (#3377) 2016-11-29 16:23:11 +02:00
Péter Szilágyi 7267f796e6
cmd: drop DAO related choice flags since ETC diverged 2016-11-29 15:50:27 +02:00
Viktor Trón e949a2ed2f cmd/bzzd: swarm daemon fixes (#3359)
* cmd/bzzd: add missing p2p/discovery flags
* cmd/bzzd: fix two bugs crashing bzzd if bootnodes flag given
* cmd/bzzd: make no swap default, renamed flag bzznoswap->bzzswap
* internal/web3ext: correct methods for bzz IPC module
* cmd/bzzd: ethapi param not mandatory. Warning if no blockchain
* cmd/bzzd: correct default IPC modules in help string
* cmd/utils: fix help description for networkId - add Ropsten
* cmd/bzz, swarm/api, swarm/network: add swarm networkId flag
* cmd/bzzd: change nosync flag to sync and BootTFlag
2016-11-28 13:29:33 +01:00
Péter Szilágyi 4c8c5e2f74 cmd, ethstats, les, mobile, params: native netstats (#3336) 2016-11-25 16:55:06 +01:00
Felix Lange d1a95c643e Merge pull request #3325 from fjl/p2p-netrestrict
Prevent relay of invalid IPs, add --netrestrict
2016-11-25 13:59:18 +01:00
Jeffrey Wilcke 6cc87a31c6 cmd/utils, internal/web3ext: removed httpGet 2016-11-25 12:31:06 +01:00
Felix Lange 586f10ecb1 cmd/utils, VERSION: 1.5.4 unstable 2016-11-24 23:06:25 +01:00
Felix Lange 978737f5d5 cmd/utils: 1.5.3 stable 2016-11-24 23:03:13 +01:00
Jeffrey Wilcke a8ca75738a core: implemented new ropsten testnet 2016-11-23 09:34:01 +01:00
Jeffrey Wilcke aad4890082 cmd/geth, core, light, mobile: removed state account StartingNonce
All account's nonce start at 0.
2016-11-23 09:34:01 +01:00
Felix Lange e5edd3b983 cmd/bootnode, cmd/geth, cmd/bzzd: add --netrestrict 2016-11-22 22:21:18 +01:00
Nick Johnson e1c1fce92c cmd/utils, VERSION: 1.5.3 unstable (#3306) 2016-11-18 12:33:47 +01:00
Nick Johnson 9b95112a2d cmd/utils: Set version string to stable (#3304) 2016-11-18 11:57:48 +01:00
Felix Lange 20eab80189 cmd/utils, VERSION: 1.5.2 unstable 2016-11-17 14:40:32 +01:00
Jeffrey Wilcke b81baf5423 utils: bump stable 2016-11-16 10:46:22 +01:00
Péter Szilágyi 64359c9417 cmd/utils, mobile, params: set the correct field on testnet EIP 155 (#3272) 2016-11-16 00:46:40 +01:00
Felix Lange a96d6c68e2 cmd/utils: remove jit flag handling (#3273)
The jit config options are not handled, but random selection still took
place, printing confusing messages.
2016-11-15 22:20:18 +01:00
Jeffrey Wilcke fd27393df3 utils: unstable 1.5.1 2016-11-15 00:40:40 +01:00
Jeffrey Wilcke 233950da40 utils: stable 1.5.0 2016-11-15 00:39:12 +01:00
Jeffrey Wilcke a672eae3d1 core, core/types: refactored tx chain id checking (#3257)
* core, core/types: refactored tx chain id checking

Refactored explicit chain id checking in to the Sender deriviation method

* cmd/utils, params: define chain ids
2016-11-14 21:37:13 +01:00
Jeffrey Wilcke 7a6c6ec946 cmd/utils, params: define chain ids 2016-11-14 17:23:39 +01:00
Péter Szilágyi dfe79cc784
cmd/utils, mobile: place bootnodes in LGPL packages 2016-11-14 18:00:14 +02:00
Péter Szilágyi 4a439c2359
mobile: port wrappers to EIP155 and EIP158 fork 2016-11-14 18:00:14 +02:00
Péter Szilágyi b61f48e5aa
cmd, mobile, node, p2p: surface the discovery V5 bootnodes 2016-11-14 18:00:14 +02:00
Péter Szilágyi de4b39a1a3
cmd/utils, mobile: update to reprice HF and light client 2016-11-14 17:56:58 +02:00
Péter Szilágyi 178da7c6a9
mobile: initial wrappers for mobile support 2016-11-14 17:56:58 +02:00
Jeffrey Wilcke ae33883c2f cmd/utils: set temporary HF number 2016-11-13 15:50:32 +01:00
Jeffrey Wilcke 4dca5d4db7 core/types, params: EIP#155 2016-11-13 14:55:30 +01:00
Jeffrey Wilcke 445feaeef5 core, core/state, trie: EIP158, reprice & skip empty account write
This commit implements EIP158 part 1, 2, 3 & 4

1. If an account is empty it's no longer written to the trie. An empty
  account is defined as (balance=0, nonce=0, storage=0, code=0).
2. Delete an empty account if it's touched
3. An empty account is redefined as either non-existent or empty.
4. Zero value calls and zero value suicides no longer consume the 25k
  reation costs.

params: moved core/config to params

Signed-off-by: Jeffrey Wilcke <jeffrey@ethereum.org>
2016-11-13 10:44:04 +01:00
Felix Lange b8bd9a71c8 all: update license information 2016-11-09 02:51:34 +01:00
Zsolt Felfoldi 49da42983a p2p/discv5: added new topic discovery package 2016-11-09 02:12:53 +01:00
zsfelfoldi 7db7109a5b cmd, eth: added light client and light server modes 2016-11-09 02:12:53 +01:00
Péter Szilágyi 63d293cdbe Merge pull request #3195 from karalabe/fix-testnet-fastsync
cmd/utils, core, params: explicitly pick reprice fork for fast sync
2016-10-29 17:21:26 +03:00
Péter Szilágyi f0dbec0c93
cmd, params: only set default fork configs for test and mainnet 2016-10-29 17:07:07 +03:00
gluk256 79789af2e7 whisper: project restructured, version 5 introduced (#3022)
whisper: project restructured, version 5 introduced

This commits adds a draft version of the new shh v5 protocol.
The new version is not on by default, --shh still selects version 2.
2016-10-29 14:11:37 +02:00
Péter Szilágyi 8639b0fae9
cmd/utils, core, params: explicitly pick reprice fork for fast sync 2016-10-29 12:10:00 +03:00
Péter Szilágyi 289b30715d Godeps, vendor: convert dependency management to trash (#3198)
This commit converts the dependency management from Godeps to the vendor
folder, also switching the tool from godep to trash. Since the upstream tool
lacks a few features proposed via a few PRs, until those PRs are merged in
(if), use github.com/karalabe/trash.

You can update dependencies via trash --update.

All dependencies have been updated to their latest version.

Parts of the build system are reworked to drop old notions of Godeps and
invocation of the go vet command so that it doesn't run against the vendor
folder, as that will just blow up during vetting.

The conversion drops OpenCL (and hence GPU mining support) from ethash and our
codebase. The short reasoning is that there's noone to maintain and having
opencl libs in our deps messes up builds as go install ./... tries to build
them, failing with unsatisfied link errors for the C OpenCL deps.

golang.org/x/net/context is not vendored in. We expect it to be fetched by the
user (i.e. using go get). To keep ci.go builds reproducible the package is
"vendored" in build/_vendor.
2016-10-28 19:05:01 +02:00
Péter Szilágyi 4f46bd19d0
cmd, core/state: allow configurable trie cache generations 2016-10-19 14:55:13 +03:00
Jeffrey Wilcke 64af2aafda core, core/vm: added gas price variance table
This implements 1b & 1c of EIP150 by adding a new GasTable which must be
returned from the RuleSet config method. This table is used to determine
the gas prices for the current epoch.

Please note that when the CreateBySuicide gas price is set it is assumed
that we're in the new epoch phase.

In addition this PR will serve as temporary basis while refactorisation
in being done in the EVM64 PR, which will substentially overhaul the gas
price code.
2016-10-14 18:09:17 +02:00
Felix Lange 44bc2e80dd Merge pull request #2914 from fjl/node-coinhabit
cmd/utils, node: make datadir reusable for bzzd
2016-09-29 14:24:37 +02:00
Felix Lange b04219fdbb cmd/utils: don't check for stderr redirect on windows
The redirect check did not work on Go 1.6 and below because Stat
returned an error for stdout and stderr. In Go 1.7 Stat works on stdout
but doesn't return anything meaningful, causing cmd/geth test failures
because the message is printed to stderr only. Fix it by printing to
stdout only.
2016-09-26 17:35:19 +02:00
Felix Lange b42a5b118f common, node: move datadir defaults into package node 2016-09-16 15:24:31 +02:00
Felix Lange eeb322ae64 node: ensure datadir can be co-inhabited by different instances
This change ensures that nodes started with different Name but same
DataDir values don't use the same nodekey and IPC socket.
2016-09-16 15:24:31 +02:00
Péter Szilágyi 30860491ba cmd, eth: drop the blockchain version from cli/eth configs 2016-09-15 14:09:47 +03:00