Commit Graph

124 Commits

Author SHA1 Message Date
rjl493456442 b6b6f52ec8
cmd: implement preimage dump and import cmds 2018-03-26 12:51:46 +03:00
Kurkó Mihály 39c16c8a1e cmd, ethdb, vendor: integrate leveldb iostats (#16277)
* cmd, dashboard, ethdb, vendor: send iostats to dashboard

* ethdb: change names

* ethdb: handle parsing errors

* ethdb: handle iostats syntax error

* ethdb: r -> w
2018-03-08 14:59:00 +02:00
Anton Evangelatov ae9f97221a metrics: pull library and introduce ResettingTimer and InfluxDB reporter (#15910)
* go-metrics: fork library and introduce ResettingTimer and InfluxDB reporter.

* vendor: change nonsense/go-metrics to ethersphere/go-metrics

* go-metrics: add tests. move ResettingTimer logic from reporter to type.

* all, metrics: pull in metrics package in go-ethereum

* metrics/test: make sure metrics are enabled for tests

* metrics: apply gosimple rules

* metrics/exp, internal/debug: init expvar endpoint when starting pprof server

* internal/debug: tiny comment formatting fix
2018-02-23 11:56:08 +02:00
Péter Szilágyi 3e89b80ccb
ethdb: reset the batch size too on reset 2018-01-30 19:12:28 +02:00
Martin Holst Swende 017b9f7eac core, ethdb: reuse database batches (#15989)
* leveldb: Update leveldb to 211f780 (poolfix)

* core, ethdb: reuse database batches
2018-01-30 19:03:31 +02:00
Péter Szilágyi 5258785c81
cmd, core, eth/tracers: support fancier js tracing (#15516)
* cmd, core, eth/tracers: support fancier js tracing

* eth, internal/web3ext: rework trace API, concurrency, chain tracing

* eth/tracers: add three more JavaScript tracers

* eth/tracers, vendor: swap ottovm to duktape for tracing

* core, eth, internal: finalize call tracer and needed extras

* eth, tests: prestate tracer, call test suite, rewinding

* vendor: fix windows builds for tracer js engine

* vendor: temporary duktape fix

* eth/tracers: fix up 4byte and evmdis tracer

* vendor: pull in latest duktape with my upstream fixes

* eth: fix some review comments

* eth: rename rewind to reexec to make it more obvious

* core/vm: terminate tracing using defers
2017-12-21 13:56:11 +02:00
Felix Lange 10181b57a9 core, eth/downloader: commit block data using batches (#15115)
* ethdb: add Putter interface and Has method

* ethdb: improve docs and add IdealBatchSize

* ethdb: remove memory batch lock

Batches are not safe for concurrent use.

* core: use ethdb.Putter for Write* functions

This covers the easy cases.

* core/state: simplify StateSync

* trie: optimize local node check

* ethdb: add ValueSize to Batch

* core: optimize HasHeader check

This avoids one random database read get the block number. For many uses
of HasHeader, the expectation is that it's actually there. Using Has
avoids a load + decode of the value.

* core: write fast sync block data in batches

Collect writes into batches up to the ideal size instead of issuing many
small, concurrent writes.

* eth/downloader: commit larger state batches

Collect nodes into a batch up to the ideal size instead of committing
whenever a node is received.

* core: optimize HasBlock check

This avoids a random database read to get the number.

* core: use numberCache in HasHeader

numberCache has higher capacity, increasing the odds of finding the
header without a database lookup.

* core: write imported block data using a batch

Restore batch writes of state and add blocks, tx entries, receipts to
the same batch. The change also simplifies the miner.

This commit also removes posting of logs when a forked block is imported.

* core: fix DB write error handling

* ethdb: use RLock for Has

* core: fix HasBlock comment
2017-09-09 19:03:07 +03:00
gary rong 73c5aba21f ethdb: return copied value from MemDatabase.Get (#14958) 2017-08-11 12:41:49 +02:00
Egon Elbre 6f8c7b0def ethdb: add basic and parallel sanity tests (#14938)
* ethdb: add basic sanity test

* ethdb: test MemDatabase

* ethdb: add parallel tests
2017-08-08 12:32:10 +03:00
Felix Lange 693d9ccbfb trie: more node iterator improvements (#14615)
* ethdb: remove Set

Set deadlocks immediately and isn't part of the Database interface.

* trie: add Err to Iterator

This is useful for testing because the underlying NodeIterator doesn't
need to be kept in a separate variable just to get the error.

* trie: add LeafKey to iterator, panic when not at leaf

LeafKey is useful for callers that can't interpret Path.

* trie: retry failed seek/peek in iterator Next

Instead of failing iteration irrecoverably, make it so Next retries the
pending seek or peek every time.

Smaller changes in this commit make this easier to test:

* The iterator previously returned from Next on encountering a hash
  node. This caused it to visit the same path twice.
* Path returned nibbles with terminator symbol for valueNode attached
  to fullNode, but removed it for valueNode attached to shortNode. Now
  the terminator is always present. This makes Path unique to each node
  and simplifies Leaf.

* trie: add Path to MissingNodeError

The light client trie iterator needs to know the path of the node that's
missing so it can retrieve a proof for it. NodeIterator.Path is not
sufficient because it is updated when the node is resolved and actually
visited by the iterator.

Also remove unused fields. They were added a long time ago before we
knew which fields would be needed for the light client.
2017-06-20 18:26:09 +02:00
Péter Szilágyi 9184249b39 Logger updates 3 (#3730)
* accounts, cmd, eth, ethdb: port logs over to new system

* ethdb: drop concept of cache distribution between dbs

* eth: fix some log nitpicks to make them nicer
2017-03-02 14:06:16 +01:00
Péter Szilágyi d4fd06c3dc
all: blidly swap out glog to our log15, logs need rework 2017-02-23 12:16:44 +02:00
Nick Johnson d30d7800e0 ethdb: Implement interface for prefixed operations to the DB (#3536) 2017-01-11 13:26:09 +01:00
Felix Lange 35a7dcb162 all: gofmt -w -s 2017-01-06 15:52:03 +01:00
zsfelfoldi 7db7109a5b cmd, eth: added light client and light server modes 2016-11-09 02:12:53 +01:00
Felix Lange 44bc2e80dd Merge pull request #2914 from fjl/node-coinhabit
cmd/utils, node: make datadir reusable for bzzd
2016-09-29 14:24:37 +02:00
Péter Szilágyi 710435b51b core, eth, trie: reuse trie journals in all our code 2016-09-28 11:27:31 +03:00
Felix Lange 52ede09b17 ethdb: add accessor for database directory 2016-09-16 15:24:31 +02:00
Felix Lange 84d11c19fd eth: remove dapp database remains 2016-08-17 17:39:03 +02:00
villesundell c7442ef0d2 ethdb, p2p/discover: replace "alloted" with "allotted" (#2785) 2016-07-12 11:42:47 +02:00
Péter Szilágyi e90958cd29 cmd, eth, ethdb, node: prioritise chaindata for resources, bump cache 2016-03-09 10:33:39 +02:00
Jeffrey Wilcke 483feb0d3f Merge pull request #2242 from jimenezrick/upstream-crypto
Closes #2241: Use Keccak-256 from golang.org/x/crypto/sha3 and mention explicitly
2016-02-24 12:57:57 +01:00
Péter Szilágyi f1ddb1a7ad ethdb: fully disable metrics if not requested (oops?) 2016-02-23 15:47:04 +02:00
Ricardo Catalinas Jiménez 53ce0a76d8 ethdb: Remove outdated README pointing to an old link 2016-02-21 22:46:31 +00:00
Péter Szilágyi f3d4ce0d16 core/state, ethdb, trie: test intermediate secure key leak, fix memdb bug 2016-01-20 16:06:28 +02:00
Felix Lange e6fb69296e common: remove old RLP implementation, Value and ExtPackage
In order to make this happen, kill all remaining trivial uses of
common/{rlp,value}.go. The non-trivial ones have been updated earlier.
2015-12-18 12:09:10 +01:00
Péter Szilágyi 5b0ee8ec30 core, eth, trie: fix data races and merge/review issues 2015-10-21 16:49:55 +03:00
Péter Szilágyi b97e34a8e4 eth/downloader: concurrent receipt and state processing 2015-10-19 10:03:10 +03:00
Felix Lange b252589960 ethdb: remove Flush 2015-09-14 23:36:30 +02:00
Felix Lange d581dfee5f ethdb: copy stored memdb values
Storing a value in LevelDB copies the bytes, modifying the value
afterwards does not affect the content of the database. This commit
ensures that MemDatabase satisfies the same property.
2015-09-14 23:36:30 +02:00
Felix Lange 8b32f10f16 ethdb: add NewBatch 2015-09-14 23:36:30 +02:00
Felix Lange 8c4dab77ba all: move common.Database to package ethdb 2015-09-14 23:36:30 +02:00
Péter Szilágyi 42f44dda54 eth, eth/downloader: handle header requests, table driven proto tests 2015-08-24 17:57:28 +03:00
Jeffrey Wilcke a23478c0be core, eth, trie, xeth: merged state, chain, extra databases in one 2015-08-07 22:29:02 +02:00
Jeffrey Wilcke 1fad8798ec Merge pull request #1515 from fjl/license-fixes
all: fix license headers one more time
2015-07-28 04:29:42 -07:00
Felix Lange bfbcfbe4a9 all: fix license headers one more time
I forgot to update one instance of "go-ethereum" in commit 3f047be5a.
2015-07-23 18:35:11 +02:00
Jeffrey Wilcke 8f56eea77d ethdb, trie: removed RLE compression 2015-07-23 12:57:33 +02:00
Jeffrey Wilcke 0cff61beda Merge pull request #1510 from fjl/license-fixes
all: license fixes
2015-07-23 01:43:11 -07:00
Felix Lange 3f047be5aa all: update license headers to distiguish GPL/LGPL
All code outside of cmd/ is licensed as LGPL. The headers
now reflect this by calling the whole work "the go-ethereum library".
2015-07-22 18:51:45 +02:00
Péter Szilágyi c7e7778f2a cmd, core, eth, ethdb: cache flag to allocate memory for db internal use 2015-07-22 14:00:52 +03:00
Péter Szilágyi eb2d168710 eth, ethdb: fix a data race during startup/shutdown 2015-07-09 12:44:07 +03:00
Felix Lange ea54283b30 all: update license information 2015-07-07 14:12:44 +02:00
Péter Szilágyi ccbb56b4f2 cmd/geth, eth, ethdb: monitor database compactions 2015-06-30 00:51:29 +02:00
obscuren 2a5a55efaf ethdb: accessor for LDB. TODO remove this interface 2015-06-29 18:51:48 +02:00
Péter Szilágyi 803b3c4a82 eth, ethdb: measure database operation latencies too 2015-06-24 18:34:05 +03:00
Péter Szilágyi 792b0ddccd core, eth, eth/fetcher, ethdb: polish metrics gathering a bit 2015-06-24 18:34:04 +03:00
Péter Szilágyi 43e4a6501b core, ethdb: instrument the block and state db
Conflicts:
	ethdb/database.go
2015-06-24 18:34:04 +03:00
obscuren c590b505ed core, ethdb, trie: validate database errors 2015-06-21 16:59:15 +02:00
obscuren 020006a8ed common, ethdb: removed caching and LastTD 2015-05-27 18:03:16 +02:00
obscuren 84cd618585 ethdb: documentation and corruption recovery 2015-05-21 11:43:05 +02:00
Bas van Kervel 95773b9673 removed redundant newlines in import block 2015-05-12 15:20:53 +02:00
Bas van Kervel b79dd188d9 replaced several path.* with filepath.* which is platform independent 2015-05-12 14:24:11 +02:00
obscuren 13f8f65a58 eth, ethdb: lower the amount of open files & improve err messages for db
Closes #880
2015-05-12 11:28:55 +02:00
obscuren 0e211ae203 ethdb: set open file limit to 128 for all leveldbs 2015-05-10 01:55:39 +02:00
Bas van Kervel c273ed7d82 Moved leveldb update loop to eth/backend
change order of block insert and update LastBlock

bugfix, wrong hash stored in blockDb
2015-04-23 17:58:58 +02:00
obscuren 7f32a08b60 Queued level db writes and batch writes. Closes #647 2015-04-07 22:19:01 +02:00
Felix Lange c161d73d42 common: drop accessors for Value.Val
I don't see why we would need two different accessors for a public field.
2015-03-20 14:00:26 +01:00
obscuren b523441361 Moved ethutil => common 2015-03-16 11:27:38 +01:00
Felix Lange 80985f97da cmd/evm, core, ethdb, state, tests/helper: remove ReadConfig calls 2015-03-10 02:39:53 +01:00
Felix Lange bdba044a80 ethutil: remove Config variable
Various functions throughout the codebase used it to grab settings. This
has to stop because I want to use them without reading the config file.

These functions can now be used without reading the config first:

* ethdb.NewLDBDatabase
* ethrepl.NewJSRepl
* vm.New
2015-03-06 02:46:56 +01:00
obscuren 38faf2c51a removed messages 2015-02-12 17:06:15 +01:00
obscuren 34f72ddb3d Decode from stream directly 2014-12-23 15:18:48 +01:00
obscuren 0f460ad26e Added caching and database interface to trie
* Reimplemented caching for trie
* Reimplemented resetting and persisting trie
2014-11-19 16:21:28 +01:00
obscuren a82b89e2d5 Added storage root to dump 2014-11-03 23:45:44 +01:00
obscuren 76c9c8d653 Compress data on db level. Closes #174 2014-11-03 00:31:15 +01:00
Felix Lange 69baa465ea Merge eth-go repository into go-ethereum
mist, etheruem have been moved to cmd/
2014-10-23 15:01:27 +02:00
obscuren 962255b373 Removed old code 2014-08-23 11:00:33 +02:00
obscuren f4fa0d48cb Moved keyring to ethutil & removed old methods. Implements #20 2014-05-14 13:54:40 +02:00
obscuren c0fcefa3a0 Added a GetKeys method to support multiple accounts 2014-02-28 12:18:41 +01:00
obscuren c9f3d1c00b leveldb API changed for NewIterator. Fixes #20 2014-02-28 10:36:06 +01:00
obscuren 4f4175a3e2 Addad db name for new ldb 2014-02-25 11:21:03 +01:00
obscuren 55c1c220d0 Added delete to database interfaces 2014-02-24 12:12:01 +01:00
obscuren 73fd358d94 Removed RlpValue in favour of Value 2014-02-15 01:34:18 +01:00
obscuren f6d1bfe45b The great merge 2014-02-14 23:56:09 +01:00