Commit Graph

163 Commits

Author SHA1 Message Date
Janoš Guljaš 996755c4a8 cmd/swarm, swarm: LocalStore storage integration 2019-05-10 12:26:26 +02:00
C. Brown be4d74f8d2 cmd, internal/build, docker: advertise commit date in unstable build versions (#19522)
* add-date-to unstable

* fields-insteadof-split

* internal/build: support building with missing git

* docker: add git history back to support commit date in version

* internal/build: use PR commits hashes for PR builds
2019-05-08 16:44:28 +03:00
Ferenc Szabo 26b50e3ebe cmd/swarm: fix resource leaks in tests (#19443)
* swarm/api: fix file descriptor leak in NewTestSwarmServer

Swarm storage (localstore) was not closed. That resulted a
"too many open files" error if `TestClientUploadDownloadRawEncrypted`
was run with `-count 1000`.

* cmd/swarm: speed up StartNewNodes() by parallelization

Reduce cluster startup time from 13s to 7s.

* swarm/api: disable flaky TestClientUploadDownloadRawEncrypted with -race

* swarm/storage: disable flaky TestLDBStoreCollectGarbage (-race)

With race detection turned on the disabled cases often fail with:
"ldbstore_test.go:535: expected surplus chunk 150 to be missing, but got no error"

* cmd/swarm: fix process leak in TestACT and TestSwarmUp

Each test run we start 3 nodes, but we did not terminate them. So
those 3 nodes continued eating up 1.2GB (3.4GB with -race) after test
completion.

6b6c4d1c27 changed how we start clusters
to speed up tests. The changeset merged together test cases
and introduced a global cluster. But "forgot" about termination.

Let's get rid of "global cluster" so we have a clear owner of
termination (some time sacrifice), while leaving subtests to use the
same cluster.
2019-04-11 12:44:15 +02:00
lash 09924cbcaa cmd/swarm, p2p, swarm: Enable ENR in binary/execadapter (#19309)
* cmd/swarm, p2p, swarm: Enable ENR in binary/execadapter

* cmd/p2p/swarm: Remove comments + config.Enode nomarshal

* p2p/simulations: Remove superfluous error check

* p2p/simulation: Move init enode comment

* swarm/api: Check error in config test

* swarm, p2p/simulations, cmd/swarm: Use nodekey in binary record sign

* cmd/swarm: Make nodekey available for swarm api config
2019-03-22 05:55:47 +01:00
Anton Evangelatov baded64d88
swarm/network: measure time of messages in priority queue (#19250) 2019-03-20 21:30:34 +01:00
Elad e7d1867964 contracts, swarm: implement EIP-1577 (#19285)
* contracts/ens: update public resolver solidity code

* contracts/ens: update public resolver, update go bindings

* update build

* fix ens.sol

* contracts/ens: change contract interface

* contracts/ens: implement public resolver changes

* contracts/ens: added ENSRegistry contract

* contracts/ens: reinstate old contract code

* contracts/ens: update README.md

* contracts/ens: added test coverage for fallback contract

* contracts/ens: added support for fallback contract

* contracts/ens: removed unused contract code

* contracts/ens: add todo and decode multicodec stub

* add encode

* vendor: add ipfs cid libraries

* contracts/ens: cid sanity tests

* contracts/ens: more cid sanity checks

* contracts/ens: wip integration

* wip

* Revert "vendor: add ipfs cid libraries"

This reverts commit 29d9b6b294.

* contracts/ens: removed multiformats dependencies

* contracts/ens: added decode tests

* contracts/ens: added eip spec test, minor changes to exiting tests

* contracts/ens: moved cid decoding to own file

* contracts/ens: added unit test to encode hash to content hash

* contracts/ens: removed unused code

* contracts/ens: fix ens tests to use cid decode and encode

* contracts/ens: adjust swarm multicodecs after pr merge

* contracts/ens: fix linter error

* constracts/ens: address PR comments

* cmd, contracts: make peoples lives easier

* contracts/ens: fix linter error

* contracts/ens: address PR comments
2019-03-20 09:33:24 +01:00
Anton Evangelatov 3d067b0cea
cmd/swarm/swarm-smoke: do not fail if a node does not respond to rpc (#19280) 2019-03-15 17:36:39 +01:00
Elad df488975bd cmd/swarm: dont connect to bootnodes in tests (#19270)
* cmd/swarm: dont connect to bootnodes in tests

* cmd/utils: check for empty string when parsing enode
2019-03-15 06:20:21 +01:00
Anton Evangelatov 1a3e25e4c1
swarm: tracing improvements (#19249) 2019-03-11 11:45:34 +01:00
Anton Evangelatov ceeb047e69 cmd/swarm/swarm-smoke: better logs when debug mode triggers (#19237)
* cmd/swarm/swarm-smoke: better logs for debug functionality;

* cmd/swarm/swarm-smoke: fixup
2019-03-08 08:52:05 +01:00
Elad 4687391213 cmd/swarm: do not ignore cache size=0 (#19231) 2019-03-07 12:44:00 +01:00
Elad 34c85def3e cmd/swarm/swarm-smoke: sliding window test should not time out (#19152) 2019-03-05 17:43:05 +01:00
Anton Evangelatov 4e9230ea7a
swarm: enable p2p/discovery and disable dynamic dialling (#19189) 2019-03-01 12:20:37 +01:00
lash 62d9d63858 swarm/network: WIP consider all nodes for healthy iteration (#19155)
* swarm/network: WIP consider all nodes for healthy iteration

* swarm/network/simulation: extend TestWaitTillHealthy to really check kads are healthy

* cmd/swarm/swarm-snapshot: fixed bugs in snapshot creation binary

* swarm/network/simulation: addressed PR comments

* swarm/network/simulation: defer sim.Clsoe()

* swarm/network/simulation: fixed wrong sim.Close()

* swarm/network/simulation: addressed PR comments

* cmd/swarm/swarm-snapshot: reducing default to 8 nodes, more to 4

* cmd/swarm/swarm-snapshot: extended timeout to 3 mins, or 256 nodes snapshot times out

* swarm/network/simulation: More PR comments
2019-02-28 08:12:50 +01:00
Matthew Halpern 90b6cdaadf cmd,swarm: enforce camel case variable names (#19060) 2019-02-24 12:39:23 +01:00
Janoš Guljaš 64d10c0872 swarm: mock store listings (#19157)
* swarm/storage/mock: implement listings methods for mem and rpc stores

* swarm/storage/mock/rpc: add comments and newTestStore helper function

* swarm/storage/mock/mem: add missing comments

* swarm/storage/mock: add comments to new types and constants

* swarm/storage/mock/db: implement listings for mock/db global store

* swarm/storage/mock/test: add comments for MockStoreListings

* swarm/storage/mock/explorer: initial implementation

* cmd/swarm/global-store: add chunk explorer

* cmd/swarm/global-store: add chunk explorer tests

* swarm/storage/mock/explorer: add tests

* swarm/storage/mock/explorer: add swagger api definition

* swarm/storage/mock/explorer: not-zero test values for invalid addr and key

* swarm/storage/mock/explorer: test wildcard cors origin

* swarm/storage/mock/db: renames based on Fabio's suggestions

* swarm/storage/mock/explorer: add more comments to testHandler function

* cmd/swarm/global-store: terminate subprocess with Kill in tests
2019-02-23 10:47:33 +01:00
Janoš Guljaš 9d5e10f5bb cmd/swarm/global-store: use kill instead interrupt in tests (#19142) 2019-02-20 22:58:25 +01:00
holisticode 62d7688d0a cmd/swarm/swarm-smoke: Trigger chunk debug on timeout (#19101)
* cmd/swarm/swarm-smoke: first version trigger has-chunks on timeout

* cmd/swarm/swarm-smoke: finalize trigger to chunk debug

* cmd/swarm/swarm-smoke: fixed httpEndpoint for trigger

* cmd/swarm/swarm-smoke: port

* cmd/swarm/swarm-smoke: ws not rpc

* cmd/swarm/swarm-smoke: added debug output

* cmd/swarm/swarm-smoke: addressed PR comments

* cmd/swarm/swarm-smoke: renamed track-timeout and track-chunks
2019-02-18 12:05:22 +01:00
Matthew Halpern fa87929a2f cmd: prefer nil slices over zero-length slices (#19077) 2019-02-15 01:02:11 +02:00
holisticode 41597c2856 swarm: Debug API and HasChunks() API endpoint (#18980) 2019-02-07 15:49:19 +01:00
Janoš Guljaš 33d0a0efa6 cmd/swarm/global-store: global store cmd (#19014) 2019-02-07 15:46:58 +01:00
Anton Evangelatov d212535ddd
cmd/swarm/swarm-smoke: refactor generateEndpoints (#19006) 2019-02-07 14:38:32 +01:00
holisticode 7f55b0cbd8 cmd/swarm: hashes command (#19008) 2019-02-07 13:51:24 +01:00
Janoš Guljaš 26aea73673 cmd, node, p2p/simulations: fix node account manager leak (#19004)
* node: close AccountsManager in new Close method

* p2p/simulations, p2p/simulations/adapters: handle node close on shutdown

* node: move node ephemeralKeystore cleanup to stop method

* node: call Stop in Node.Close method

* cmd/geth: close node.Node created with makeFullNode in cli commands

* node: close Node instances in tests

* cmd/geth, node: minor code style fixes

* cmd, console, miner, mobile: proper node Close() termination
2019-02-07 12:40:36 +02:00
Anton Evangelatov c5c9cef5c0
cmd/swarm/swarm-smoke: remove wrong metrics (#18970) 2019-01-30 14:02:15 +01:00
Elad b91bf08876 cmd/swarm/swarm-smoke: sliding window test (#18967) 2019-01-30 09:46:44 +01:00
Anton Evangelatov 21acf0bc8d cmd/utils: allow for multiple influxdb tags (#18520)
This PR is replacing the metrics.influxdb.host.tag cmd-line flag with metrics.influxdb.tags - a comma-separated key/value tags, that are passed to the InfluxDB reporter, so that we can index measurements with multiple tags, and not just one host tag.

This will be useful for Swarm, where we want to index measurements not just with the host tag, but also with bzzkey and git commit version (for long-running deployments).
2019-01-29 09:14:24 +01:00
gluk256 ad13d2d407 swarm/version: commit version added (#18510) 2019-01-24 12:35:10 +01:00
Anton Evangelatov bbd120354a
swarm: bootnode-mode, new bootnodes and no p2p package discovery (#18498) 2019-01-24 12:02:18 +01:00
holisticode 560957799a cmd/swarm/swarm-smoke: use ResettingTimer instead of Counters for times (#18479) 2019-01-18 18:14:06 +01:00
Anton Evangelatov a0b0db6305
cmd/swarm: use resetting timer to measure fetch time (#18474) 2019-01-18 13:27:27 +01:00
Elad 632135ce4c cmd/swarm/swarm-snapshot: disable tests on windows (#18478) 2019-01-18 13:22:05 +01:00
holisticode 257bfff316 Upload speed (#18442) 2019-01-17 17:25:27 +01:00
Elad 34f11e752f cmd/swarm/swarm-snapshot: swarm snapshot generator (#18453)
* cmd/swarm/swarm-snapshot: add binary to create network snapshots

* cmd/swarm/swarm-snapshot: refactor and extend tests

* p2p/simulations: remove unused triggerChecks func and fix linter

* internal/cmdtest: raise the timeout for killing TestCmd

* cmd/swarm/swarm-snapshot: add more comments and other minor adjustments

* cmd/swarm/swarm-snapshot: remove redundant check in createSnapshot

* cmd/swarm/swarm-snapshot: change comment wording

* p2p/simulations: revert Simulation.Run from master

https://github.com/ethersphere/go-ethereum/pull/1077/files#r247078904

* cmd/swarm/swarm-snapshot: address pr comments

* swarm/network/simulations/discovery: removed snapshot write to file

* cmd/swarm/swarm-snapshot, swarm/network/simulations: removed redundant connection event check, fixed lint error
2019-01-16 14:33:02 +01:00
Dave McGregor 33d233d3e1
vendor, crypto, swarm: switch over to upstream sha3 package 2019-01-04 09:26:07 +02:00
Anton Evangelatov 9e9fc87e70
swarm: remove unused/dead code (#18351) 2018-12-23 17:31:32 +01:00
Elad bb724080ca cmd/swarm, metrics, swarm/api/client, swarm/storage, swarm/metrics, swarm/api/http: add instrumentation (#18274) 2018-12-11 09:21:58 +01:00
Anton Evangelatov 7e7781ffaa cmd/swarm: add flag for application name (swarm or swarm-private) (#18189)
* cmd/swarm: add flag for application name (swarm or swarm-private)

* cmd/swarm/swarm-smoke: return correct exit code

* cmd/swarm/swarm-smoke: remove colorable

* remove swarm/grafana_dashboards
2018-11-29 18:43:15 +02:00
Javier Peletier 4f0d978eaa cmd/swarm: update should error on manifest mismatch (#18047)
* cmd/swarm: fix ethersphere/go-ethereum#979:

update should error on manifest mistmatch

* cmd/swarm: fixed comments and remove sprintf from log.Info

* cmd/swarm: remove unnecessary comment
2018-11-26 17:37:59 +01:00
Javier Peletier 2714e8f091 Remove multihash from Swarm bzz:// for Feeds (#18175) 2018-11-26 16:10:22 +01:00
Ferenc Szabo 76f5f662cc cmd/swarm: FUSE do not require --ipcpath (#18112)
- Have `${DataDir}/bzzd.ipc` as IPC path default.
- Respect the `--datadir` flag.
- Keep only the global `--ipcpath` flag and drop the local `--ipcpath` flag
  as flags might overwrite each other. (Note: before global `--ipcpath`
  was ignored even if it was set)

fixes ethersphere#795
2018-11-23 01:32:34 +01:00
Anton Evangelatov d31f1f4fdb
cmd/swarm/swarm-smoke: update smoke tests to fit the new scheme for the k8s cluster (#18104) 2018-11-19 14:58:10 +01:00
Anton Evangelatov 6b6c4d1c27
cmd/swarm: speed up tests - use global cluster (#18129) 2018-11-19 14:57:22 +01:00
Alexey Sharov eb8fa3cc89 cmd/swarm, swarm/api/http, swarm/bmt, swarm/fuse, swarm/network/stream, swarm/storage, swarm/storage/encryption, swarm/testutil: use pseudo-random instead of crypto-random for test files content generation (#18083)
- Replace "crypto/rand" to "math/rand" for files content generation
- Remove swarm/network_test.go.Shuffle and swarm/btm/btm_test.go.Shuffle - because go1.9 support dropped (see https://github.com/ethereum/go-ethereum/pull/17807 and comments to swarm/network_test.go.Shuffle)
2018-11-14 09:21:14 +01:00
Andrew Chiw a0876f7433 Imply that SwarmApiFlag is the API endpoint to connect to, not to listen on (#18071) 2018-11-12 13:04:13 +01:00
Anton Evangelatov cf3b187bde swarm, cmd/swarm: address ineffectual assignments (#18048)
* swarm, cmd/swarm: address ineffectual assignments

* swarm/network: remove unused vars from testHandshake

* swarm/storage/feed: revert cursor changes
2018-11-07 20:39:08 +01:00
Elad 126dfde6c9 cmd/swarm: auto resolve default path according to env flag (#17960) 2018-11-04 07:59:58 +01:00
Roc Yu 3e1cfbae93 cmd/swarm/swarm-smoke: fix issue that loop variable capture in func (#17992) 2018-10-29 10:00:00 +01:00
lash 862d6f2fbf cmd/swarm: Smoke test for Swarm Feed (#17892) 2018-10-12 16:24:00 +02:00
Elad 4868964bb9 cmd/swarm: split flags and cli command declarations to the relevant files (#17896) 2018-10-12 14:51:38 +02:00