2014-04-18 23:30:19 -05:00
|
|
|
# btc-crawl
|
2014-04-18 23:07:31 -05:00
|
|
|
|
|
|
|
|
Bitcoin node network crawler (written in golang).
|
2014-04-18 23:26:47 -05:00
|
|
|
|
|
|
|
|
This is a for-fun project to explore the Bitcoin protocol and network.
|
|
|
|
|
|
2014-04-25 21:54:24 -05:00
|
|
|
Current status:
|
|
|
|
|
* It crawls with all kinds of nice parameters but stores everything in memory
|
|
|
|
|
until dumping a giant JSON blob at the end.
|
2014-04-25 21:54:45 -05:00
|
|
|
* ~~It crawls from hard-coded values and spits a bunch of stuff to
|
2014-04-25 21:54:24 -05:00
|
|
|
stdout.~~
|
2014-04-18 23:29:36 -05:00
|
|
|
|
2014-04-18 23:30:19 -05:00
|
|
|
|
|
|
|
|
## Usage
|
|
|
|
|
|
2014-04-18 23:29:36 -05:00
|
|
|
```
|
|
|
|
|
$ go get github.com/shazow/btc-crawl
|
2014-04-25 22:00:17 -05:00
|
|
|
$ btc-crawl --concurrency 100 --output btc-crawl.json --verbose
|
2014-04-18 23:29:36 -05:00
|
|
|
...
|
|
|
|
|
```
|
2014-04-18 23:30:19 -05:00
|
|
|
|
2014-04-25 22:00:17 -05:00
|
|
|
*Estimated crawl time:* Unknown.
|
|
|
|
|
|
|
|
|
|
There should be under 10,000 active network nodes at any given time, according
|
|
|
|
|
to (bitnodes.io)[https://getaddr.bitnodes.io/]. Each node returns around ~2,500
|
|
|
|
|
known nodes, but usually only several have timestamps within the last hour.
|
|
|
|
|
|
2014-04-18 23:30:19 -05:00
|
|
|
|
|
|
|
|
## Todo
|
|
|
|
|
|
2014-04-25 21:54:24 -05:00
|
|
|
(In approximate order of priority)
|
|
|
|
|
|
2014-04-25 22:00:17 -05:00
|
|
|
* Add some sensible non-verbose output
|
2014-04-25 21:54:24 -05:00
|
|
|
* Apply peer-age filter to results
|
|
|
|
|
* Stream JSON rather than accumulate into a giant array.
|
|
|
|
|
* Add timeout option.
|
|
|
|
|
* Graceful cleanup on Ctrl+C
|
|
|
|
|
* Namespace packages properly (outside of `main`)
|
