interesting
/
btc-crawl
mirror of https://github.com/shazow/btc-crawl.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
34 Commits 1 Branch 0 Tags 75 KiB
Go 99.1%
Makefile 0.9%
Go to file
Andrey Petrov bbbfa3cf69 Unbroken, yay. 2014-05-17 21:53:15 -07:00
queue Unbroken, yay. 2014-05-17 21:53:15 -07:00
.gitignore Dont worry about jsons. 2014-05-14 15:47:36 -07:00
LICENSE Initial commit 2014-04-18 21:07:31 -07:00
Makefile Refactored the parallelized bits, still broken though. 2014-05-17 19:44:33 -07:00
README.md Unbroken, yay. 2014-05-17 21:53:15 -07:00
client.go Flags for ALL the things. 2014-04-23 16:03:34 -07:00
cmd.go Unbroken, yay. 2014-05-17 21:53:15 -07:00
crawler.go Unbroken, yay. 2014-05-17 21:53:15 -07:00
ext_btcd.go Importing code. 2014-04-18 21:26:47 -07:00
log.go Superhappyfuntimelogging. 2014-04-24 19:13:33 -07:00
peer.go Properly block on first item. 2014-05-15 17:04:09 -07:00
seed.go Superhappyfuntimelogging. 2014-04-24 19:13:33 -07:00

README.md

btc-crawl

Bitcoin node network crawler (written in golang).

This is a for-fun project to explore the Bitcoin protocol and network.

Current status:

  • JSON streaming is in place, and graceful shutdown.
  • It crawls with all kinds of nice parameters but stores everything in memory until dumping a giant JSON blob at the end.
  • It crawls from hard-coded values and spits a bunch of stuff to stdout.

Usage

$ go get github.com/shazow/btc-crawl
$ btc-crawl --help
...
$ btc-crawl --concurrency 100 --output btc-crawl.json --verbose
...

Estimated crawl time: Unknown.

There should be under 10,000 active network nodes at any given time, according to bitnodes.io. Each node returns around ~2,500 known nodes, but usually only several have timestamps within the last hour.

Todo

(In approximate order of priority)

  • Apply peer-age filter to results
  • Output using some space-conscious format. Right now the output file grows fairly quickly.
  • Namespace useful sub-packages properly (outside of main)

License

MIT (see LICENSE file).