How much RAM is required to run Iguana for blockchain syncing?
"The more RAM you configure for it the better, but even with 8 cores it doesn't go above 32GB during sync. At 4GB, it will mostly serialize the processing toward the end when it is just the large bundles. I haven't optimized the minimum memory setting as my target machine is my 3 year old laptop with 4GB of RAM.
Iguana doesn't use any DB and achieving this performance requires constant time performance for all operations within a bundle. One of the final steps would happen as the main chain is validated linearly as the parallel sync is proceeding. Code that does a parallel sync of the blockchain is mostly working, creating read-only bundle files (of 2000 blocks) and is able to stream at 500mbps to 1gbps speeds without too many slow spots." - JL777
BTC and BTCD Blockchain SYNC
"I now have Iguana doing a parallel BTC and BTCD sync in about one hour and catching up to realtime blocks. I still need to make sure the volatile data is properly updated. At that point, RPC like listunspent will be able to be easily completed as long as the data is valid. But Iguana listunspent will be able to be queried as of any block for any address, including multisig. Bitcoind can't do that, neither could insight DB, so we ended up making a special mod to bitcoind to create a snapshot.
info: block chain size = 63.487 Gigs
info: initializing hash tables
info: estimated number of blocks = 498.82K
info: estimated number of transactions = 128.85M
info: done initializing hash tables - mem = 0.000 Gigs
info: pass 1 -- walk all blocks and build headers ...
1.48% (0.90/60.55 Gigs) -- 163729 blocks -- 3.19 Megs/sec -- ETA 19613 secs -- ELAPSED 295 secs
Just iterating through the existing database takes 19613 seconds, or over 5 hours, when Iguana syncs from the network in one hour and builds a dataset that is able to be queried as of any block for any address. But Iguana uses less HDD space and protects its dataset with read-only files. However, Iguana is taking 2MB of codespace. Now I am
testing BTC and BTCD at the same time. It seems to catch up to realtime while parallel syncing. I found and fixed a bug that had several instances and since then things have been pretty stable." - JL777
Do you anticipate Bitcoin Core taking an interest in implementing your code?
"Well I am not too concerned with whether or not Bitcoin Core feels they want to adopt a 10x faster way to sync. End users will follow the path of least resistance and, without any external dependencies, the one-click install Chrome app won't have any adoption barrier."
Because of the way jl777 has written the Iguana codebase with almost no external dependencies, it is very portable and could be ran as a standalone binary without needing to have it wrapped in a Chrome extension. In fact, compiling the codebase natively is much easier than cross-compiling it into a Chrome app. The build team has it built for a Chrome app, Android, UNIX, OSX, and Win64. There are still some issues to be solved with the Win32 build. Emscripten could also be supported for non-Chrome browsers.
Due to the fact that jl777 has thus far been unable to get the size of his validated Bitcoin blockchain dataset under Google's maximum Chrome extension size of 10GB, the Chrome app will initially run a pruning Bitcoin node by default. It will, however, have no problem running full nodes for most altcoins. A possible solution which might help the Chrome app in running a full Bitcoin node is to run multiple separate Chrome apps that act as file servers. JL777 recognizes, however, that most users interested in running Iguana via a browser probably favor convenience and efficiency over running a full node so finding a solution for this is not atop his priority list at this time.
JL777 also postulated this week in SuperNET Slack about the possibility of increasing the speed of Iguana significantly by creating a CUDA/OpenSSL version of Iguana for use with GPUs. This would allow for dedicating a core per bundle among other optimizations.
With most all the data being read-only, the biggest challenge in implementing GPU coding (out of sync data between cores) is already solved. It might take a few GPU cards to store the complete dataset, but with this method all RPC, even ones that need to scan the entire blockchain, happen within milliseconds and the CPU can be used for the latest bundle and create a system that keeps up with new blocks.
Because Iguana running on most decent modern hardware will be plenty fast, such a GPU method would be overkill. However, this gives you an idea what kinds of things can be achieved with this technology in the future as things require more speed, efficiency, and security.
Once Iguana is ready and it runs a node via a one-click installation, one can imagine that it won't take long for there to be more Iguana nodes than any other type of node. Because Iguana is a multicoind capable of interacting with many blockchains at the same time, it will appeal to altcoin fans looking to run all their coins on the same computer. The inclusion of atomic swap technology and Pangea decentralized poker in the same app will also increase its appeal significantly.
It is remarkable what jl777 and his team have been able to accomplish by assuming a holistic perspective of the entire system, working backwards from the desired result and making optimizations along the way. Operating in this way, it is likely just a matter of time until most every obstacle facing Bitcoin and blockchain technology in general is overcome.
For more infomation, read individual weekly blogs at http://coremedia.info/.
19