17
What would you say is the primary goal you wish to accomplish with Iguana tech?
"The goal for Iguana is to create a scalable Bitcoin Core implementation that is backward compatible and a drop-in replacement. The design needs to meet many constraints, primarily to be as small as possible without sacrificing speed and to be able to work in a parallel sync. That means that each block or bundle needs to be as self-contained as possible, but have external references that can be resolved. This is similar to object files and linkers. The main thing that has these external references are the vins [unspent inputs used in a transaction].
The end goal is an end-to-end, fully integrated end user solution, which is really all the end users care about. Iguana tech is still young, but already it proves that Bitcoin can easily scale to max transaction capacity without causing any issues for the typical nodes. After initial sync, since the vast majority of the data set is in memory-mapped files, it doesn't need a lot of RAM to run. The code size is small enough that it could run on a smartphone and other low-powered devices. Does that make Bitcoin good for IoT?" - JL777
It's been said that Iguana technology allows for a parallel sync of the full BTC blockchain in 30 minutes which uses half the space and starts up instantly. How is this so?
"The files it creates are read-only files and never change after they are created. This allows them to be further compressed into a read-only file system. Without signatures the data set ends up at around 15GB in a read-only volume and 25GB when not compressed. These files are invariant so, once validated, they never need to be verified again which allows for an "instant-on" after each restart. A layer of data added to the read-only files can be used for an in-memory hash table directly as memory-mapped files. This means the startup time is close to 3 seconds and thereafter it is ready to start syncing new blocks.
The first thing you notice when looking at the raw blockchain is that there is a lot of redundancy. So by mapping the high entropy hashes to a 32-bit integer you get a 28-byte savings for each use. For a transaction ID with N outputs, up to N*28 bytes can be saved as each vin refers to the transaction ID.
However, both endian forms need to be put into the
BitTorrent network as the Iguana files are designed to be directly memory mapped. This allows the serialization/deserialization of each and every multibyte field to be skipped. Since the files are less than half the size, even with the doubling due to both endian forms, they still consume less data overall than the raw blockchain.
A Bitcoin RPC compatibility layer is being added currently so that it will be compatible with existing Bitcoin apps. When that is in place, it can be put through the blocktester and other standard Bitcoin validation tests. In the meantime, a GUI team is creating an all HTML/JS GUI that will interface via Bitcoin RPC." - JL777
In Regards to Interleaving
"Each interleave is a mostly independent blockchain, but we need to enforce a fixed ordering based on the interleave regardless of when the block is solved for that interleave. This does create the situation where there is a solved block that can't be confirmed due to a prior interleave not there yet.
This provides the possibility of a rapid delivery feature, where the user can generate 10 different variants of the same tx, so each interleave gets that tx. This would confirm in each interleave, but of course only be valid in the first one that gets confirmed. So a provision needs to exist for charging extra for these redundant tx and to invalidate the ones that are in the unconfirmed later interleaves.
Miners would be able to optimize their revenues by intelligently managing their hashpower based on the state of the 10 interleaves. Also, by having different service levels that come for free, this would create a larger tx fee revenue base via the premium fees and of course from just having 10x the transaction capacity.
10x interleaving works just as well with 1MB blocks, 8MB blocks, subchains, etc." - JL777
In regards to UTXO
"All things needed for block explorer level queries are precalculated and put into the read-only files.
By having each bundle of 2000 blocks being mostly independent of all other blocks, this not only allows for parallel syncing of the blockchain but also using multiple cores for all queries regarding tx history.
At the 30MB size or smaller, the entire UTXO bitmap will fit into CPU L cache and provide 10x faster performance than RAM. RAM is actually quite slow compared to things that can be done totally inside the CPU.
With everything happening in parallel, it is easy to end up thinking a bundle is ready before it is totally ready. So I had to combine the "1st" field along with just waiting 3 seconds after a bundle is written out. That gives time for avoiding race conditions, I am seeing some cases where writing the data out to file is not completed, even though it is memory-mapped. When dealing with multiple threads, it does take time to get a fully synchronized state across all threads, especially when dealing with buffered files." - JL777
Writing Bitcoin Core from Scratch
"I use only stdio, read-only mapped files, bsd sockets, system clock and a few other standard system functions. Openssl/libsecp256k1 is the only crypto dependency I use. The rest is all in my repo (add-on services utilize curl, but that dependency is outside of the bitcoind equivalent). I use no boost, not even libevent. Everything from the protocol, to the blockchain, scripts, signing, transaction construction, and RPC has been coded from sratch. I started writing Iguana last November when I had already written several iterations of MGW which gave me a good understanding of the blockchain and multisig." - JL777