Commit Revealed: Issue 1
Welcome to the first issue of Commit Revealed—a newsletter where top engineers in the crypto space share their best (and latest) work, so makers can stay up to date on the state of the art. Let’s dive right in.
Contribution 1: Jeremy Rubin on OP_CHECKTEMPLATEVERIFY
It only seems right to kick things off with Bitcoin. As serendipity would have it, a new primitive—OP_CHECKTEMPLATEVERIFY (OP_CTV)—was recently proposed for inclusion into the protocol. If adopted via soft fork, this opcode can reduce Bitcoin’s congestion, enable businesses to spend less on fees, guarantee payments when fees are high, and set the stage for new types of Bitcoin smart contracts.
In practice, OP_CTV separates the concerns of receiving and spending Bitcoin. What this means is that payers can issue an unlimited number of provably collateralized ‘certified checks’, pack them into a single commitment, and leave them to be redeemed by a recipient at some future point. This offers significant cost and complexity reduction when compared with transaction sizes that scale based on the number of participants. With OP_CTV, payers can send funds to many different parties (e.g., businesses making payroll, exchanges processing withdrawals), simultaneously, without worrying about fees, and recipients can now receive funds without delay.
And all this in a package that promises to be simple to understand, easy to use, and safe to deploy. With public discussion ongoing, bip-ctv author Jeremy Rubin unpacks his implementation of OP_CTV.
Commit aeab1e4
Rather than walk through some arbitrary implementation aspect of the BIP, Jeremy wanted to show how developers will use OP_CTV, as well as how the current sendmany function compares to his proposed sendmanycompacted function.
This commit (from the checktemplateverify-rpcs branch) is what actually implements the new RPC call sendmanycompacted. And it is this function that decreases the size of a transaction (in bytes) from being proportional to the number of recipients to a constant-sized transaction commitment. Here’s how:
The Interesting Bits
File: src/wallet/rpcwallet.cpp
Lines: 1100-1122
Highlighting the process of setting up transaction templates from bottom up, then finalizing them from the top down, this code is what gives OP_CHECKTEMPLATEVERIFY its name.
It operates by way of accepting a map of interior ‘template transactions’, then hydrating them into ‘real transactions’ by attaching the proper transaction ID. First, Jeremy creates a tree of transaction templates—from the bottom up. This tree describes the interior nodes of a payment created using the new OP_CTV opcode. He then iterates over these nodes—this time from the top down—grabbing the correct transaction template, adding the parent hash (from the one computed previously), then placing its children in the work-queue for similar expansion later. Once all expansions are complete, the transactions are committed to the network.
That’s OP_CHECKTEMPLATEVERIFY in a nutshell. Jeremy also pointed out that by turning the work stack (LIFO) into a queue (FIFO), you can modify the generation order of the outputs (stack being depth-first, queue breadth-first). And that, while the technique shown is applied to Bitcoin, it works with nearly all networks, including account-based ones like Ethereum.
Fantasy Code Review
What good is code without a good code review? In the interest of creating a forum for industry crosstalk that might not happen otherwise, we introduce Fantasy Code Reviews for all highlighted commits.
For his latest BIP, Jeremy thought it would be interesting to have Brian Armstrong comment on what this BIP could mean for Bitcoin and the industry. From the perspective of Bitcoin contributors discussing OP_CTV inclusion, his comments would be quite useful.
Brian, if you’re open to commenting, get in touch and we’ll share your thoughts in the next issue.
Tip of the Hat
I also asked Jeremy to highlight someone who has been inspiring him recently, and he wanted to call out Amiti Uttarwar’s latest work on rebroadcasting and privacy.
I think it's her first-major contribution and I think it's really cool.
- Jeremy
You can check out one of her contributions to Bitcoin here. Brilliant work, Amiti!
Contribution 2: Ben Jones on Optimistic Rollup
Next, let’s step into the world of Ethereum and Optimistic Rollup, a design pattern that promises to improve the lives of Ethereum users throughout the stack. In addition to supporting more total throughput (potentially resulting in more simultaneous users), these techniques improve the developer and consumer experience for building and interacting with blockchain apps.
In our second commit, the Plasma Group’s Ben Jones shows us Will Meister and Karl Floersch’s work on Optimistic Rollup—a layer 2 scaling solution that works by “optimistically executing” transactions on the slower moving, underlying chain.
The “optimistic” part of the name means that, instead of processing a large number of transactions up front, a cryptoeconomically incentivized party can propose a merkle root of what the resulting transactions should be. In the event that this proposal is incorrect, a fraud proof may be used to dispute the outcome.
Commit 24d0bea
This commit is the key component to executing those disputes.
It allows a client to calculate a new Merkle root for the entire main chain state, without ever putting the entire state on-chain. Instead, only a subset of the state that was touched during the transaction must be put on chain. This is what enables significant scale for Optimistic Rollup. Practically speaking, this means that even after 1,000 valid transactions, a client can prove an invalid state by playing back a single transaction, rather than a full history of 1,001 transactions.
The Interesting Bits
File: RollupMerkleUtils.sol
Lines: 161-181
This code executes a Merkle replacement operation on a specified leaf in a Merkle tree. Effectively, this means computing the new root of a Merkle tree—specifically, the root after an individual leaf in the tree that has been replaced—in log(N) time and space (i.e., without having to process the full Merkle tree). This works because Merkle replacement operations precisely affect the same nodes in a Merkle tree that were used in the original inclusion proof for that node.
What does this mean? It means that given some “pre-state” root, the inclusion of a single leaf can be verified with an inclusion proof, replaced with an entirely new value, and re-hashed with the inclusion proof nodes to generate a new root. Think Merkle verification, but in reverse.
Similar patterns are in development for off-chain usage, both on Ethereum as stateless clients, and Bitcoin as UTreeXO. Succinctly put, Optimistic Rollup contracts can be thought of as implementations of stateless clients on L1 itself.
Managing Trees
Cryptographic hash trees can be difficult to manage; there’s a significant amount of legwork for a relatively simple functionality. The team implemented a Sparse Merkle Tree (SMT) using EVM storage because, even though modifications aren’t persisted between L1 blocks, Solidity doesn’t support in-memory mappings. Eventually an optimized SMT was built for off-chain.
There’s something to be said for avoiding premature optimization, and starting with an optimized SMT could have been the wrong approach—it required a good bit of debugging.
- Ben
Despite their apparent simplicity, Merkle trees are a powerful tool that Ben thinks are great for any new crypto engineer to understand. Here are a few good jumping off points to learn more:
Fantasy Code Review
This call for public comment goes out to Tadge Dryja, because of his L2 knowledge (via Lightning) and his understanding of the overall approach (via UTreeXO).
Tadge, Ben would love your input on Optimistic Rollup as an L2, not necessarily this commit specifically, but the design pattern in general. If you’re up for it, send me an email and I’ll include your comments in the next issue.
Tip of the Hat
I had a great opportunity to improve my understanding of Merkle trees with my friend Luke, one of the creators of Sia. He’s extremely knowledgeable about Merkle tree engineering, [and he’s written] some amazing code which does even more complex, batched Merkle modifications for Sia.
- Ben
Parting Wisdom
In chatting with Jeremy about contributing to Bitcoin core, he shared a thought I’ll leave with you in parting:
When you're making spaghetti you throw a piece against the wall to see if it sticks. When you're trying to change Bitcoin you need be a champion for your work; putting it out there again and again and again until you get it to stick. It can be discouraging, but if you keep on developing out the idea and making it more concrete you can get it done.
- Jeremy
Seems like a good bit of life wisdom ;)
A huge thanks to both Jeremy and Ben for sharing their work in this inaugural issue. And another thank you to my IDEO CoLab Ventures colleagues, Joe, Tara, Dan, and Ian for their support in shipping this.
Want to support Commit Revealed?
Here are some ways you can contribute:
If you enjoyed this first issue, share it with talented engineers, creators. investors, and anyone else who wants to dig in.
If you’re shipping amazing code and want to share it, sign up as a contributor or email me directly for things that are particularly interesting.
If you have feedback or ideas on the format, let me know. I consider this a living prototype.
See you in the next issue.