By Guy Golan Gueta
After more than two years of internal development, we are excited to introduce Project Concord—an open source distributed trust infrastructure. We are thrilled to join the robust and thriving open source blockchain community and together deepen and broaden a shared blockchain vision focused on true trust decentralization.
Centralized vs. Decentralized Trust Paradigms
Blockchain addresses challenges presented by centralized trust paradigms. In a centralized trust environment, one entity or agent acts as a single point of validation or control. This agent introduces friction (additional cost or time) into transactions and becomes a potential bottleneck or single point of failure and security attack point. A decentralized trust infrastructure eliminates that central authority by “virtualizing” trust and distributing it to participating entities (nodes). A blockchain that relies on decentralized trust infrastructure is typically a permissioned blockchain—one in which known entities (agents or nodes) are invited to participate.
The blockchain forms an unbroken, sequential “ledger” that’s replicated and stored on multiple, independent computers (nodes). Each node follows identical instructions (protocols) that dictate how to update the shared ledger with new transactions and how to determine whether those updates are valid. Many blockchains rely on resource-intensive “proof of work/proof of stake” to validate transactions. In decentralized trust-based infrastructures, consensus protocols validate transactions.
Consensus Protocols and Rogue Generals
In a consensus-protocol system, all parties (nodes) must reach agreement for transactions to be committed. Lacking a swift, single, unified result, the system fails. A functioning system must provide its users both ‘liveness’ (the ability to transact business in a timely fashion) as well as “safety” (protection from faults and false actors). That’s where consensus protocols and specifically Byzantine Fault Tolerance comes into play. (For information on Byzantine Fault Tolerance and the General’s Problem, refer to this wiki page).
When a simple fault occurs—whether a failed node or a rejected transaction—the system must have a mechanism to resolve it and still arrive at a consensus. When a Byzantine fault occurs (“any fault presenting different symptoms to different observers”), special handling is required. In either case, if the system cannot address these faults and assure safety and deliver consensus, the transaction fails and the environment collapses.
Project Concord: Safe, Alive – and Scalable
Project Concord uses Byzantine fault-tolerant consensus protocols to deliver a functioning distributed trust system: one that is both “safe” and “alive.” Concord is a generic state machine replication library that can handle malicious (Byzantine) replicas.
While BFT technology and its application are well-understood, BFT-based systems require substantial communication between nodes and, thus, don’t scale well—an impediment for enterprise-scale blockchain environments. Project Concord solves this problem by simplifying and streamlining communication between nodes, enabling greater scalability while increasing overall network throughput.
Project Concord’s BFT engine obtains significant scaling improvements via three major advances:
- It uses a linear communication consensus protocol—many other BFT consensus protocols (including PBFT) require quadratic communication
- It exploits optimism to provide a common case fast-path execution (like Zyzzyva and with a correct view change protocol)
- It uses modern cryptographic algorithms (BLS threshold signatures)
More details about the BFT consensus protocol can be found in the recent paper, “SBFT: A Scalable Decentralized Trust Infrastructure for Blockchains,” published by a notable group of researchers, including a team from VMware: Guy Golan Gueta, Ittai Abraham and Dahlia Malkhi. In early tests, Project Concord’s engine supported a 200-node environment, far larger than the typical four to eight node systems in place today.
Project Concord’s foundations stem from years of academic and industrial research on Byzantine Fault Tolerant replication, cryptography and distributed computing. The cryptocurrency revolution and, in particular, Bitcoin and Ethereum have also tremendously influenced our understanding of this emerging field of trust decentralization. The Project Concord library is designed to be used as a core building block for replicated distributed data stores and is thus especially suited to serve as the basis for highly scalable, permissioned enterprise Blockchain systems.
In the following months we plan to add many more components to Project Concord; in particular, a generic key-value interface and an execution engine for executing Ethereum virtual machine (EVM) based smart contracts. For more information on EVM and smart contracts, visit https://en.wikipedia.org/wiki/Ethereum or download the Ethereum yellow paper. We welcome the community to contribute and provide feedback. In our next post, we will discuss some of the technical aspects of Concord’s BFT algorithm and why it can scale so well.
Stay tuned to the Open Source Blog for further deep dives into Project Concord and be sure to follow us on Twitter (@vmwopensource) as well!
If only I’d predicted that VMWare would have been among the first companies to see the vast possibilities of the Ethereum blockchain when it was first introduced, I would have held fast to all of my ETH. For now I’ll continue to add value to both VMWare and Ethereum’s smart contract concept by continuing to pursue my training and development in both.
Congratulations thus far. I trust there will be a certification path for this product line.