LOG-003 ·
Consensus, for People Who Do Not Care About Consensus
How thousands of strangers agree on one ledger without a referee, explained with a potluck and no mathematics beyond counting.
- Words
- 669
- Est. read
- 2.9 min
- Confidence
- 0.96
- Topics
- explainer, primitives, networks
Every decentralized system eventually runs into the same problem: how do many machines that do not trust each other agree on one version of the truth? The solutions are called consensus mechanisms, the literature around them is dense, and almost none of that density is required to understand what is actually going on. Today's entry is the explanation I give at dinner parties, written down so I can stop repeating it.
The problem, stated plainly
Imagine a neighborhood potluck ledger. Everyone tracks who brought what, because next year's invitations depend on it. With a centralized ledger, one person (call her the Secretary) keeps the list. Simple, efficient, and fragile in exactly three ways: the Secretary can make a mistake, the Secretary can lie, and the Secretary can move to Florida with the notebook.
The decentralized version: everyone keeps their own complete copy of the list. Now nobody can lose it and nobody can quietly rewrite it. But you have created a new problem. When my copy says the Hendersons brought a casserole and your copy says they brought napkins, whose copy wins?
That question is the entire field of consensus. Everything else is implementation detail.
The three answers that work
Answer one: take a vote. If most copies say casserole, casserole wins. This works beautifully when you know exactly who the voters are, and it is roughly how a category of systems (the Raft and Paxos families) keeps clustered databases agreeing with themselves. The limit is the guest list. Voting requires knowing who is allowed to vote, which means someone maintains a membership list, which means you have reinvented a small Secretary.
Answer two: make lying expensive. Anyone can propose the next page of the ledger, but proposing requires burning real resources (electricity, in proof-of-work) or posting a deposit that misbehavior forfeits (stake, in proof-of-stake). Honest participants earn a reward; cheaters eat a loss. You no longer need a guest list. You need the cheater's cost to exceed the cheater's gain, and that is a number you can measure. Attacking the largest such network today would cost on the order of millions of dollars per hour, sustained. Security became arithmetic.
Answer three: do not agree at all, reconcile later. Some data does not need one global winner. If my grocery list and my wife's grocery list disagree, the merge rule is the union of both, and nobody needed a vote. A class of data structures called CRDTs formalizes this: design the data so any two copies can always merge without conflict. This is how good collaborative editors and local-first apps work, and it is criminally underrated because it lacks a token and a conference circuit.
The table I wish someone had shown me
| Approach | Needs a member list? | Tolerates liars? | Cost | Best at |
|---|---|---|---|---|
| Voting (Raft/Paxos) | Yes | Crashes, not lies | Low | Databases in one organization |
| Proof of work/stake | No | Yes, up to a price | High | Money between strangers |
| Merge rules (CRDTs) | No | Not applicable | Nearly free | Documents, notes, lists |
The engineering sin of the last decade was using answer two for problems that answer one or three solves at a thousandth of the cost. Not everything needs to resist nation-state attackers. My notes app does not have adversaries. It has a sync problem.
Why a non-technical reader should care
Because the question "whose copy wins?" is not a computer question. Your bank balance is a copy dispute resolved permanently in the bank's favor. Your social media reach is a copy dispute resolved by an algorithm you cannot read. Once you see disagreement-resolution as the load-bearing design decision in every system you use, you start asking the only question that matters: who resolves disputes here, and what does it cost them to cheat me?
Systems where that answer is "a specific company, and nothing" are the ones I have been steadily walking away from.
Confidence that this explanation survives contact with an actual distributed-systems engineer: 0.96. The missing 0.04 is for the details I sanded off, and I measured the sanding.