Why One Leader?

Five copies of your database run on five machines. If all five accept writes independently, they'll disagree about the data. If one of them is in charge of ordering every write, the others can simply follow. That one is the leader.

But machines crash and networks split. When the leader disappears, the others must agree on a new one, quickly, and never end up with two leaders at once. That process is leader election, and it sits underneath Kubernetes, Kafka, etcd, MongoDB and most distributed databases.

🎯 Quick Answer (30-Second Read)

  • A leader coordinates work (ordering writes, running a scheduler) so others don't conflict.
  • Election by majority: a node becomes leader only if a majority (quorum) votes for it. Two majorities always overlap, so two leaders can't be elected in the same term.
  • Raft uses terms (numbered election rounds), randomised timeouts and heartbeats.
  • Split brain (two nodes acting as leader) is prevented with quorums, terms and fencing.
  • Don't build your own: use etcd, ZooKeeper, Consul or Kubernetes Lease objects.

How Raft Elects a Leader

Every node is a follower, candidate or leader:

stateDiagram-v2 [*] --> Follower Follower --> Candidate: no heartbeat before
election timeout Candidate --> Leader: wins majority of votes Candidate --> Follower: sees leader with
equal/higher term Candidate --> Candidate: split vote → timeout,
new term, retry Leader --> Follower: sees higher term
  1. Followers expect heartbeats from the leader. If none arrive before a randomised timeout (e.g. 150–300 ms), a follower assumes the leader is gone.
  2. It becomes a candidate, increments the term number, votes for itself and asks others for votes.
  3. Each node votes for at most one candidate per term, and only for candidates whose log is at least as up to date as its own.
  4. A candidate with votes from a majority becomes leader and starts sending heartbeats.
sequenceDiagram participant A as Node A (candidate, term 5) participant B as Node B participant C as Node C participant D as Node D participant E as Node E Note over A: Leader heartbeat missing → timeout A->>B: RequestVote(term 5) A->>C: RequestVote(term 5) A->>D: RequestVote(term 5) A->>E: RequestVote(term 5) B-->>A: Yes C-->>A: Yes D-->>A: No (already voted) Note over A: 3 of 5 (incl. itself) = majority A->>B: Heartbeat: I'm leader for term 5 A->>C: Heartbeat

Why random timeouts? If every follower timed out at the same moment, they'd all become candidates and split the vote forever. Randomness makes one usually go first and win.

Why Majorities Prevent Two Leaders

Key takeaway: in a 5-node cluster, a majority is 3. Any two groups of 3 must share at least one node, and that node can vote only once per term. So two leaders can never both win the same term.
Cluster size Majority needed Failures tolerated
3 2 1
5 3 2
7 4 3

That's why clusters use odd numbers: 4 nodes tolerate only 1 failure, the same as 3.

Split Brain and Network Partitions

flowchart LR subgraph Majority["Side 1: 3 nodes"] A["A"] B["B"] C["C (new leader, term 6)"] end subgraph Minority["Side 2: 2 nodes"] D["D (old leader, term 5)"] E["E"] end Majority -.-x|"network split"| Minority
  • The majority side elects a new leader (term 6) and keeps working.
  • The old leader on the minority side can't get a majority to confirm writes, so its writes never commit.
  • When the network heals, the old leader sees term 6 > term 5 and steps down.

The remaining danger is a slow old leader still acting on external systems (e.g. writing to storage). Fencing tokens (the term number) let those systems reject stale leaders. (See Distributed Locks Explained.)

Leader Election in Practice

Tool How you use it
etcd Built on Raft; its election API and leases let your services elect a leader
ZooKeeper Ephemeral sequential nodes: the lowest sequence number is leader
Kubernetes Lease objects; controllers like the scheduler use them so only one replica is active
Consul Sessions and locks on keys
Your database Postgres advisory locks for simple single-job leadership

Typical use: run 3 replicas of a scheduler for availability, but only the leader actually schedules jobs.

Where You'll See It

  • Kafka: controller election (KRaft mode uses a Raft-based quorum)
  • MongoDB replica sets: elect a primary
  • etcd (and therefore Kubernetes' state store)
  • CockroachDB, TiKV: Raft per data range

Frequently Asked Questions

How does Raft leader election work?

Followers expect regular heartbeats from the leader. If one times out after a randomised interval, it becomes a candidate, increments the term and requests votes. Each node votes once per term for a sufficiently up-to-date candidate, and a candidate with a majority becomes leader.

What is split brain in distributed systems?

Split brain is when two nodes both believe they are the leader, typically after a network partition, which can cause conflicting writes. Requiring a majority quorum, using term numbers and applying fencing tokens prevent split brain from corrupting data.

Why do clusters use an odd number of nodes?

A majority of 3 is 2, tolerating one failure; a majority of 4 is 3, still tolerating only one failure. Adding a fourth node adds cost without extra fault tolerance, so clusters typically use 3, 5 or 7 nodes.

Conclusion

Leader election lets a cluster have exactly one coordinator at a time. Raft does it with terms, randomised timeouts and majority votes, which mathematically prevent two leaders in the same term. In practice, use etcd, ZooKeeper or Kubernetes Leases rather than writing your own, and fence stale leaders with term numbers.

Sources

Last verified: 2 November 2026.

Related reads: Distributed Locks Explained · CAP Theorem Explained · Kubernetes Explained · Write-Ahead Log Explained

I write about system design and how real products are built. Follow on LinkedIn or X, or invite me to speak.