Load Balancer Explained: How It Works (2026 Guide)

What Is a Load Balancer?

One server can only handle so many requests. When traffic grows, you add more servers, but then something has to decide which server handles each request.

That's a load balancer. It sits in front of your servers, accepts every incoming request and forwards it to one of them. If a server dies, the load balancer stops sending it traffic. If traffic doubles, you add servers behind it and users never notice.

Nearly every large website uses one. This guide explains how load balancers work, the main algorithms, and how to choose.

🎯 Quick Answer (30-Second Read)

• What it does: spreads incoming traffic across multiple servers so no single server is overloaded.
• Why it matters: better performance, no single point of failure, and room to scale by adding servers.
• Common algorithms: round robin, least connections, weighted, and IP hash (keeps a user on the same server).
• Two layers: L4 balances by IP and port (fast, simple), L7 reads the HTTP request (routes by URL, headers, cookies).
• Health checks remove broken servers automatically.
• Examples: Nginx, HAProxy, AWS Elastic Load Balancing, Google Cloud Load Balancing, Cloudflare.

How a Load Balancer Handles a Request

The user only ever talks to the load balancer's address. The servers behind it can change at any time.

Load Balancing Algorithms

L4 vs L7 Load Balancing

Load balancers work at two different layers of the network:

Many systems use both: an L4 balancer at the edge in front of L7 proxies such as Nginx or Envoy.

Health Checks and Failover

A load balancer regularly pings each server, for example GET /health every few seconds.

• If a server fails several checks in a row, it's marked unhealthy and gets no traffic.
• When it recovers and passes checks again, it's added back.

This is how a site stays up when individual servers crash. A good /health endpoint checks the things the app really needs (like the database connection), not just "the process is running".

Sticky Sessions: Useful but Risky

Sometimes a user must keep hitting the same server, for example when login sessions are stored in that server's memory. Load balancers support this with sticky sessions (by cookie or IP hash).

The better long-term fix is stateless servers: keep sessions in a shared store like Redis or in signed tokens, so any server can handle any request. Stateless servers scale and fail over far more easily.

Step-by-Step: A Simple Nginx Load Balancer

1. Define a pool of servers (an upstream) and pick an algorithm:

upstream app_servers {
    least_conn;              # remove for round robin (the default)
    server 10.0.0.11:3000;
    server 10.0.0.12:3000;
    server 10.0.0.13:3000 weight=2;   # bigger machine gets more traffic
}

2. Send traffic to the pool:

server {
    listen 80;
    location / {
        proxy_pass http://app_servers;
    }
}

3. Reload Nginx (nginx -s reload) and watch the traffic spread across servers.

Open-source Nginx removes a server that fails requests (using max_fails and fail_timeout on each server line). Active health checks that probe a URL are part of NGINX Plus or other proxies like HAProxy.

Managed Load Balancers

If you run on a cloud, you rarely build this yourself:

Production Architecture: Load Balancers in Layers

Big sites don't use one load balancer. They stack them, each layer spreading traffic a little further:

1. Global layer (DNS or anycast): picks the closest healthy region.
2. L4 layer: handles raw connections very fast, run as a redundant pair so it isn't a single point of failure.
3. L7 layer: reads the request and routes by path (/api vs /images), terminates HTTPS and applies rules like rate limits.
4. Servers: stateless, so any of them can handle any request.

If a whole region fails, the global layer stops sending traffic there. That's how large services survive a data-centre outage.

Limitations and Trade-offs

• The load balancer can become the single point of failure. Production setups run it redundantly.
• It adds a network hop, usually small, but measurable at very low latency targets.
• It doesn't fix a slow database. If every server waits on the same database, adding servers won't help.
• Sticky sessions reduce flexibility. Prefer stateless servers.

Real-World Example: Live Cricket Streaming

When a big cricket match starts, millions of people open the same app within minutes. Hotstar's engineers have described adding capacity before the match and scaling on the number of viewers rather than CPU, because traffic can grow by over a million users per minute. Load balancers spread that flood across the pre-warmed servers. Read the full story: How Hotstar Handles Crores of Live Viewers.

Frequently Asked Questions

Conclusion

A load balancer spreads traffic across servers, removes unhealthy ones automatically and lets you scale by adding machines. Start with round robin on stateless servers, use least connections for uneven workloads, and choose L7 when you need to route by URL or headers.

Sources

• NGINX docs: HTTP load balancing
• AWS: What is load balancing?
• Gaurav Kamboj (Hotstar), Scaling hotstar.com for 25 million concurrent viewers, AWS re:Invent 2019

Last verified: 14 October 2026.

Related reads: Why Apps Crash During High Traffic · Redis Explained · Microservices Architecture Explained · Kubernetes Explained

I write about system design and how real products are built. Follow on LinkedIn or X, or invite me to speak.