Rate Limiter Explained: Algorithms and Examples (2026)

What Is a Rate Limiter?

A rate limiter caps how many requests a user, IP address or API key can make in a period of time, for example 100 requests per minute. Requests over the limit are rejected, usually with HTTP 429 Too Many Requests.

Every serious API has one. It's what stops one buggy script, scraper or attacker from using up capacity meant for everyone. It's also one of the most common system design interview questions.

🎯 Quick Answer (30-Second Read)

• Purpose: protect services from overload and abuse, and keep usage fair between users.
• Response: over the limit → 429 Too Many Requests, often with a Retry-After header.
• Main algorithms: token bucket (allows bursts), leaky bucket (smooth output), fixed window (simple), sliding window (accurate).
• Most popular: token bucket, used widely by API gateways and cloud providers.
• Distributed systems keep counters in a shared store like Redis so every server sees the same count.

Why Rate Limiting Matters

• Stops overload: a traffic spike or retry storm can't take the whole system down.
• Blocks abuse: brute-force logins, scraping and spam get cut off.
• Fairness: one heavy user can't starve everyone else.
• Cost control: caps usage of expensive downstream services, such as paid AI APIs.

The Four Main Algorithms

1. Token bucket

A bucket holds up to N tokens and refills at a steady rate. Each request takes one token. No tokens left means the request is rejected.

• ✅ Allows short bursts (up to the bucket size) while enforcing an average rate
• ✅ Memory-cheap: two numbers per user (tokens left, last refill time)
• Used widely by API gateways and cloud APIs

2. Leaky bucket

Requests enter a queue (the bucket) and are processed at a constant rate, like water leaking from a hole. If the queue is full, new requests are dropped.

• ✅ Very smooth output, which protects fragile downstream systems
• ❌ Doesn't allow bursts, and queued requests wait

3. Fixed window counter

Count requests per fixed window (for example per calendar minute) and reset at the boundary.

• ✅ Simplest to build
• ❌ Boundary problem: a user can send 100 requests at 12:00:59 and another 100 at 12:01:00, so 200 requests land in two seconds.

4. Sliding window

Fixes the boundary problem by looking at the last 60 seconds at any moment.

• Sliding window log: stores a timestamp per request. Exact, but memory-heavy.
• Sliding window counter: blends the current and previous window counts. A good approximation that's cheap.

Which Algorithm Should You Use?

Step-by-Step: A Token Bucket in Python

A minimal, single-server version to understand the idea:

import time

class TokenBucket:
    def __init__(self, capacity: int, refill_per_sec: float):
        self.capacity = capacity
        self.tokens = capacity
        self.refill_per_sec = refill_per_sec
        self.last = time.monotonic()

    def allow(self) -> bool:
        now = time.monotonic()
        # add tokens for the time that passed, up to capacity
        self.tokens = min(self.capacity, self.tokens + (now - self.last) * self.refill_per_sec)
        self.last = now
        if self.tokens >= 1:
            self.tokens -= 1
            return True
        return False

bucket = TokenBucket(capacity=10, refill_per_sec=1)   # bursts of 10, 1 req/s average
print([bucket.allow() for _ in range(12)])            # first 10 True, then False

In production, you'd keep these counters in Redis so all servers share them, and update them atomically (with a Lua script or atomic commands) so two servers can't both spend the last token.

Where Rate Limiters Live

• API gateway / load balancer (most common): limits before traffic reaches your app
• CDN / edge (Cloudflare, AWS WAF): blocks abusive traffic close to the user
• Inside the app: per-feature limits such as "5 password resets per hour"
• Client side: good clients respect Retry-After and back off with jitter

Distributed Rate Limiting Architecture

With one server, a counter in memory works. With ten servers behind a load balancer, each server only sees a tenth of a user's requests, so in-memory limits stop working. The usual fix is a shared counter store, typically Redis:

Design decisions to make:

• Atomic updates: check-and-increment must happen in one step (a Redis INCR, or a Lua script for token buckets), or two servers can both let request number 61 through.
• Fail open or fail closed: if Redis is down, do you allow all traffic (keep the product working) or block it (protect the backend)? Most public APIs fail open.
• Latency: the counter check adds a network hop on every request, so keep Redis close to the gateways.
• Approximate is fine: a few extra requests slipping through at a window edge rarely matters. Exact global counting across regions is expensive.

Good 429 Responses

A helpful rate-limited response tells clients what to do:

HTTP/1.1 429 Too Many Requests
Retry-After: 30
Content-Type: application/json

{"error": "rate_limited", "message": "Too many requests. Try again in 30 seconds."}

Clients should wait for the Retry-After time and use exponential backoff with jitter. Retrying instantly makes overload worse. See Why Apps Crash During High Traffic.

Frequently Asked Questions

Conclusion

A rate limiter protects your system by capping requests per user, IP or key and returning 429 when they go over. Token bucket is the go-to choice because it allows bursts while enforcing an average rate. Use sliding windows when you need accuracy, and keep counters in Redis once you run more than one server.

Sources

• MDN: 429 Too Many Requests
• MDN: Retry-After header
• Redis docs: rate limiting patterns

Last verified: 14 October 2026.

Related reads: Load Balancer Explained · Why Apps Crash During High Traffic · Redis Explained · Microservices Architecture Explained

I write about system design and how real products are built. Follow on LinkedIn or X, or invite me to speak.