What Is a Message Queue?

When you place a food order, the app confirms it in a second. But a lot more has to happen: notify the restaurant, assign a delivery partner, send you an SMS, update analytics. If the app did all of that before replying, you'd wait, and one slow step could break the whole order.

A message queue fixes this. The app drops a message ("order #123 placed") into a queue and replies immediately. Other services pick up the message and do their part in the background, at their own pace.

🎯 Quick Answer (30-Second Read)

  • A message queue is a buffer between services: producers add messages, consumers process them later.
  • Benefits: faster responses, services that don't depend on each other being up, and smoothing of traffic spikes.
  • Three models: work queue (each message to one worker), pub/sub (each message to every subscriber), event log (messages kept and replayable).
  • Most systems deliver "at least once," so consumers must handle duplicates safely.
  • Rough guide: SQS for simple managed queues, RabbitMQ for flexible routing, Kafka for high-volume event streams.

Without vs With a Queue

❌ Without a queue
The order service calls the restaurant, SMS and analytics services one by one. If SMS is down, the order fails. If traffic spikes, every service gets hit at once.
✅ With a queue
The order service writes one message and replies. Each service reads it when ready. If SMS is down, messages wait in the queue and are sent when it recovers.
flowchart LR APP["Order service
(producer)"] -->|"order.placed"| Q[["Message queue / broker"]] Q --> R["Restaurant service"] Q --> D["Delivery assignment"] Q --> N["SMS / notifications"] Q --> A["Analytics"]

Why Teams Use Message Queues

  1. Faster user responses: slow work happens after the reply.
  2. Decoupling: the producer doesn't need to know who consumes the message, or whether they're online.
  3. Absorbing spikes: a sale floods the queue, and consumers work through it steadily instead of crashing.
  4. Retries: a failed message can be retried automatically.
  5. Scaling: add more consumers to process faster.

Three Messaging Models

Model How it works Example
Work queue Each message goes to one of many workers Resize uploaded images
Pub/sub Each message goes to every subscriber "Order placed" → email, analytics, loyalty points
Event log (stream) Messages are stored in order; consumers track their own position and can replay Activity tracking, audit trails, data pipelines

How a Message Moves Through a Queue

sequenceDiagram participant P as Producer participant B as Broker participant C as Consumer P->>B: Send message B-->>P: Stored ✓ B->>C: Deliver message Note over C: Process (e.g. send SMS) alt Success C-->>B: Ack → message removed else Crash or timeout B->>C: Redeliver later (retry) end

The acknowledgement (ack) is the key idea. The broker only forgets a message once a consumer confirms it's done. If the consumer crashes first, the message is delivered again.

Delivery Guarantees

At most once
Never duplicated, but can be lost. OK for metrics you can afford to drop.
At least once
Never lost, but can arrive twice. The most common default.
Exactly once
Hard and limited. Usually built as at-least-once plus deduplication.
Watch out: with at-least-once delivery, the same "charge customer" message can arrive twice. Make consumers idempotent: track processed message IDs, or design operations so repeating them changes nothing.

Dead Letter Queues

Some messages fail every time, for example because of bad data. Retrying forever blocks the queue. A dead letter queue (DLQ) collects messages that failed a set number of times so engineers can inspect them while everything else keeps flowing.

Kafka vs RabbitMQ vs SQS

Apache Kafka RabbitMQ Amazon SQS
Model Distributed event log Message broker with flexible routing Fully managed queue
Messages after reading Kept for a retention period, replayable Removed once acknowledged Removed once deleted by consumer
Ordering Within a partition Within a queue FIFO queues available
Best for Very high-volume event streams, data pipelines Task queues, complex routing rules Simple, serverless-friendly queues on AWS
Operations Heaviest to run yourself (managed options exist) Moderate None, AWS runs it

Rule of thumb: need to replay events or feed many systems from one stream → Kafka. Need smart routing or classic task queues → RabbitMQ. On AWS and want zero maintenance → SQS. For small apps, a Redis- or Postgres-backed job queue is often enough.

Frequently Asked Questions

Is Kafka a message queue?

Kafka is usually described as a distributed event streaming platform or commit log. It can be used like a queue, but unlike a traditional queue it keeps messages for a retention period, and each consumer group tracks its own position, so data can be replayed.

When should I not use a message queue?

When the user needs the result immediately, such as a login check or showing a page, a direct request is simpler. Queues add moving parts, eventual consistency and debugging complexity, so use them for work that can happen in the background.

What is the difference between a queue and pub/sub?

In a queue, each message is processed by one consumer, which spreads work across workers. In pub/sub, each message is delivered to every subscriber, so several services can react to the same event independently.

Conclusion

A message queue lets services hand off work instead of waiting on each other. It speeds up responses, absorbs spikes and keeps failures contained. Pick the model (work queue, pub/sub or event log) that fits your problem, design consumers to handle duplicates, and add a dead letter queue for messages that keep failing.

Sources

Last verified: 14 October 2026.

Related reads: Microservices Architecture Explained · Redis Explained · Rate Limiter Explained · WhatsApp System Design

I write about system design and how real products are built. Follow on LinkedIn or X, or invite me to speak.