Why Notifications Are Harder Than They Look

"Your order is out for delivery." "Your OTP is 482913." "India need 12 runs off 6 balls!" Every app sends notifications, and they seem simple: call an API, message sent.

At scale, it's a real system: millions of messages across push, SMS, email and in-app, each channel with its own provider, limits and failures. Users must never get the same alert five times, OTPs must arrive in seconds, and marketing must respect quiet hours and opt-outs. Designing this is a popular system design interview question, and a real problem every growing product hits.

🎯 Quick Answer (30-Second Read)

  • Services don't send notifications directly. They call a central notification service.
  • The service checks user preferences, renders a template, then puts the message on a queue per channel.
  • Channel workers send through providers: APNs (iOS) and FCM (Android/web) for push, SMS gateways, email providers.
  • Priority matters: OTPs and payment alerts get their own fast lane, separate from marketing blasts.
  • Reliability: retries with backoff, deduplication keys, a dead letter queue, and delivery tracking.

Requirements

Functional
Send push, SMS, email and in-app notifications. Templates in multiple languages. User preferences and opt-outs. Scheduled and bulk sends. Delivery status.
Non-functional
OTPs in seconds. Millions of marketing messages per hour. No duplicates. One slow channel must not block the others. Nothing lost.

High-Level Architecture

flowchart TB subgraph Callers OS["Order service"] AUTH["Auth service (OTP)"] MKT["Marketing campaigns"] end OS --> NS AUTH --> NS MKT --> SCH["Scheduler
(campaigns, reminders)"] --> NS NS["Notification service
validate · dedupe · preferences ·
render template · rate limit"] NS <--> PREF[("User preferences
+ device tokens")] NS <--> TPL[("Templates")] NS --> QH[["High-priority queue
OTP, payments"]] NS --> QP[["Push queue"]] NS --> QS[["SMS queue"]] NS --> QE[["Email queue"]] QH -->|"read first"| WP QH -->|"read first"| WS QP --> WP["Push workers"] --> APNS["APNs / FCM"] QS --> WS["SMS workers"] --> SMS["SMS gateway"] QE --> WE["Email workers"] --> EM["Email provider"]

Every worker also writes the result (sent, delivered, failed) to a delivery log.

Component Job
Notification service Single entry point: validate, deduplicate, check preferences, render, route
Preferences + device tokens Which channels each user allows, quiet hours, and push tokens per device
Queues per channel Isolate channels so a slow SMS provider doesn't delay push or email
High-priority queue Keeps OTPs and payment alerts fast during big marketing sends
Workers Call providers, handle rate limits and retries
Delivery log Sent / delivered / opened / failed status for debugging and analytics

How One Notification Flows

sequenceDiagram participant O as Order service participant N as Notification service participant P as Preferences DB participant Q as Push queue participant W as Push worker participant F as FCM / APNs participant U as User's phone O->>N: notify(user 42, "order_shipped", key=order-981-shipped) N->>N: Seen key before? No N->>P: Channels allowed? Device tokens? P-->>N: push ✓, email ✓, tokens [t1, t2] N->>N: Render template (Hindi) N->>Q: Enqueue push for t1, t2 N-->>O: Accepted Q->>W: Message W->>F: Send to t1, t2 F->>U: 🔔 "Your order has shipped" F-->>W: OK / token invalid W->>W: Log status, remove invalid tokens

Push Notifications: APNs and FCM

Your servers can't push directly to a phone. They go through the platform's service:

  • APNs (Apple Push Notification service) for iPhones and other Apple devices
  • FCM (Firebase Cloud Messaging) for Android, and also web and iOS via Firebase

The app registers with the platform and gets a device token, which it sends to your backend. One user can have several tokens (phone, tablet, web). When a provider reports a token as invalid (app uninstalled), delete it so you stop sending to dead devices.

Reliability Essentials

Retries with backoff
Retry temporary provider errors after 1s, 2s, 4s... with random jitter.
Deduplication
Callers pass a unique key (e.g. order-981-shipped). Repeats within a time window are dropped.
Dead letter queue
Messages that keep failing are parked for inspection instead of blocking the queue.
Fallback channels
If push fails for a critical alert, fall back to SMS.
Watch out: queues usually deliver at least once, so a worker can process the same message twice after a crash. Deduplication keys are what stop users getting "Your OTP is..." three times. (See Idempotency Explained.)

Respecting Users

  • Preferences per channel and category: transactional (always), marketing (opt-in)
  • Quiet hours in the user's time zone for non-urgent messages
  • Frequency caps: no more than N marketing pushes per day
  • Unsubscribe links in every marketing email
  • Local rules: for example, in India, commercial SMS must follow TRAI's DLT registration rules for sender IDs and templates

Too many notifications is the fastest way to get your app's notifications turned off, or the app uninstalled.

Scaling Big Sends

When a cricket app pushes "India won!" to tens of millions of users:

  • Fan out in batches: a campaign job splits the audience into chunks and enqueues them.
  • Scale workers horizontally based on queue length.
  • Respect provider rate limits: APNs, FCM, SMS and email providers all throttle. Workers should slow down on rate-limit errors rather than hammer them.
  • Keep transactional traffic separate, so OTPs still arrive in seconds during the blast.

Frequently Asked Questions

How do push notifications work on Android and iOS?

Apps register with Apple Push Notification service (APNs) or Firebase Cloud Messaging (FCM) and receive a device token, which they send to the app's backend. To notify a user, the backend sends the message and token to APNs or FCM, which delivers it to the device.

How do you prevent duplicate notifications?

Callers include a unique deduplication key for each logical notification, such as the order ID plus event type. The notification service stores recently seen keys and drops repeats. Workers should also be idempotent, because queues may redeliver messages after failures.

Why use separate queues for each channel?

Each channel has different speeds, limits and failure modes. Separate queues mean a slow SMS gateway or a huge email campaign can't delay push notifications or one-time passwords. They also let you scale workers per channel independently.

Conclusion

A good notification system centralises sending behind one service, checks preferences and deduplicates first, routes messages into separate queues per channel and priority, and lets workers handle providers, retries and failures. That design keeps OTPs fast, avoids duplicate alerts, and scales to millions of messages without annoying users.

Sources

Last verified: 14 October 2026.

Related reads: Message Queue Explained · Idempotency Explained · Rate Limiter Explained · WhatsApp System Design

I write about system design and how real products are built. Follow on LinkedIn or X, or invite me to speak.