Instagram System Design: The Problem
Instagram started in 2010 as a simple photo-sharing app. Within months it had millions of users, run by a tiny team. Today it serves billions of people photos, Reels and Stories.
Designing Instagram is a classic system design interview question because it combines three hard problems: storing huge amounts of media, generating personalised feeds fast, and scaling the database beyond a single machine. This post covers all three, including one clever trick Instagram's engineers published: how they generate unique IDs across thousands of database shards.
🎯 Quick Answer (30-Second Read)
- Media and metadata are stored separately: photos/videos in object storage served via a CDN; captions, likes and follows in databases.
- Uploads are processed asynchronously: resizing and transcoding happen in background workers.
- Postgres was sharded into thousands of logical shards spread across fewer physical servers.
- IDs are 64-bit numbers = 41 bits time + 13 bits shard + 10 bits sequence, so they sort by time and reveal their shard.
- Feeds are built by combining precomputed timelines (fan-out on write) with on-demand fetching for celebrities, then ranked.
Requirements
Upload photos and videos, follow users, view a feed, like and comment, view profiles.
Reads far outnumber writes. Feed loads fast. Media never lost. Highly available. Eventual consistency is fine for likes and counts.
High-Level Architecture
photos + videos")] LB --> WEB["App servers
(Django, Python)"] WEB --> CACHE[("Memcached / cache
hot data")] WEB --> PG[("Sharded Postgres
users, posts, follows")] WEB --> CASS[("Cassandra
feeds, activity")] WEB --> Q[["Task queue"]] Q --> W["Workers
resize, transcode,
fan-out to feeds,
notifications"] W --> OBJ W --> CASS
Instagram famously runs its web tier on Django (Python). It has used PostgreSQL for core data and Cassandra for data like feeds and activity, with heavy caching in front.
Part 1: Uploading a Photo
The user gets a response quickly. Resizing, video transcoding and feed updates happen in the background via a queue.
Serving media: images and videos are delivered by a CDN from servers close to the user. The app servers never send image bytes themselves, which keeps them light.
Part 2: Scaling the Database With Sharding
One Postgres server can't hold everything. Instagram's engineers described splitting data into thousands of logical shards (separate schemas) mapped onto far fewer physical database servers. When a server fills up, some logical shards move to a new machine, without changing any IDs.
The ID Trick
Each table needed unique IDs across every shard, IDs that sort by time (for "newest first"), and no central ID service to become a bottleneck. Their solution, generated inside Postgres:
milliseconds since a custom epoch
logical shard ID
sequence (mod 1024)
The math:
- 13 bits → up to 2¹³ = 8,192 logical shards
- 10 bits → 1,024 IDs per shard per millisecond
- 41 bits of milliseconds → 2⁴¹ ms ≈ 69 years before the clock field runs out
Because the timestamp comes first, sorting by ID sorts by time. And any ID tells you which shard holds it, with no lookup needed.
Part 3: Generating the Feed
When you open Instagram, your feed must load in a fraction of a second. Two classic approaches:
| Fan-out on write (push) | Fan-out on read (pull) | |
|---|---|---|
| How | When someone posts, add it to every follower's feed list | When you open the app, fetch recent posts from everyone you follow |
| Feed load | Very fast | Slower |
| Problem | A celebrity with 500M followers = 500M writes per post | Expensive for users who follow thousands |
Real systems use a hybrid: push posts from normal accounts into followers' feeds, but pull posts from celebrity accounts at read time and merge them in. Then the feed is ranked by predicted interest rather than shown purely by time. (See How the Instagram Algorithm Works.)
Part 4: Caching and Counters
- Cache hot data (profiles, popular posts) in memory.
- Like counts on viral posts can get thousands of updates per second. Instead of updating one database row each time, counts are often aggregated and written in batches. A like count being a second out of date is fine.
- Denormalise where reads dominate, such as storing a post's like count with the post.
Frequently Asked Questions
What database does Instagram use?
Instagram's engineers have described using PostgreSQL, sharded into thousands of logical shards, for core data such as users and posts, and Cassandra for data like feeds and activity, with caching layers in front. Media files are kept in object storage and served through CDNs.
How does Instagram generate unique IDs?
Instagram's 2012 engineering post describes 64-bit IDs made of 41 bits of time in milliseconds, 13 bits of logical shard ID and a 10-bit sequence. IDs are generated inside each Postgres shard, sort by time, and reveal which shard holds the data.
How would you design an Instagram feed in an interview?
Describe a hybrid fan-out: push new posts into followers' precomputed feeds for most accounts, pull posts from high-follower accounts at read time, merge, and rank. Mention caching, pagination with cursors, and that media is served from a CDN rather than app servers.
Conclusion
Instagram's design rests on a few ideas: separate media (object storage + CDN) from metadata (databases), process uploads asynchronously, shard Postgres into many logical shards with IDs that encode time and shard, and build feeds with a hybrid push/pull approach before ranking. Each one keeps reads fast as the user count climbs.
Sources
- Instagram Engineering: Sharding & IDs at Instagram (2012)
- Instagram talk at SFPUG: Postgres at Instagram (slides)
Last verified: 14 October 2026.
Related reads: How the Instagram Algorithm Works · How Twitter's Architecture Handles 500 Million Tweets · Consistent Hashing Explained · PostgreSQL vs MySQL
I write about system design and how real products are built. Follow on LinkedIn or X, or invite me to speak.