Sweaty Imposter

Design Instagram

System DesignRead HeavyMediaMid3 min readUpdated

Photo uploads are steady, feed reads are enormous, and almost every image byte comes from the CDN. Expect to defend how the feed gets built and where the follow graph lives.

1 · Requirements

The Prompt

Photo upload, feed generation, follow graph, search by hashtag; mostly read-heavy.

Functional

  • Upload a photo with a caption and hashtags
  • Follow other users
  • Load a home feed of followed users' posts, with paging
  • Search posts by hashtag
  • Like posts

Non-Functional

  • Feed loads feel instant at about 700,000 reads a second at peak
  • Images load fast on mobile networks worldwide
  • A new post reaches followers' feeds within seconds

2 · Estimates

Drill This Phase First

Show the Answer and Estimates

Answer

About 290,000 a second

500M times 50 is 25B reads a day. 25B divided by 86,400 seconds is about 290,000 a second, and about 700,000 at peak.

Uploads
About 100M photos a day: roughly 1,200 writes a second and 3,000 at peak
Storage
About 500 KB a photo plus thumbnails: roughly 75 TB a day and 27 PB a year in object storage
Reads
500M daily users loading the feed 50 times a day: 25B reads, about 290,000 a second and 700,000 at peak
Cache
500 post IDs per user at 16 bytes each: about 4 TB across a Redis cluster
Graph
500M users following about 150 accounts each: about 1.2 TB, sharded by follower_id
CDN
The CDN serves about 98% of image bytes, and origin egress is about 1.5 PB a day

3 · API

POST /media (multipart) → { media_id, cdn_url, thumb_url }
POST /posts { media_id, caption, hashtags[] } → { post_id, created_at }
GET /feed?cursor=… → { posts[], next_cursor }
POST /follow { user_id } → 204
GET /search/hashtag/:tag → { posts[], next_cursor }
POST /posts/:id/like → { like_count }

Data Model

posts (wide-column, partition = user_id, cluster = ts DESC):
  post_id (PK)    : snowflake
  user_id         : uuid
  media_ids       : list<uuid>
  caption         : string
  hashtags        : list<string>
  created_at      : timestamp

feeds (Redis, key = "feed:{user_id}"):
  list of post_id : capped at ~500 entries
  ttl             : 7d (rebuild on miss)

follows (wide-column, partition = follower_id):
  follower_id     : uuid
  followee_id     : uuid
  created_at      : timestamp

hashtag_index (Elasticsearch):
  hashtag, post_id, ts, user_id, like_count

4 · High-Level Design

Writes go upload, resize, and then Kafka, which fans the post ID out to followers' cached feeds and the hashtag index. Reads hit the feed API, which returns cached post IDs from Redis and image URLs that point at the CDN.

5 · Deep Dives

Building the Feed

Push each new post ID into followers' cached feeds, capped at about 500 entries. For accounts with huge followings, skip the push and merge their recent posts in at read time. Feeds of users who have been away for a week expire and rebuild on their next visit.

Images at the Edge

Generate a few fixed sizes at upload, store them under content-addressed keys, and serve them from the CDN with long cache lifetimes. A key never changes, so the CDN never needs an invalidation, and the origin sees about 2% of image bytes.

Hashtag Search

Index each post's hashtags in Elasticsearch from the Kafka stream, sorted by recency and likes. Search can trail posting by a few seconds without anyone noticing, which keeps indexing off the upload path.

6 · Wrap Up

End with what you would tackle next: ranking the feed beyond newest first, Stories with a 24 hour expiry, and abuse filtering on uploads.

The Short Version

Images on object storage behind a CDN. Feed via fan-out (push or hybrid). Hashtag search via inverted index.

Building Blocks