Stories look like a photo feed with a countdown, and people design them like one. That is the trap. A story is not a permanent post you happen to hide after a day - the 24-hour lifetime, the ordering, and the read patterns are all fundamentally different from a feed, and they push the design in directions Instagram’s main feed never has to go.

Three things make Stories their own animal. First, everything is ephemeral: a story exists for exactly 24 hours and then it is gone, so the entire data model is built around a TTL and auto-expiry rather than durable storage. Second, ordering is not ranked, it is per-viewer state: the story tray at the top of the app is sorted by “who has stuff I have not seen yet,” which means the sort key is a function of me, the viewer, not a global score. Third, the read amplification is brutal: 5B story views a day, most of which are cheap “has anything new happened” tray checks that must return in tens of milliseconds. The media bytes are the easy part (that is a solved CDN problem, same as any photo/video service); the hard part is the seen/unseen state and the tray, so that is where I spend the depth.

The decision that colors everything: per-viewer seen state and the tray are a cache problem, not a storage problem. Stories die in 24 hours, so the entire working set is small and transient. Treat it as durable and you will over-engineer storage while under-engineering the state that actually gets hammered 5B times a day.

Functional Requirements (FR)

In scope:

  • Post a story. A user uploads a photo or short video. It becomes visible to their followers (or friends) for exactly 24 hours, then auto-deletes.
  • View the story tray. When a user opens the app, show a horizontal tray of accounts that have active (unexpired) stories, ordered so unseen stories come first.
  • Watch a story. Tap an account in the tray, play its stories oldest-to-newest in chronological order, then advance to the next account. Mark each as seen for this viewer.
  • Seen / unseen ordering. The tray must reflect what this specific viewer has and has not watched. An account with unseen stories sorts ahead of one whose stories the viewer already watched.
  • Viewer list (for the author). The author can see who viewed each of their stories - the list of viewer IDs, newest first.
  • Auto-expiry. After 24 hours a story disappears from every tray, cannot be watched, and its bytes are reclaimed.

Explicitly out of scope, said out loud so I control the scope:

  • Direct messages, replies to stories, reactions, and stickers/polls. Replies are just DMs triggered from a story; I will note where the hook is but not design messaging.
  • Story Highlights (pinning a story permanently to your profile). That is the opposite of ephemeral - it is a durable copy - and it is a small separate feature, not the core.
  • Ranking / ML ordering of the tray beyond seen-first + recency. Instagram does rank the tray with a model; I treat that as a thin read-time layer over the seen-state sort, same as any feed.
  • Ads in stories, close friends lists, content moderation. Moderation runs async off the upload pipeline; I am not designing classifiers.
  • The video transcoding pipeline internals. Stories are short clips and images; the upload/transcode/CDN path is the same shape as any media service and I reuse it rather than re-derive it.

The single decision that drives the architecture: this is a read-heavy, state-heavy, short-lived workload. The dominant operation is not “fetch media,” it is “given viewer V, which of the accounts V follows have stories V has not seen.” That is a per-viewer computation over a tiny, constantly-expiring dataset, and getting it fast at 5B reads/day is the whole game.

Non-Functional Requirements (NFR)

  • Scale: 500M DAU. 5B story views/day. Assume ~50M stories posted/day (a small fraction of users post; most only watch). Each story lives 24 hours, so the entire active corpus is only ever ~1 day of stories.
  • Latency: tray load p99 under 150ms (it is the first thing the user sees on app open). Story media playback start under 100ms from CDN edge. Marking a story seen is fire-and-forget from the client’s view - it must not block playback.
  • Availability: 99.99% on the read paths (tray + playback). A broken tray is the first thing every user sees. Posting can tolerate slightly lower availability (retryable from client).
  • Consistency: eventual everywhere, with one soft guarantee - read-your-own-seen-state within a session. If I watch a story and swipe back, it should show as seen. Across devices a few seconds of lag is fine. The tray can lag reality by a few seconds (a story I post reaches followers’ trays within seconds, not instantly).
  • Durability: deliberately low. Stories are ephemeral by design. Losing an in-flight seen-marker is acceptable (worst case a story I watched shows as unseen once). The media itself should not be lost during its 24h window, but after expiry we want it gone. This is the rare system where low durability is a feature.
  • Freshness / expiry: a story must vanish from every tray and become unplayable within a small window (seconds) of its 24h mark. No zombie stories.

Back-of-the-Envelope Estimation (BoE)

Real numbers with the arithmetic, because the read/write asymmetry and the tiny corpus size dictate the design.

Writes (posting stories):

50M stories posted / day
= 50,000,000 / 86,400 sec
≈ 580 posts/sec average
Peak (3x) ≈ 1,750 posts/sec

Posting is tiny. This is not a write-heavy system on the post path.

Reads - two very different kinds. The 5B/day splits into cheap tray checks and heavier story watches:

Tray loads: 500M DAU open the app ~10x/day (tray check each time)
  500M * 10 = 5B tray loads/day
  = 5B / 86,400 ≈ 58,000 tray loads/sec average
  Peak (3x) ≈ 175,000 tray loads/sec

Story watches (the "5B views" headline): each opened story = 1 view + 1 seen-write
  5B views/day = 5B / 86,400 ≈ 58,000 story views/sec average
  Peak (3x) ≈ 175,000 story views/sec

So ~58K tray loads/sec and ~58K story-view+seen-writes/sec at average, each tripling at peak. The seen-writes are the sneaky part: 5B seen-markers written per day is a bigger write load than the 50M posts by a factor of 100.

Active corpus size - the number that shapes everything. Stories live 24h, so at any instant we hold only ~1 day of them:

50M active stories at any time.
Metadata per story: story_id(8) + author_id(8) + media_key(~40)
                  + type/dims/ts/expiry(~40) ≈ ~100 bytes
50M * 100 B ≈ 5 GB of story metadata, total, ever.

Five gigabytes. The entire active story metadata set fits in RAM on a single beefy cache node (we will still shard it, for QPS not size). This is the opposite of the 73 PB/year a permanent photo service holds. Ephemerality shrinks the corpus to nothing.

Seen-state size - the real state problem. This is per (viewer, story) pair, and it is where the volume hides:

Seen edges are the number of (viewer, story) pairs recorded per day.
5B views/day = 5B seen edges/day, but they expire with the story (24h).
At steady state we hold ~1 day of seen edges = ~5B edges.
Per edge: story_id(8) + viewer_id(8) + ts(8) ≈ 24 B, call it ~50 B with overhead.
5B * 50 B ≈ 250 GB of seen-state at any time.

250 GB of transient seen-state, all of which expires within 24h. That is a cache-cluster-sized number, not a database-sized one, and it self-cleans via TTL. Design decision made for us: seen-state lives in a TTL’d cache/KV store, never in a relational DB.

Media storage (the easy part):

Avg story media (image or short video, transcoded) ≈ 2 MB.
50M * 2 MB ≈ 100 TB/day of new media...
...but it all expires in 24h, so steady-state footprint ≈ 100 TB total.

100 TB, flat, forever - because every day’s expiry roughly cancels the new day’s uploads. A permanent service would be at petabytes/year; ephemerality caps us at ~1 day of media. Object store with an aggressive 24-25h lifecycle policy handles it.

Bandwidth (egress) - the CDN’s job:

58K story views/sec * 2 MB avg ≈ 116 GB/sec average
Peak ≈ 175K/sec * 2 MB ≈ 350 GB/sec

Hundreds of GB/sec, served entirely from the CDN edge, same as any media service. This is real but it is a solved problem; I will not spend the design’s depth here.

Cache size for trays: we do not materialize a per-viewer tray; we cache the author-side “who has active stories” and compute the viewer sort at read time. The author index (author -> list of active story IDs) is:

~5M distinct authors with active stories at any time.
Per author: author_id(8) + ~5 story_ids(40) ≈ 50 B
5M * 50 B ≈ 250 MB. Trivial.

The heavy cache is the 250 GB of seen-state; everything else is small.

High-Level Design (HLD)

Three paths, and unlike a permanent feed they are dominated by the tray + seen-state path, not media. The post path is low-QPS. The media playback path is high-bandwidth but a solved CDN problem. The tray/seen path is the hard, high-QPS, latency-critical core.

                          +---------------------------+
                          |          Clients          |
                          |   (mobile, web)           |
                          +-------------+-------------+
                                        |
                          +-------------v-------------+
                          |       Load Balancer        |
                          +--+---------+----------+---+-+
              POST path    |           | TRAY path |   VIEW/SEEN path
          +----------------v-+   +------v-------+  +--v------------------+
          |  Story Service    |   | Tray Service |  |  View Service        |
          | (presign,confirm) |   | (stateless)  |  |  (record seen)       |
          +--+-------------+--+   +--+-----------+  +--+----------------+--+
             |             |         | 1.get followees   | mark seen      |
   1.presign |    3.confirm|         |   with active     |                |
             |     (meta)  |         |   stories         |                |
   +---------v---+  +------v----+  +--v-------------+  +-v--------------+  |
   | Object Store|  | Story Meta |  | Author-Story   |  | Seen-State     |  |
   | (media, 25h |  | (KV, TTL   |  | Index (KV:     |  | Store (KV:     |  |
   |  lifecycle) |  |  24h)      |  |  author->[sid])|  |  viewer:sid    |  |
   +------+------+  +-----+------+  +--+-------------+  |  -> ts, TTL24h)|  |
          |               |            |                +-------+--------+  |
          | object        | emit       | 2.filter unseen        |          |
          | created       | index-add  |   per viewer           |          |
   +------v------+   +-----v-------+  +-v--------------+         |          |
   | (nothing;   |   | Index Writer|  | Social Graph    |        |          |
   |  media      |   | (author idx)|  | Service         |        |          |
   |  served by  |   +-------------+  | (followees /    |        |          |
   |  CDN)       |                    |  friends)       |  +-----v-------+  |
   +-------------+                    +-----------------+  | Viewer-List  |  |
                                                           | Store (KV:   |  |
   +----------------+   media GETs   +----------------+    | sid->[viewer]|  |
   |  Object Store   | <-- origin --- |      CDN        |   |  TTL 24h)    |  |
   +----------------+     fill        +----------------+   +--------------+  |
          ^                                  ^                               |
          | playback                         | clients watch stories        |
          +----------------------------------+-------------------------------+

Post flow (upload a story):

  1. Client calls POST /stories/upload-url. Story Service authenticates, assigns a story_id (Snowflake, time-sortable, encodes creation time), returns a presigned PUT URL into the object store plus the story_id.
  2. Client uploads the media directly to the object store (bytes never touch the app tier). Video is transcoded async, same pipeline as any media service.
  3. Client calls POST /stories/{id}/confirm. Story Service writes the story metadata row into the Story Meta KV with a 24h TTL and computes expires_at = created_at + 24h.
  4. Story Service (or an index writer off the object-created event) appends story_id to the Author-Story Index entry for this author (author -> [story_ids]), also TTL’d. The story is now discoverable.
  5. No fan-out to followers on post. We do not push into 100M follower trays (see deep dive). The tray is computed at read time.

Tray read flow (the hot path - open the app):

  1. Client GET /tray. Tray Service fetches the viewer’s followee list from the Social Graph Service (cached).
  2. For those followees, batch-read the Author-Story Index to find which have active (unexpired) stories. This is a small set - only ~5M authors globally have active stories, and a given viewer follows a few hundred, so typically tens of authors with active stories.
  3. For each such author, check the viewer’s seen-state (batched read against the Seen-State Store) to determine has_unseen.
  4. Sort: authors with unseen stories first (by recency), then fully-seen authors. Return the tray as a list of author entries with an unseen flag and story IDs. Media URLs are CDN links resolved per story.

Watch + seen flow (per story opened):

  1. Client requests the story media URL (a CDN URL); CDN edge serves the bytes (>95% hit rate, immutable media).
  2. Client fires POST /stories/{id}/seen (fire-and-forget). View Service writes a (viewer, story_id) -> ts entry into the Seen-State Store with a 24h TTL, and appends viewer_id to the story’s Viewer-List (story_id -> [viewer_ids], also 24h TTL) for the author’s viewer list.
  3. Playback is never blocked by the seen-write; it is async and idempotent (writing the same seen edge twice is a no-op overwrite).

Expiry flow (the defining behavior):

  1. Every store holding story data (Story Meta, Author-Story Index, Seen-State, Viewer-List, and the object store) is written with a 24h TTL at creation time. Expiry is passive - entries simply vanish; no scheduled sweeper is on the critical path.
  2. A story past its expires_at is filtered out at read time even in the tiny window before TTL eviction actually reclaims it, so there is never a zombie story.

The whole point of the HLD: ephemerality plus TTLs turns storage into a self-cleaning cache, and the only genuinely hard problem is computing per-viewer unseen state fast. The app tier moves kilobytes; the media is a CDN’s problem; the state is small and self-expiring.

Component Deep Dive

The hard parts, naive-first then evolved: (1) per-viewer seen/unseen state and tray ordering, (2) the tray computation - fan-out on read vs write, (3) auto-expiry and TTL correctness, (4) the viewer list and the 5B seen-writes/day.

1. Per-viewer seen/unseen state and tray ordering

This is the part that is genuinely different from a permanent feed, so it gets the most room. In a normal feed the sort key is a global score. Here the sort key - “have I seen this?” - is a function of the viewer, so it cannot be precomputed once for everyone.

Naive approach: a seen boolean column, or a per-user story_status table in SQL

The obvious first cut: a relational table story_views(viewer_id, story_id, seen_at) with a composite primary key, and on tray load you JOIN the viewer’s followees’ active stories against this table to figure out what is unseen.

Where it breaks, on every axis:

  • Write volume destroys the DB. 5B seen-markers/day = ~58K inserts/sec average, ~175K peak, into one table. That is a firehose of tiny writes with a two-column key. A relational store’s index maintenance, WAL, and vacuum/compaction cannot keep up, and none of these rows matter after 24h - you are paying durable-write cost for disposable data.
  • The JOIN is per-viewer and unbounded. Every tray load (58K-175K/sec) runs a join between “active stories of accounts I follow” and “stories I have seen.” Even with indexes this is a scatter across the viewer’s followees plus a lookup per candidate story, recomputed on every app open. p99 is uncontrollable.
  • Expiry is a delete storm. 5B rows/day must be deleted after 24h. Batch DELETE jobs over billions of rows thrash the DB and fight the live insert load. You are now running a garbage collector inside your primary datastore.

Everything wrong here comes from treating disposable, per-viewer state as durable relational data.

Evolved approach: seen-state as a TTL’d KV set, tray sort computed at read

Store seen-state in a horizontally sharded KV store (Redis/Cassandra-with-TTL/DynamoDB-with-TTL) keyed for the exact read we need, and let the TTL do expiry for free.

Key design for seen-state. We need to answer two questions cheaply: “has viewer V seen story S?” (per-story mark during playback) and “which of author A’s active stories has V not seen?” (tray sort). The clean structure is a per-viewer set of seen story IDs, or equivalently a per-(viewer,author) high-water mark:

Option A - per-story edges (precise viewer lists too):
  key:  seen:{viewer_id}:{story_id}   value: seen_ts   TTL: 24h
  "has V seen S?"  -> point GET (single key)

Option B - per-author high-water mark (cheaper for the tray):
  key:  seen:{viewer_id}:{author_id}  value: last_seen_story_id (or ts)  TTL: 24h
  "unseen from author A?" -> compare A's newest active story_id
                            to V's high-water mark for A. One GET.

Since story IDs are Snowflake (monotonic in time), Option B’s high-water mark is a clean trick: a viewer has unseen stories from author A iff A’s newest active story_id is greater than V’s last-seen story_id for A. One comparison, no per-story scan. Use Option B for the tray sort (the hot 58K/sec read) and keep Option A only where you need exact per-story granularity.

Tray sort at read time:

on load_tray(viewer):
    followees = graph.followees(viewer)                 # cached, few hundred
    active = author_index.mget(followees)               # authors with active stories
    marks  = seen_hwm.mget([(viewer, a) for a in active_authors])  # batched
    tray = []
    for a in active_authors:
        newest = max(active[a].story_ids)               # newest active story
        unseen = (marks.get((viewer,a)) is None) or (newest > marks[(viewer,a)])
        tray.append({author: a, unseen: unseen, newest_ts: id_to_ts(newest)})
    # seen-first ordering: unseen bucket (by recency) then seen bucket
    return sort(tray, key=lambda x: (not x.unseen, -x.newest_ts))

Two batched KV reads (author index + seen high-water marks) and an in-memory sort over tens of authors. No JOIN, no scan, bounded by followee count. p99 stays flat regardless of how much anyone has posted.

Marking seen during playback is a single SET seen:{viewer}:{author} = story_id (only advancing the high-water mark) plus, if you want exact viewer lists, an append to the story’s viewer list. Both are idempotent, fire-and-forget, TTL’d.

2. The tray computation - fan-out on read vs fan-out on write

A permanent feed fans out on write (push post IDs into follower feeds). Should the tray do the same? This is the key architectural fork.

Naive approach: fan-out on write, like a feed

On post, push the story_id into every follower’s “story tray” list. Tray load becomes a single read of the viewer’s precomputed tray.

Where it breaks:

  • Ephemerality wrecks the economics of fan-out. In a feed, the write you fan out is durable - you pay the fan-out cost once and read it for years. A story is read for at most 24h, so you pay a huge write-amplification cost for a tiny read lifetime. A celebrity with 100M followers posting a story = 100M tray writes, and every one of those entries is garbage in 24 hours.
  • Seen-state does not fit the precomputed tray anyway. Even if you fanned out story IDs, the ordering (seen-first) is per-viewer and changes every time the viewer watches something. You would have to re-sort each precomputed tray on every seen-event, which is another write storm. The expensive fanned-out list does not even answer the question the tray asks.
  • The active set is tiny and cheap to gather. Only ~5M authors have active stories at any instant; a viewer follows a few hundred, so gathering “which of my followees have active stories” is a small batched read. There is nothing to precompute.

Fan-out on write is the right call for a durable, ranked-by-global-score feed. It is the wrong call for ephemeral, per-viewer-sorted content.

Evolved approach: fan-out on read from a per-author index, hybridized for the tail

Keep an Author-Story Index (author -> [active story_ids], TTL’d) and compute the tray on read by gathering the viewer’s followees’ entries. This is fan-out on read, and it is correct here because the corpus is tiny and the sort is per-viewer.

Where naive fan-out-on-read strains: a viewer who follows tens of thousands of accounts (rare, but real) means a large batched read of the author index per tray load. And a hyper-active viewer opening the app constantly re-runs this.

The fixes (hybrid):

  • Short-TTL tray cache per viewer. Cache the computed tray (author list + unseen flags) for a few seconds per viewer. App opens cluster in bursts; a 5-10s cache absorbs pull-to-refresh spam without staleness anyone notices. Invalidate lazily; it expires fast anyway.
  • Author index is cheap and shared. author -> [story_ids] is one entry per author, read by all their followers. The 5M-author index is ~250 MB and trivially cacheable; a celebrity’s entry is one hot key read by millions, handled by hot-key replication (below), not by fanning their story into millions of trays.
  • Bound the followee scan. For accounts following huge numbers of people, cap the tray candidate gather at the most-recently-active N followees; the tray only shows a limited horizontal strip anyway.
StrategyPost costTray-read costBreaks onVerdict
Fan-out on write (feed-style)O(followers) writes/post, all disposable in 24hO(1) list read, but needs per-seen re-sortCelebrities, ephemerality, per-viewer sortWrong model for stories
Fan-out on read (naive)O(1) index appendO(followees) batched read + sortUsers following tens of thousandsRight shape, tail strains
Fan-out on read + tray cache + hot-key (hybrid)O(1) index appendO(active followees), cached few secNothing at scaleThe answer

The reason this inverts the feed’s answer: fan-out on write amortizes a one-time write across a long read life. Stories have almost no read life, so there is nothing to amortize, and the per-viewer seen-sort cannot be precomputed anyway. Read-time computation over a tiny active set wins.

3. Auto-expiry and the 24-hour TTL

Auto-delete after 24h is the product’s defining behavior, and doing it as a scheduled batch job is the classic wrong answer.

Naive approach: a cron job that scans and deletes expired stories

A scheduler runs every few minutes, scans stores for stories older than 24h, and issues deletes across metadata, seen-state, viewer lists, and object storage.

Where it breaks:

  • Delete-storm load. 50M stories/day and 5B seen edges/day expiring means the cron is constantly issuing tens of thousands of deletes/sec, competing with live traffic on the same shards. The GC job becomes a second workload as large as the primary one.
  • Timeliness vs cost tension. Run it often and it hammers the cluster; run it rarely and stories linger past 24h (zombie stories - a correctness and privacy bug, since ephemerality is a promise to the user).
  • Multi-store consistency. A story’s data lives in five places. A cron deleting them in sequence can leave a story gone from metadata but still in someone’s viewer list, or vice versa.

Evolved approach: native TTL at write time + read-time expiry filter

Do not delete on a schedule. Write every piece of story data with a native TTL of 24h (media 25h, a small buffer) at creation. The datastore reclaims it lazily and for free - Redis key expiry, Cassandra/DynamoDB TTL, S3 lifecycle policy. Expiry becomes a property of the data, not a job.

POST /stories/{id}/confirm:
  story_meta.set(story_id, {...}, EX=86400)            # 24h TTL
  author_index.append(author_id, story_id, EX=86400)
  # object already uploaded; lifecycle rule deletes raw+renditions at 25h

seen-state and viewer-list writes:
  seen_hwm.set("seen:{viewer}:{author}", story_id, EX=86400)
  viewer_list.append(story_id, viewer_id, EX=86400)

Two things make this correct:

  • Read-time expiry filter closes the eviction gap. TTL eviction is lazy - a key can physically linger a little past its expiry. So every read also checks now < expires_at and filters. The story is logically gone the instant it expires; physical reclamation happens whenever the store gets to it. No zombie is ever served.
  • Independent TTLs, no coordination needed. Because each store expires its own copy on its own timer, there is no cross-store delete transaction to get wrong. They all converge to empty within the same window. The tray/read filter guarantees a consistent user-visible expiry even if the five stores reclaim at slightly different physical times.

Media expiry: the object store lifecycle policy deletes story media at 25h. The extra hour covers clock skew and any late in-flight playback of a story watched right at the boundary; the read filter still refuses to start playback of an expired story. This is the one place we accept a store holding data slightly longer than 24h, and it is invisible because reads gate on expires_at, not on physical presence.

TTL-at-write plus read-time filtering turns “auto-delete after 24h” from a giant recurring batch job into a no-op the datastore does for us.

4. The viewer list and 5B seen-writes/day

The author wants to see who viewed each story - and generating that list is where the 5B/day write load actually lands.

Naive approach: append every viewer to a per-story list, read it back sorted

On each view, append viewer_id to story:{id}:viewers. When the author opens the viewer list, read and sort.

Where it breaks:

  • Hot-key write contention on popular stories. A celebrity story gets millions of views in minutes - millions of appends to one key. That single key becomes a write hot spot that no single shard can absorb.
  • Unbounded list, but capped display. A story with 50M views produces a 50M-entry list, but the UI shows a bounded, paginated view-count-plus-recent-viewers. Storing and sorting all 50M synchronously is wasted work.
  • Double-write coupling with seen-state. Every view writes both the seen high-water mark and the viewer-list append; naive coupling means one firehose becomes two.

Evolved approach: async buffered writes, sharded counters, capped lists

  • Seen-mark and viewer-list append are async and decoupled from playback. The client fires seen after the media is already playing from the CDN; the write path is a best-effort queue. Losing one is acceptable (low durability is in the NFRs).
  • Split the count from the list. The view count is a sharded/approximate counter (increment one of N counter shards per story, sum on read), so millions of increments spread across shards instead of hammering one key. The viewer list itself only needs to retain a bounded window (e.g. most recent 1-2k viewers) for display; older entries are trimmed. Exact full lists are not a product requirement and would be pure cost.
  • Buffer and batch. For hot stories, a per-node local buffer coalesces many appends/increments into periodic flushes, collapsing millions of tiny writes into far fewer batched ones. This is the same shape as any high-write counter (video view counts, likes).
  • Idempotency via the high-water mark. Because the seen mark is a per-(viewer,author) high-water mark (monotonic story ID), re-sending a seen event is a harmless no-op max() update - so at-least-once delivery from the client is safe.
on view(story_id, viewer_id):
    enqueue seen_event{viewer_id, author_id, story_id}     # async
  worker (batched):
    seen_hwm.max("seen:{viewer}:{author}", story_id, EX=86400)  # idempotent
    view_counter.incr(shard_of(story_id, viewer_id), 1)         # sharded count
    viewer_list.append_capped(story_id, viewer_id, cap=2000, EX=24h)

The 5B/day is absorbed by making it async, sharded, capped, and idempotent - never a synchronous durable write, because none of this data outlives the day.

API Design & Data Schema

API

POST /api/v1/stories/upload-url
Response 200:
{
  "story_id": "1793244112233000",
  "upload_url": "https://blob.cdn.example/raw/1793...?X-Signature=...",  // presigned PUT
  "expires_in": 300
}

PUT  <upload_url>              # client uploads media DIRECTLY to object store
Body: <binary image/video>    # never passes through the app tier

POST /api/v1/stories/{id}/confirm
Body: { "type": "image", "duration_ms": 5000 }
Response 202:
{ "story_id": "1793244112233000", "expires_at": "2026-08-01T00:00:00Z" }  // created_at + 24h
Errors: 400 blob missing / invalid media | 401 not auth | 409 already confirmed

GET /api/v1/tray
Response 200:
{
  "tray": [
    {
      "author": { "id": "42", "handle": "alice" },
      "unseen": true,
      "newest_ts": "2026-07-31T09:12:00Z",
      "stories": [ { "story_id": "1793...", "type": "image",
                     "url": "https://img.cdn.example/1793.../v.jpg",
                     "expires_at": "2026-08-01T09:12:00Z" } ]
    }
  ]
}
Ordering: unseen authors first (by recency), then seen authors.
Latency target: p99 < 150ms (metadata only; media fetched from CDN).

POST /api/v1/stories/{id}/seen        # fire-and-forget, async
Body: { }                             # viewer inferred from auth
Response 202 Accepted                 # never blocks playback; idempotent

GET  /api/v1/stories/{id}/viewers?limit=50&cursor=..   # author only
Response 200:
{ "count": 1284302,                   # sharded approximate counter
  "viewers": [ { "id": "77", "handle": "bob", "seen_at": "..." } ],
  "next_cursor": "<opaque>" }         # only recent capped window retained

Pagination is cursor-based, not offset - viewer lists change under you as views stream in. The cursor encodes the last-seen viewer position.

Data store choices

Access patterns differ per store; be explicit about each and its shard key. Note the theme: everything is a TTL’d KV store, nothing is relational, because the data is disposable, partition-scoped, and needs write throughput over transactions.

1. Object Store (media) - not a database. Story media keyed {story_id}/v.jpg (or .mp4), immutable, 25h lifecycle expiry, fronted by the CDN. ~100 TB steady state. Same shape as any media service.

2. Story Meta - KV with TTL (Redis / DynamoDB / Cassandra). Point-lookup by story_id. ~5 GB total. TTL 24h.

Table/keyspace: story_meta
  story_id     BIGINT   PARTITION KEY   -- Snowflake, encodes created_at
  author_id    BIGINT
  media_key    STRING                   -- {story_id}/v.ext
  type         ENUM     -- image | video
  duration_ms  INT
  created_at   TIMESTAMP                -- embedded in story_id
  expires_at   TIMESTAMP                -- created_at + 24h (read-time filter)
  TTL:  86400 sec
  Shard by: story_id  (uniform; single-key reads land on one shard)

3. Author-Story Index - KV with TTL. author -> [active story_ids], read on every tray load to find who has active stories.

Key:   author:{author_id}:stories
Value: sorted set / list of story_ids (newest last)
TTL:   86400 sec (per entry; individual story IDs also age out via read filter)
Shard by: author_id
Read pattern: MGET over a viewer's followees -> active authors

4. Seen-State Store - KV with TTL (the 250 GB hot core). Per-(viewer,author) high-water mark for the tray sort.

Key:   seen:{viewer_id}:{author_id}
Value: last_seen_story_id   (monotonic; unseen iff newest_active > this)
TTL:   86400 sec
Shard by: viewer_id   (so a viewer's whole seen-set colocates -> one batched read)

5. Viewer-List + View-Counter - KV with TTL. story_id -> capped recent viewer list and a sharded counter for the count.

Key:   viewers:{story_id}       -> capped list (recent ~2000), TTL 24h
Key:   viewcount:{story_id}:{shard}  -> integer, TTL 24h, summed on read
Shard by: story_id (list); counter fanned across N sub-shards to kill hot key

Why NoSQL/KV over SQL across the board: the workload is disposable, partition-scoped, write-heavy (5B seen-writes/day), and needs no cross-entity ACID transactions or joins - the exact features SQL sells and we do not want. Crucially, native TTL does our expiry for free, which a relational store would force into a delete-storm cron. The only durable relational data in the wider product (user accounts, the follow graph’s source of truth) lives in a separate small store and is unrelated to the ephemeral story hot path; we read a cached copy of the graph, never join against it live.

Bottlenecks & Scaling

Where it breaks first, in order, and the fix for each:

1. Seen-state write volume (breaks first - 5B writes/day, ~175K/sec peak). This is the real load, 100x the post rate. Fix: async, best-effort, idempotent high-water-mark writes into a TTL’d KV store sharded by viewer_id; buffer and batch on hot stories; accept low durability (a lost seen-marker just re-shows one story once). Never a synchronous durable write.

2. Tray-read QPS and the per-viewer sort (58K-175K/sec). The sort key is per-viewer, so it cannot be globally precomputed. Fix: fan-out on read over the tiny active-author set, using a per-author high-water-mark comparison (Snowflake IDs make “unseen?” a single integer compare). Two batched KV reads + in-memory sort over tens of authors. Short-TTL per-viewer tray cache absorbs refresh bursts.

3. Celebrity hot key (author index + viewer count). One author’s story index is read by millions; one story’s view count is incremented by millions. Fix: replicate hot author-index keys across cache nodes (read a random replica) + per-instance local LRU with a few-second TTL. Shard the view counter across N sub-keys, sum on read, so no single key takes millions of increments. Note we do not fan a celebrity’s story into millions of trays - fan-out on read is precisely what avoids that write storm.

4. Media egress bandwidth (116-350 GB/sec). Real but solved. Fix: CDN in front of immutable media, >95% edge hit rate, right-sized renditions, origin shield for viral stories. Same as any media service; not where the design’s difficulty lives.

5. Expiry / auto-delete correctness. Zombie stories are a privacy-breaking bug. Fix: native TTL at write time on every store (media 25h, rest 24h) plus a read-time now < expires_at filter to close the lazy-eviction gap. No scheduled delete job on the critical path. Independent TTLs mean no cross-store delete transaction to botch.

6. Followee scan for high-follow accounts. A viewer following tens of thousands strains the tray gather. Fix: cap the candidate gather at the most-recently-active N followees; the tray is a bounded horizontal strip anyway. Cache the viewer’s followee list.

7. Post-path spikes. Low average (580/sec) but can burst (live events). Fix: presigned direct-to-blob upload keeps bytes off the app tier; confirm just writes a couple of TTL’d KV entries. Trivially scalable stateless Story Service.

8. Shard keys, stated plainly: Object store -> story_id (uniform, no hot dir). Story meta -> story_id. Author index -> author_id. Seen-state -> viewer_id (colocate a viewer’s whole seen-set for one batched read - this is the important one; sharding seen-state by story_id would scatter every tray load across many shards). Viewer list -> story_id, counter sub-sharded. Sharding anything on created_at would pin all “now” writes to one shard - the classic mistake, and especially tempting here because everything is time-bounded; do not.

9. Single points of failure. Story / Tray / View Services are stateless behind load balancers. Every KV store, the CDN, and the object store are replicated. Async seen-writes flow through a partitioned, replicated queue with idempotent consumers. No single box whose loss stops the tray or playback; a lost async worker just delays a seen-marker.

10. Multi-region for a global user base. Fix: CDN edges are already global. Deploy Tray/View Services and the KV stores per region, routed by GeoDNS; presigned uploads target the nearest region. Because all story data is eventually consistent and expires in 24h, cross-region replication is maximally forgiving - even a region losing an hour of seen-state self-heals within a day as everything expires. Ephemerality makes the whole system trivially region-resilient.

Wrap-Up

The trade-offs that define this design:

  • Ephemerality inverts the storage problem. Because stories die in 24h, the entire active corpus is ~5 GB of metadata + ~250 GB of seen-state + ~100 TB of media, flat forever. There is no petabyte growth curve. This is why every store is a TTL’d cache, not a durable database, and why expiry is a free datastore property rather than a batch job.
  • Fan-out on read, not on write - the opposite of a feed. A durable feed amortizes fan-out writes across years of reads; a story is read for at most a day, so there is nothing to amortize, and the per-viewer seen-sort cannot be precomputed anyway. Compute the tray at read time over the tiny active-author set.
  • Seen-state is the real system. The sort key is “have I seen this,” a function of the viewer, answered by a per-(viewer,author) high-water mark that makes “unseen?” a single Snowflake-ID compare. The 5B/day seen-writes are absorbed by making them async, idempotent, sharded, capped, and deliberately low-durability.
  • Auto-expiry via TTL-at-write + read-time filter. Every store expires its own copy on its own 24h timer; a read-time expires_at check guarantees a consistent user-visible expiry with no scheduled delete storm and no cross-store coordination.
  • Low durability is a feature, embraced. Losing an in-flight seen-marker or a few seconds of cross-region lag costs nothing, so we trade durability for the throughput and simplicity the 5B-view/day workload demands.

One-line summary: an ephemeral-first system where stories, seen-state, viewer lists, and media all live in TTL’d stores that self-expire at 24h, the tray is computed fan-out-on-read over a tiny active-author set using per-viewer high-water marks for seen/unseen ordering, and 5B views/day of seen-writes are absorbed as async, idempotent, sharded, low-durability updates while the CDN handles the media bytes.