People see the swipe and think the hard part is video. It is not. Serving a 15-second clip is a CDN problem that YouTube solved a decade ago. The hard part of Reels is that every single swipe is a fresh ranking decision, the feed has no end, nobody chose what comes next, and the system has roughly 400 milliseconds to pick the next video out of billions while you are still watching the current one. That is the whole game.

Three things make this its own animal, different from a timeline feed or a Netflix grid. First, it is a single-column, full-screen, one-video-at-a-time pull feed - there is no grid of thumbnails to browse, so a bad pick is not one tile among twenty you ignore, it is the entire screen and you swipe away in under a second. The cost of a bad recommendation is brutal and immediate. Second, it is engagement-driven, not social-graph driven - most of what you watch is from accounts you do not follow, surfaced purely because the model thinks you will watch it to the end. The follow graph is a weak signal here, which breaks every “fan-out the follower feed” instinct. Third, the signal loop is tight and real-time - watch-time, replays, swipe-away-at-2-seconds, like, share, these must feed back into the next few videos within the same session, not tomorrow’s batch job.

The decision that colors everything: this is a recommendation system with a video player attached, not a video system with recommendations attached. Spend the depth on candidate generation, ranking, and the real-time signal loop. The media delivery is real load but it is a solved shape, and I will say so where it comes up rather than re-deriving a CDN.

Functional Requirements (FR)

In scope:

  • Endless personalized feed. Open the app, get a full-screen video. Swipe up, get the next one. Forever. No two users see the same sequence, and the sequence adapts within the session to what you just did.
  • Watch a video. 15-60 second clip, autoplays, loops. Playback starts effectively instantly (the next videos are already prefetched before you swipe).
  • Engagement actions. Like, comment, share, follow-the-creator, “not interested”, long-press to save. Each is a signal that must influence ranking.
  • Upload a video. Creator posts a clip; it is transcoded, and critically it must be discoverable by the recommender - including a brand-new video from a zero-follower creator (the cold-start problem is a product requirement here, not an edge case).
  • Creator and sound surfaces. Tap a creator to see their grid; tap a sound to see videos using it. These are secondary pull surfaces, not the main feed.

Explicitly out of scope, said out loud so I control the scope:

  • The ML model internals - architecture of the two-tower retrieval net or the ranking DNN, feature engineering, training. I design the serving system around the models (how candidates are generated, ranked, logged, and fed back), and treat the models themselves as versioned artifacts behind an inference service.
  • Comments threading, DMs, duets/stitches as a full editing feature. I note where the hooks are.
  • Monetization / ads insertion. Ads are a parallel candidate source blended into the ranked list; I will mention the insertion point but not design the ad auction (that is its own system).
  • Content moderation and policy classification internals. Moderation runs async off the upload pipeline and gates a video’s eligibility; I treat its output as a boolean the recommender respects.
  • The video transcoding ladder and DRM. Same shape as any video service; reused, not re-derived.

The single decision that drives the architecture: the dominant operation is “given user U and their live session context, return the next K videos ranked by predicted engagement.” That is a retrieval-plus-ranking pipeline run billions of times a day under a tight latency budget, and getting it fast, fresh, and good is the entire problem.

Non-Functional Requirements (NFR)

  • Scale: 1B total users, ~300M DAU. 5B videos watched/day. Assume each watch session surfaces ~50 videos and each is a ranking event. Uploads are a small fraction: ~30M new videos/day.
  • Latency: the next-batch recommendation call p99 under 400ms (it runs ahead of the user, masked by the current video playing, but it cannot fall behind the swipe rate). Playback start under 100ms (prefetched, so effectively zero perceived). Signal ingestion (a like, a watch-complete) visible to ranking within seconds, not minutes.
  • Availability: 99.99% on the feed-serving path. A blank feed is the app being broken. Degradation is graceful: if the personalized ranker is down, fall back to a cached/trending feed rather than an error.
  • Consistency: eventual everywhere. There is no “correct” feed, only a better or worse one. A like taking a few seconds to register, or a signal arriving slightly late, costs nothing. This looseness is what buys the throughput.
  • Durability: engagement events are durably logged (they are training data and the business’s lifeblood), but a single lost “watch 80%” signal in-flight is acceptable. Video media is durably stored. The serving-side ranking state (cached candidate lists, session context) is disposable and rebuildable.
  • Freshness: a brand-new video must be eligible for recommendation within minutes of upload (cold start). A viral video must be able to blow up within an hour. Yesterday’s batch-only recommender would miss both.

Back-of-the-Envelope Estimation (BoE)

Real numbers with arithmetic, because the read/write asymmetry and the per-swipe ranking cost dictate everything.

Feed-serving reads (the hot path):

5B videos watched/day, each is a ranking event.
  5B / 86,400 sec ≈ 58,000 ranking events/sec average
Peak (3x, evening prime time) ≈ 175,000 ranking events/sec

But we do not rank one video per call. We rank a BATCH of ~10 and
hand the client a buffer, so the actual recommendation RPC rate is lower:
  5B watches / 10 per batch = 500M feed-RPCs/day
  = 500M / 86,400 ≈ 5,800 feed-RPCs/sec average, ~17,000 peak.
Each RPC, though, scores hundreds of candidates -> real CPU is in ranking.

So ~17K peak recommendation RPCs/sec, but each fans out to retrieve and score hundreds of candidate videos. The compute is candidate scoring, not request count.

Candidate scoring volume - the real CPU load:

Per feed-RPC: retrieve ~500 candidates, rank them, return top ~10.
  17,000 RPC/sec * 500 candidates = 8.5M candidate scorings/sec at peak.
Each scoring is a DNN forward pass over a feature vector.
This is the number that sizes the ranking fleet (GPU/accelerator inference).

8.5M model inferences/sec at peak. That is the load that actually costs money, and it is why candidate generation must cut billions of videos down to hundreds before the expensive ranker ever runs.

Engagement signal writes (the feedback firehose):

Per video watched we emit multiple events: impression, watch-time,
completion/replay, and sometimes like/share/comment/skip.
Call it ~3 events per watch on average.
  5B watches * 3 ≈ 15B engagement events/day
  = 15B / 86,400 ≈ 174,000 events/sec average, ~520,000 peak.
Per event ~200 bytes (user, video, type, ts, watch_ms, context).
  15B * 200 B ≈ 3 TB/day of raw event log.

Half a million events/sec at peak into the signal pipeline. This dwarfs the upload rate by ~500x and is the write-side problem.

Uploads and storage:

30M videos/day uploaded.
  30M / 86,400 ≈ 350 uploads/sec average, ~1,000 peak. Small.
Avg video after transcode (all renditions) ≈ 5 MB * ~4 renditions ≈ 20 MB.
  30M * 20 MB ≈ 600 TB/day of new media
  Over a year ≈ 220 PB/year. Unlike Stories, this is PERMANENT content.
Metadata per video: ~500 bytes (ids, author, sound, hashtags, dims,
  moderation flags, embedding ref) -> 30M * 500 B ≈ 15 GB/day metadata.

220 PB/year of durable media (object store + CDN). The metadata is tiny by comparison but there is a lot of it cumulatively (billions of rows), which matters for the index.

Egress bandwidth (CDN’s job):

58,000 watches/sec * ~3 MB delivered per watch (compressed clip) ≈ 174 GB/sec avg
Peak ≈ 175,000/sec * 3 MB ≈ 525 GB/sec.

Hundreds of GB/sec from the CDN edge. Real, but a solved shape; not where the depth goes.

Cache / serving-state size:

Per-user recent-interaction state (last N videos seen, recent signals,
session embedding): ~5 KB/user.
  300M DAU * 5 KB ≈ 1.5 TB of hot session state. Sharded cache cluster.
Candidate-list cache (prebuilt per-user candidate pools): optional, bounded.
Video embedding index (ANN) for ~billions of candidate-eligible videos:
  ~1B active-ish videos * 256-dim float16 embedding (512 B) ≈ 512 GB,
  held in a sharded vector index for approximate nearest-neighbor retrieval.

The ANN embedding index (~hundreds of GB) and the per-user session state (~1.5 TB) are the serving hot sets. Both are rebuildable, both sharded.

High-Level Design (HLD)

Four paths. The feed-serving path (retrieve -> rank -> prefetch) is the hard, hot, latency-critical core. The signal path (ingest 15B events/day, feed them back) is the write firehose that makes the feed smart. The upload path is low-QPS but must make a video discoverable fast. The media playback path is high-bandwidth but a solved CDN problem.

                            +-----------------------------+
                            |          Clients            |
                            |  (prefetch buffer of ~10)   |
                            +--+----------+----------+-----+
                     feed RPC |          | signals  | media GET
                              |          |          |
                       +------v------+   |    +-----v---------+
                       | Feed / Rec   |  |    |      CDN       |
                       | Service (GW) |  |    | (video bytes) |
                       +--+-------+---+  |    +-----+---------+
           1. context    |       |      |          | origin fill
          (session,user) |       |      |    +-----v---------+
              +----------v-+  +---v----+ |    | Object Store  |
              | Session /   |  | Rank  | |    | (220 PB, perm)|
              | User Store  |  | Svc   | |    +---------------+
              | (hot state) |  |(DNN   | |
              +-------------+  | infer)| |
                       |       +---^---+ |
        2. candidates  |           |     |  3. events (impression,
   +-------------------v-------+    |     |     watch_ms, like, skip)
   |   Candidate Generation    |    |     |          |
   |   (parallel sources)      |    |     |   +------v----------+
   |  +---------------------+  |    |     |   | Signal Ingest   |
   |  | ANN / embedding ret |  |    |     |   | (Kafka, part.   |
   |  | collab-filter pool  |  |    |     |   |  by user_id)    |
   |  | trending / fresh    |  |    |     |   +---+--------+----+
   |  | follow-graph pool   |  |    |     |       |        |
   |  +---------------------+  |    |     | realtime|       | batch
   +---------+-----------------+    |     |  +------v--+  +--v-------+
             | reads                |     |  | Stream  |  | Data Lake|
   +---------v-----------+   +------v----+ |  | Proc    |  | / Feature|
   | Video Embedding      |   | Feature  |<+--| (update |  | Store    |
   | ANN Index (vector DB)|   | Store    |   | session, |  | (training|
   +---------------------+    | (online) |   |  counts)|  |  + batch |
                              +----------+   +---------+  |  features)|
                                                          +----------+
   +----------------+   upload   +----------------+
   |  Upload Svc     |<-- client | Transcode +    |--> embedding gen
   | (presign+confirm|           | Moderation     |    -> ANN index +
   +----------------+            | pipeline (async)|       metadata DB
                                 +----------------+

Feed-serving flow (the hot path - every swipe batch):

  1. Client’s prefetch buffer is running low (it holds ~10 ranked videos and plays through them). It calls GET /feed/next with the session token.
  2. Feed Service loads the session context (recent videos shown, recent signals this session, user long-term profile / embedding) from the hot Session/User Store.
  3. Feed Service calls Candidate Generation, which runs several retrieval sources in parallel and unions them into a pool of ~500 candidates: ANN retrieval over the user embedding, a collaborative-filtering pool, a trending/fresh pool, and a (small) follow-graph pool. Already-seen videos are filtered out here.
  4. The ~500 candidates plus their features (from the online Feature Store) go to the Ranking Service, which runs the heavy DNN forward pass per candidate, producing a predicted-engagement score, and returns the top ~10.
  5. Feed Service applies final business rules (diversity - do not show three videos from the same creator or sound in a row; freshness injection; ad slots; “not interested” exclusions) and returns the ordered batch with CDN URLs.
  6. Client appends to its prefetch buffer and pre-downloads the next 2-3 videos’ first segments so the swipe is instant.

Signal flow (the feedback firehose):

  1. As the user watches, the client emits events (impression on show, watch_ms on progress, completion/replay, like/share/skip) to the Signal Ingest layer (Kafka), partitioned by user_id.
  2. A real-time stream processor consumes these, updates the user’s session state and short-term counters (recent likes, recent skips, session embedding drift) in the online Feature Store within seconds - this is what makes the feed adapt mid-session.
  3. The same stream tees into the data lake for durable logging, batch feature computation, and model training. Video-level aggregates (view counts, like rates, completion rates) are updated here and feed the trending pool and the ranker’s video features.

Upload flow (make a video discoverable fast):

  1. Creator gets a presigned URL, uploads media directly to the object store.
  2. Async transcode + moderation pipeline produces renditions and a policy verdict. In parallel, an embedding generator computes the video’s content embedding (from frames/audio/caption) and inserts it into the ANN index, and writes metadata to the video DB with an “eligible” flag gated on moderation.
  3. The moment the embedding is in the ANN index and moderation passes, the video is retrievable by candidate generation - even with zero followers. This is the cold-start path, and it must complete in minutes.

The whole point of the HLD: a two-stage retrieve-then-rank pipeline fronts a prefetch buffer, fed by a real-time signal loop. Candidate generation makes the billion-video corpus tractable; ranking makes it good; the signal loop makes it adapt; the client buffer makes it feel instant.

Component Deep Dive

The hard parts, naive-first then evolved: (1) candidate generation - cutting a billion videos to hundreds, (2) ranking under the latency budget, (3) the real-time signal loop and session adaptation, (4) cold start for new videos and the prefetch/playback illusion.

1. Candidate generation - a billion videos down to hundreds

This is the part that makes an “endless personalized feed” tractable. You cannot score a billion videos per swipe; you must cheaply retrieve a few hundred good ones first.

The two obvious first cuts. Either (a) run the ranking model over every eligible video and take the top ones, or (b) skip personalization and serve a global trending list to everyone.

Where it breaks:

  • Scoring everything is computationally impossible. ~1B candidate-eligible videos * 8.5M-inference/sec budget means you could serve roughly one user per second globally. The ranker is a DNN forward pass; running it a billion times per feed request is off by ~six orders of magnitude. Non-starter.
  • Global trending is not a personalized feed. It ignores the entire premise. Everyone sees the same thing, niche interests are never served, and engagement collapses because the median user does not want the globally-most-viral clip. It also creates a rich-get-richer loop where only already-popular videos ever surface, killing the creator ecosystem.
  • A single SQL/graph query cannot express “videos like what this user engages with.” Interest similarity is a dense-embedding notion, not a filter over columns. WHERE hashtag IN (...) is a crude proxy that misses the semantic match the whole product depends on.

The failure is treating retrieval as either brute force or a dumb filter. It is neither; it is approximate nearest-neighbor search plus a union of cheap specialized pools.

Evolved approach: multi-source candidate generation, union, then dedup

Generate candidates from several cheap, parallel sources, each good at something different, and union them into a ~500-candidate pool that the expensive ranker then sorts.

candidate_pool(user U, session ctx):
    parallel:
      A = ann_index.query(U.embedding, k=300)       # "videos like your taste"
      B = collab_filter(U, k=100)                    # "users like you watched"
      C = trending_fresh_pool(U.region, k=50)        # exploration / freshness
      D = follow_graph_pool(U, k=50)                 # creators U follows
    pool = dedup(A | B | C | D)
    pool = filter(pool, not_seen(U), moderation_ok, not_blocked(U))
    return pool            # ~500 candidates

Source A - embedding ANN retrieval (the backbone). Every video has a content/behavior embedding; every user has an embedding summarizing their taste (updated from their engagement). Retrieval is approximate nearest-neighbor search in a sharded vector index (HNSW / IVF). This is the “for you” core - it finds videos semantically close to what you engage with, including from creators you have never heard of. The user embedding is the single most important piece of state: it is what makes the feed yours.

Source B - collaborative filtering pool. “People with watch histories like yours also finished these.” Captures signals the content embedding misses (a video blowing up with your cohort before its content signal is strong). Precomputed in batch, refreshed frequently, looked up by user cluster.

Source C - trending / fresh / exploration pool. Deliberately injects videos the model is unsure about - new uploads, rising videos, slightly-off-your-usual content - to avoid a filter bubble and to gather signal on fresh content (this is the exploration half of the explore/exploit trade-off, and it is also the cold-start on-ramp).

Source D - follow-graph pool. A small slice from creators you follow. Deliberately small, because Reels is engagement-driven, not follow-driven - the follow graph is one weak source among four, not the spine of the feed. This is the exact inversion of a Twitter/Instagram timeline.

Why union-of-sources beats one clever retriever: each source has a blind spot. ANN over-exploits your current taste (bubble); collab-filter lags for new users; trending ignores you; follow-graph is narrow. Unioning them and letting the ranker arbitrate gives coverage no single source has. Dedup and already-seen filtering happen once, post-union, against a per-user bloom filter / recent-seen set in the session store.

Retrieval strategyCost per requestPersonalizationCold-start / new videoVerdict
Score all videosImpossible (~1B inferences)PerfectFineNon-starter
Global trending onlyO(1) cachedNoneGoodNot a feed
Single ANN retrieverCheap, boundedStrong but bubble-proneWeak (no embedding history)Incomplete
Multi-source union + rankBounded (~500 scored)Strong + diverseHandled by trending/fresh poolThe answer

2. Ranking under a 400ms budget

Candidate generation handed us ~500 videos. Now order them by predicted engagement, well, and fast.

Naive approach: one giant model scoring candidates sequentially, synchronously

Load every feature for all 500 candidates, run them through one large model one by one, sort, return.

Where it breaks:

  • Sequential scoring blows the latency budget. 500 forward passes serialized, even at 1ms each, is 500ms - over the 400ms p99 budget before you have added feature fetching or network. The swipe rate outruns you and the buffer drains to a stall.
  • Feature fetching is the hidden cost. Each candidate needs video features (counts, embeddings, author stats) and cross features (this user x this video’s topic). Fetching 500 candidates’ features one-at-a-time from the Feature Store is 500 round trips. The model is not even the bottleneck; the I/O is.
  • “Predicted engagement” is not one number. Optimizing only for watch-time breeds clickbait and rage-bait; optimizing only for likes ignores silent watchers. A single scalar target degrades the feed over time.

Evolved approach: batched multi-task scoring, two-stage ranking, cached features

  • Batched inference. Score all 500 candidates in one (or a few) batched forward passes on an accelerator, not a Python loop. 500 rows through a GPU batch is a handful of milliseconds, not half a second. This single change is what makes the budget feasible.
  • Batched feature fetch. One MGET per feature source for all 500 candidates, from an online Feature Store colocated with the ranker. Video features are cached hot (they change slowly relative to a request); user features come from the session context already loaded. Round trips drop from 500 to a handful.
  • Two-stage ranking (lightweight pre-rank -> heavy fine-rank). If 500 is still too many for the full model, a cheap linear/shallow model pre-ranks 500 down to ~50, and the expensive multi-task DNN fine-ranks those 50. Classic retrieval -> pre-rank -> rank funnel; each stage is cheaper per item and handles more items.
  • Multi-task prediction, then a blend. The ranker predicts several heads - P(watch-to-completion), P(replay), P(like), P(share), P(comment), P(follow), and P(skip-in-2s as a negative) - and a weighted blend produces the final score. The weights encode product values (shares and follows weighted up as strong intent; 2-second skips weighted down hard). This is where “engagement-driven” gets its definition, and tuning these weights is a core product lever, not an ML detail.
rank(pool ~500, user ctx):
    feats = feature_store.mget_batch(pool.video_ids)      # 1 batched call
    cross = build_cross_features(user ctx, feats)         # in-memory
    pre   = prerank_model.batch_score(pool, feats)        # cheap, 500 -> top 50
    heads = rank_model.batch_predict(top50, feats, cross) # multi-task DNN
    score = ( w_complete*heads.complete + w_like*heads.like
            + w_share*heads.share + w_follow*heads.follow
            - w_skip*heads.skip2s )                        # business-tuned blend
    return topK(score, K=10)
  • Graceful degradation. If the ranker fleet is overloaded or down, the Feed Service falls back to serving the candidate pool ordered by a cheap heuristic (recency + global engagement), or a cached recent feed. A slightly-worse feed beats a blank screen; availability wins over optimality on this path.

The ranker’s forward pass was never the scary part on its own; serializing it per-candidate and fetching features per-candidate is what kills you. Batch both, funnel the candidate count down in stages, and predict multiple objectives so the feed stays healthy over time.

3. The real-time signal loop and mid-session adaptation

This is what makes the feed feel like it is reading your mind: watch three cooking videos to completion and the next five tilt toward cooking, within the same session. A batch-only recommender cannot do this.

Naive approach: update the user profile in a nightly batch job

Log all engagement, run a big batch job overnight that recomputes every user’s embedding and interest profile, serve from yesterday’s profile all day.

Where it breaks:

  • It misses the session entirely. The strongest signal about what you want right now is what you just did in the last 60 seconds. A nightly batch ignores it completely, so the feed cannot react to your mood, the fact that you are clearly in a rabbit hole, or that you just hit “not interested” three times. The product’s core magic is gone.
  • New-interest latency is a full day. Pick up a new interest at 9am and the feed does not reflect it until tomorrow. For a feed whose entire value is “it knows what I want,” 24-hour latency is fatal.
  • “Not interested” and skips must be near-instant. If I swipe away from a video in 1 second or tap “not interested,” and the next three videos are the same thing, I am gone. Negative feedback especially must round-trip in seconds.

Evolved approach: a real-time stream layer updating short-term state, layered over batch long-term state

Split user state into two layers and update them on different clocks:

  • Long-term profile (batch). The stable user embedding and interest distribution, recomputed in batch (hourly/daily) from the full engagement history in the data lake. This is the slow, high-quality base.
  • Short-term session state (real-time stream). A rolling window of this session’s signals - recent watch-completions, recent skips, recent likes, a session embedding that drifts toward what you are engaging with now, and an exclusion set from “not interested”. Updated by a stream processor consuming the Kafka event firehose, written to the online Feature Store within seconds.
Kafka (partitioned by user_id)
   -> Stream Processor (Flink/equivalent):
        on watch_complete(u, v):  session_embed[u] <- drift_toward(v.embed)
                                   recent_pos[u].add(v)
        on skip_2s(u, v):         session_embed[u] <- drift_away(v.embed)
                                   recent_neg[u].add(v.topic)
        on not_interested(u, v):  exclude[u].add(v.creator, v.topic, v.sound)
   -> online Feature Store (per-user session state, few-sec freshness)

How the two layers combine at serve time: candidate generation queries the ANN index with a blend of the long-term and session embeddings (weighted toward the session embedding when the session has strong signal), and the ranker gets both long-term and short-term features. So the base is stable and high-quality, but the session steers it live.

Why partition by user_id: all of a user’s events land on the same stream partition in order, so the session state is updated by a single consumer without cross-partition coordination - the same colocation logic as sharding seen-state by viewer. Ordering within a user matters (a like after a watch means something different than before); ordering across users does not.

Exactly-once vs at-least-once: the event pipeline is at-least-once (cheaper, simpler). Session updates are designed to be idempotent-ish and tolerant of a duplicate or a dropped event - double-counting one watch barely moves the session embedding, and the NFRs already accept losing an in-flight signal. The durable training log in the data lake deduplicates properly offline where correctness matters; the serving path trades it for throughput.

The loop closes in seconds: swipe away -> event to Kafka -> stream processor drifts the session embedding away -> next candidate-gen call retrieves less of that -> next batch reflects it. That round-trip being seconds, not hours, is the product.

4. Cold start and the prefetch illusion

Two smaller-but-essential pieces: making a brand-new video (and a brand-new user) discoverable, and making the swipe feel instant.

Naive approach: rank by engagement counts; download video on swipe

Rank videos by their like/view counts, and fetch each video’s bytes when the user swipes to it.

Where it breaks:

  • New videos can never start. A video with zero views ranks last by any engagement-count signal, so it is never shown, so it never gets views - a dead-on-arrival loop. The creator ecosystem dies; only established creators ever surface. Cold start is a product requirement (anyone can go viral), not an edge case to ignore.
  • New users get nothing personalized. A brand-new user has no history, no embedding, so ANN retrieval has nothing to query with. Their first session is make-or-break for retention and cannot be blank.
  • Download-on-swipe stalls constantly. Fetching a 3 MB clip when the user swipes to it adds hundreds of ms to a second of latency - a visible stall on every swipe. The whole illusion of an effortless infinite feed collapses.

Evolved approach: content-based cold start + exploration budget; aggressive client prefetch

Cold-start new videos:

  • A new video gets a content embedding at upload time (from frames, audio, caption, hashtags) - independent of any engagement. So ANN retrieval can surface it to users whose taste embedding is near it, from minute one, with zero views.
  • The exploration pool (Source C) reserves a budget of feed slots for under-explored videos, deliberately showing new uploads to a sample of plausibly-interested users to gather engagement signal. If early signal is strong (high completion rate relative to impressions), the video graduates into the mainstream pools and can go viral; if weak, it fades. This is a multi-armed-bandit explore/exploit at the feed level.

Cold-start new users:

  • First session falls back to a popular-by-region / demographic-prior feed blended with rapid exploration - show a diverse spread, watch what the user engages with, and build a session embedding within the first few swipes. New-user personalization latency is minutes-into-the-first-session, driven by the same real-time loop.

The prefetch illusion:

  • The client holds a buffer of ~10 ranked videos and pre-downloads the first segment(s) of the next 2-3 while the current one plays. Swiping plays from local buffer - zero perceived latency. The recommendation RPC runs ahead of the user, refilling the buffer before it drains (this is why we rank a batch of 10, not one video at a time).
  • Prefetch is adaptive: on a fast connection or a user on a known hot streak of quick swipes, prefetch deeper; on cellular/limited data, prefetch shallower to avoid wasting bandwidth on videos that get skipped. The buffer depth is a bandwidth-vs-instant-feel trade-off tuned per network condition.
  • Prefetched-but-skipped videos are wasted bytes - the cost of the illusion. Keeping that waste bounded (don’t prefetch 10 deep) is why the buffer is ~2-3 fully downloaded, not the whole batch.

Cold start via content embeddings plus a reserved exploration budget keeps the ecosystem alive; the client prefetch buffer plus run-ahead ranking is what turns a 400ms server pipeline into a swipe that feels instant.

API Design & Data Schema

API

GET /api/v1/feed/next?session=<token>&count=10&net=wifi
Response 200:
{
  "items": [
    {
      "video_id": "1892441122330",
      "author": { "id": "42", "handle": "chef_ria" },
      "sound_id": "55123",
      "duration_ms": 28000,
      "renditions": [
        { "q": "720p", "url": "https://v.cdn.example/1892.../720.m3u8" },
        { "q": "480p", "url": "https://v.cdn.example/1892.../480.m3u8" }
      ],
      "prefetch_hint": "high",          // client prefetch depth signal
      "rank_debug": { "score": 0.81 }   // internal / experiments only
    }
  ],
  "next_session": "<opaque cursor; carries served-set + session state ref>"
}
Latency target: p99 < 400ms. Runs ahead of the user; must not fall behind swipe rate.

POST /api/v1/events                     // batched, fire-and-forget
Body:
{
  "session": "<token>",
  "events": [
    { "t": "impression",     "video_id": "1892...", "ts": 1699999999000 },
    { "t": "watch",          "video_id": "1892...", "watch_ms": 26000,
                             "video_ms": 28000, "replays": 1 },
    { "t": "like",           "video_id": "1892...", "ts": ... },
    { "t": "skip",           "video_id": "1893...", "watch_ms": 1200 },
    { "t": "not_interested", "video_id": "1894...", "reason": "topic" }
  ]
}
Response 202 Accepted                   // enqueued to Kafka; never blocks playback

POST /api/v1/videos/upload-url
Response 200: { "video_id": "...", "upload_url": "<presigned PUT>", "expires_in": 600 }

POST /api/v1/videos/{id}/confirm
Body: { "caption": "...", "hashtags": ["cooking"], "sound_id": "55123" }
Response 202: { "video_id": "...", "status": "processing" }   // transcode+moderate async

GET /api/v1/videos/{id}                  // metadata + current CDN urls
GET /api/v1/creators/{id}/videos?cursor= // creator grid (secondary pull surface)
GET /api/v1/sounds/{id}/videos?cursor=   // videos using a sound

Pagination on secondary surfaces is cursor-based; the main feed is not paginated at all - it is an infinite stream driven by /feed/next and the next_session cursor that carries the already-served set (so we never repeat within a session) plus a reference to live session state.

Data store choices

Different data, different stores. The theme: durable, append-mostly content and events in scalable stores; disposable, rebuildable serving state in fast caches; semantic retrieval in a vector index. Little of this is relational.

1. Object Store (media) + CDN - not a database. Transcoded renditions keyed {video_id}/{quality}.m3u8 + segments, immutable, permanent (unlike Stories). ~220 PB/year, lifecycle to colder tiers for old/unpopular videos. Fronted by CDN, >95% edge hit for popular clips.

2. Video Metadata - wide-column NoSQL (Cassandra / Bigtable). Billions of rows, point and small-range reads, append-heavy, no joins needed. SQL would not scale the row count and we need no cross-video transactions.

Table: videos
  video_id      BIGINT   PARTITION KEY      -- Snowflake, time-sortable
  author_id     BIGINT
  sound_id      BIGINT
  caption       STRING
  hashtags      LIST<STRING>
  duration_ms   INT
  created_at    TIMESTAMP
  moderation    ENUM     -- pending | ok | blocked   (recommender gates on 'ok')
  embedding_ref STRING   -- pointer into the ANN index
  stats_ref     STRING   -- pointer to the counter rollups
  Shard by: video_id   (uniform; single-video reads hit one partition)
  Secondary index: author_id -> video_ids (for creator grid), sound_id -> video_ids

3. Video Embedding ANN Index - vector database (HNSW/IVF, sharded). The retrieval backbone. ~1B eligible-video embeddings (~256-dim). Approximate nearest-neighbor query by user embedding. Rebuilt/updated incrementally as videos are uploaded and as embeddings are refreshed.

Index: video_embeddings
  key: video_id, vector: float16[256], filterable: {region, lang, freshness, moderation}
  Query: ANN(user_embedding, k=300, filter=eligible)
  Shard by: embedding-space partition (so each shard is a coherent subspace) or by id with scatter-gather

4. Online Feature Store - KV cache (Redis / low-latency KV). Hot features for ranking and session state for the real-time loop. Rebuildable from the data lake.

Key: user:{user_id}:profile    -> long-term embedding + interest dist  (batch-updated)
Key: user:{user_id}:session    -> session embedding, recent_pos/neg, exclude set,
                                  recently-served bloom filter          (stream-updated, TTL hours)
Key: video:{video_id}:feats    -> cached stats, author stats, topic feats (hot, slow-changing)
  Shard by: user_id (user state) / video_id (video feats)

5. Engagement Event Log - Kafka -> Data Lake (columnar, e.g. Parquet on object store). 15B events/day, 3 TB/day, durable, append-only. Source of truth for training and batch features. Kafka partitioned by user_id for ordered per-user stream processing.

6. Counters / aggregates - sharded counters (as in any high-write count system). Per-video view/like/completion counts, sub-sharded to kill hot keys on viral videos, summed on read, rolled into the Feature Store.

Why not SQL for the hot paths: the workloads are append-heavy, partition-scoped, and need throughput and horizontal scale over ACID transactions and joins. The only naturally relational data (user accounts, the follow graph’s source of truth, creator payout ledgers) lives in a separate transactional store and is read as a cached copy by the feed path - never joined against live under a 400ms budget.

Bottlenecks & Scaling

Where it breaks first, in order, and the fix for each:

1. Ranking compute (breaks first - 8.5M candidate scorings/sec at peak). This is the dollar cost and the latency risk. Fix: two-stage funnel (retrieve ~500 -> cheap pre-rank to ~50 -> heavy DNN on 50), batched accelerator inference (not per-candidate loops), batched feature fetch, and horizontal scaling of a stateless ranking fleet. Graceful fallback to a heuristic/cached feed when the fleet saturates - availability beats optimality here.

2. Signal ingestion firehose (15B events/day, ~520K/sec peak). The write side. Fix: Kafka partitioned by user_id, at-least-once, consumed by a horizontally scaled stream processor. Batch events client-side (send arrays, not one RPC per event). Tee to the data lake for durable training data. Idempotent-ish, loss-tolerant session updates so a dropped event costs nothing.

3. ANN retrieval latency and index scale (~1B vectors). Retrieval must be fast and fresh. Fix: sharded ANN index (HNSW/IVF) with scatter-gather, replicas for QPS, incremental inserts so new videos appear in minutes (cold start). Approximate (not exact) nearest-neighbor keeps it sub-10ms. Cache per-user candidate pools for a few seconds to absorb buffer-refill bursts.

4. Celebrity / viral hot key (one video’s stats, one creator’s grid). A viral video’s counter takes millions of increments; its metadata and bytes are read by millions. Fix: sub-shard the view/like counters across N keys and sum on read; replicate hot metadata and the CDN segments (origin shield for virality); the creator grid is a secondary surface served from a cache. Note that because the feed is fan-out-on-read (computed per request), a viral video does not trigger a write-fanout into a billion feeds - the pull model avoids that storm entirely.

5. Media egress (174-525 GB/sec). Real but solved. Fix: multi-CDN, >95% edge hit on popular clips, ABR renditions, aggressive client prefetch of the next 2-3 clips. Adaptive prefetch depth by network to avoid wasting cellular bandwidth on skipped videos.

6. Session-state read/write volume (300M DAU, ~1.5 TB hot). Read on every feed RPC, written on every signal. Fix: shard the online Feature Store by user_id so a user’s whole session state colocates (one batched read per feed call, one consumer per user partition for writes). TTL session state (hours) - it is rebuildable from the durable log, so the cache can be lossy. Never put this in a relational DB.

7. Filter bubble / feedback-loop collapse (a quality bottleneck, not a load one). Pure exploit converges to a narrow, degrading feed and starves new content. Fix: a reserved exploration budget in candidate generation (explore/exploit), multi-task ranking (not single-metric watch-time, which breeds rage-bait), diversity rules at the final re-rank (no three-in-a-row same creator/sound/topic), and “not interested” as a hard, instant exclusion. These are as load-bearing as any infra fix - the feed degrades without them.

8. Cold start (new video dead-on-arrival, new user blank first session). Fix: content-based embeddings at upload (retrievable with zero views) + the exploration budget to gather early signal; new-user feed from region/demographic priors + rapid in-session exploration to build an embedding within the first few swipes.

9. Shard keys, stated plainly: Object store -> video_id (uniform). Video metadata -> video_id. ANN index -> embedding-space partition (coherent subspaces) or video_id with scatter-gather. Feature Store user state -> user_id (colocate a user’s session for one batched read - the important one). Video feats -> video_id. Kafka events -> user_id (ordered per-user stream). Counters -> video_id, sub-sharded for viral hot keys. Sharding session state by video_id or events by anything but user_id would scatter a user’s state and break ordered session updates - the classic mistake.

10. Single points of failure. Feed, Candidate-Gen, and Ranking Services are stateless behind load balancers and scale horizontally. ANN index, Feature Store, metadata store, Kafka, CDN, and object store are all replicated. The real-time loop is best-effort: a lost stream-processor instance delays some session updates but the batch profile still serves. The whole feed degrades gracefully to cached/trending rather than erroring.

11. Multi-region for a global 1B-user base. Fix: CDN edges already global. Deploy Feed/Rank/Candidate services, the Feature Store, ANN replicas, and regional Kafka per region, routed by GeoDNS; uploads target the nearest region and replicate media + embeddings globally. Because the feed is eventually consistent and there is no single correct answer, cross-region replication is maximally forgiving - a region serving a slightly staler user embedding or video pool is invisible to the user.

Wrap-Up

The trade-offs that define this design:

  • It is a recommender, not a video store. Media delivery is a solved CDN shape; the depth lives in a two-stage retrieve-then-rank pipeline that cuts a billion videos to hundreds (candidate generation) and then orders those well and fast (batched multi-task ranking under 400ms). Everything else serves that pipeline.
  • Fan-out on read, driven by engagement, not the follow graph. Unlike a timeline, there is nothing to fan out on write - the feed is computed per request from a union of candidate sources (ANN, collaborative, trending, and a deliberately-small follow slice). This is also why a viral video causes no write storm: the pull model never pushes into a billion feeds.
  • The real-time signal loop is the magic. A two-layer user state - stable batch profile plus a stream-updated session state - lets the feed adapt within seconds of what you just did, while staying high-quality. At-least-once, loss-tolerant signals buy the throughput for 15B events/day.
  • Cold start and exploration are product requirements, not edges. Content embeddings make zero-view videos discoverable; a reserved exploration budget gathers their signal and keeps the ecosystem (and the user out of a bubble) alive. Multi-task ranking and diversity rules keep the feed healthy over time, which is as load-bearing as any infra decision.
  • Instant-feel is a client illusion over a 400ms server. A prefetch buffer of ~10 ranked videos with the next 2-3 pre-downloaded, refilled by run-ahead ranking, turns a real pipeline latency into a swipe with zero perceived delay - at the cost of some wasted bytes on skipped prefetches, bounded by adaptive prefetch depth.

One-line summary: an engagement-driven recommendation system with a video player attached - a two-stage retrieve-then-rank pipeline over multi-source candidate generation, steered by a real-time two-layer signal loop, cold-started via content embeddings and an exploration budget, and masked behind a client prefetch buffer so an endless personalized feed of 15-60s clips feels instant at 1B users and 5B watches a day.