Twitch is not a video problem with a chat bolted on. It is a real-time interaction problem that happens to carry video. That is the thing that trips people in the interview: they reach for the YouTube answer (transcode once, serve immutable bytes from the edge forever) or the live-sports answer (one pro feed, fan it to millions through a CDN) and both are wrong in the way that matters. Twitch has two features neither of those has. First, the content comes from millions of amateur streamers at once, each a separate live ingest with its own transcode, most of them watched by twelve people and a handful watched by a million - a brutal long tail with no pre-encode and no warning about which stream goes viral. Second, the whole point is the loop: a viewer types in chat, the streamer reads it on screen two seconds later and reacts, and the viewer sees that reaction. If the video is 30 seconds behind, that loop is dead and you have built YouTube. So the latency target is not “feels live,” it is sub-5-second glass-to-glass, tight enough that chat and video stay in sync.

So the problem is this: accept hundreds of thousands of simultaneous live ingests from random people on random uplinks, transcode each into an adaptive ladder on the fly, deliver every stream under 5 seconds of latency, and fan a firehose of chat messages out to every viewer of a stream - where one hot stream can have 1,000,000 concurrent viewers all receiving the same messages in real time - while also taking people’s money for subscriptions and donations without ever double-charging. Everything hard here lives in two places: the per-stream transcode economics of the long tail, and the chat fanout to a million sockets on a single channel. Let me build it properly.

Functional Requirements (FR)

In scope:

  • Go live. A streamer pushes a live feed (RTMP/SRT/WebRTC from OBS or a console) to an ingest endpoint and is live within a couple of seconds. The stream gets a channel URL immediately.
  • Watch live, low latency. A viewer opens a channel and sees the stream with sub-5-second glass-to-glass latency, with adaptive bitrate so a phone on 4G and a desktop on fiber both watch without stalling.
  • Real-time chat. Every viewer of a channel sees a shared chat. Messages posted by any viewer are delivered to all other viewers of that channel in well under a second. Chat survives the stream going from 10 to 1,000,000 viewers.
  • Donations / bits / cheers. A viewer sends money (or platform currency “bits”) that pops up as an on-stream alert and a highlighted chat message. Money must be charged exactly once and never lost.
  • Subscriptions. A viewer subscribes to a channel (recurring monthly), unlocking emotes, a badge, and ad-free viewing. Sub events also show as chat alerts.
  • VOD / replay. When a stream ends, it is retained as a replay (video-on-demand) that can be watched later. Clips (short highlights) can be cut from live or VOD.
  • Presence and viewer count. The live viewer count on a channel, and who is in chat, roughly in real time.

Explicitly out of scope (own the scope out loud):

  • Recommendations / the browse directory ranking. Which streams surface on the home page is a ranking system of its own; I consume signals (concurrent viewers, follows) and note the hook, I do not build the ranker.
  • The full payments backend (card processing, fraud, payouts to streamers). I integrate a payment provider (Stripe-style) and own the idempotency and the event that money triggers; I do not build the card rails or the streamer payout/tax engine.
  • DRM, content moderation ML, DMCA/content-ID. Real and large. I flag the hook points (a moderation consumer on chat, a scanner on VOD) and move on.
  • The emote/CDN asset system, follows graph, whispers (DMs). Mentioned where they touch the core planes; not designed in full.
  • VOD transcode ladder details. VOD is essentially the YouTube problem (transcode once, serve immutable segments); I reuse that and spend my depth on the live and chat planes, which are the actual Twitch-specific hard parts.

The decision that drives everything: Twitch is three loosely-coupled real-time planes - a video plane (ingest, transcode, deliver), a chat/events plane (a pub/sub firehose per channel), and a money plane (idempotent transactions that emit events) - glued by one shared identity (channel_id) and one shared clock constraint (chat and video must feel simultaneous, so video must be sub-5s). The video plane and the chat plane scale on completely different axes and must be built as separate systems that only meet at the player. Conflating them is the classic mistake.

Non-Functional Requirements (NFR)

  • Scale: 30M DAU. Assume ~5M concurrent viewers at a typical moment, ~15M at peak. ~100,000-150,000 concurrent live channels, following a heavy power law: the top ~100 channels hold a large share of all viewers, a single hot stream can hit 1,000,000 concurrent viewers, and the long tail is hundreds of thousands of channels with single-digit audiences. Chat: at peak, millions of messages per minute platform-wide; a single big channel can see thousands of messages per second inbound and must fan each out to up to 1M sockets.
  • Latency: Video glass-to-glass under 5 seconds at p95 (that is the product-defining number; “low latency” mode pushes toward 2s). Chat delivery under ~500ms from send to appearing on other viewers’ screens. Player start (time-to-first-frame) under ~2s from a warm edge. Donation/sub alert on screen within a couple of seconds of payment confirmation.
  • Availability: 99.9%+ on the watch path. A specific guarantee: a streamer’s ongoing broadcast must not drop because one server died - ingest and transcode must fail over. Chat degrading (dropping some messages under extreme load) is more acceptable than chat going fully down; video stalling is the worst failure.
  • Consistency: Eventual everywhere except money. Viewer count can be approximate. Chat is best-effort ordered per channel, at-least-once-ish delivery, and may shed load (drop messages) on a 1M-viewer channel rather than fall over. Money is strongly consistent and exactly-once: a bit donation or a subscription charge is an idempotent transaction; double-charging or losing a charge is unacceptable.
  • Durability: Money events and subscription state: 11 nines, never lost. VODs: durable blob storage, 11 nines, like YouTube. Live segments are ephemeral - a live media segment lives for seconds in a cache and is then discarded (or rolled into the VOD); losing a single live segment just means a viewer skips 2 seconds, it is not a durability event.
  • The real budget has two numbers. On video it is transcode cost per concurrent stream (because you pay to transcode 150,000 streams most of which nobody watches) and sub-5s latency. On chat it is fanout amplification (one inbound message to a 1M-viewer channel becomes 1M outbound sends). The whole design optimizes those two.

Back-of-the-Envelope Estimation (BoE)

Real numbers with the arithmetic, because they pick the architecture.

Concurrency and channels:

30M DAU. Peak concurrent viewers ~15M (heavy during evenings/events).
Concurrent live channels: ~150,000 at peak.
Power law: top ~100 channels might hold ~40% of viewers.
   0.40 * 15M = 6M viewers across ~100 channels => ~60,000 avg on a big one,
   with the single hottest at 1,000,000.
Long tail: ~140,000 channels sharing the remaining ~9M => ~64 viewers each,
   and most of THOSE have <10. Many streams nobody watches at all.

The long tail is the cost bomb: you transcode a stream the moment it goes live, before you know if anyone will watch.

Video ingest and transcode:

150,000 concurrent live streams.
Each ingest ~3-6 Mbps from the streamer (source).
   150,000 * 4.5 Mbps = ~675 Gbps of raw INGEST bandwidth. Big but bounded.
Transcode: each source becomes a ladder (source passthrough + 720p + 480p + 360p + 160p).
   Say ~4 transcoded renditions per stream (source passed through as-is).
   A single live transcode of one stream needs ~0.5-1 CPU-core-equiv (or GPU slice).
   150,000 streams * ~1 transcode unit = 150,000 transcode units at peak.
   This is the dominant compute cost. => transcode-on-demand is mandatory (below).

Video delivery bandwidth (the egress, where the money goes):

15M concurrent viewers, avg bitrate watched ~2.5 Mbps (mix of 1080p/720p/480p).
   15M * 2.5 Mbps = 37.5 Tbps of egress at peak.
A single 1M-viewer stream: 1M * 2.5 Mbps = 2.5 Tbps on ONE channel.
   => served entirely from CDN edge; origin never sees 1M requests for a segment.
Segment size: 2s segment at 2.5 Mbps = ~0.6 MB. New segment every 2s per rendition.

Chat fanout (the other hard axis):

Hot channel: 1,000,000 concurrent viewers.
Inbound chat on a hot channel: say ~1,000 messages/sec posted (most viewers lurk).
Outbound = inbound * viewers = 1,000 * 1,000,000 = 1,000,000,000 sends/sec on ONE channel.
   => 1 billion socket writes per second for a single hot stream. This is the real monster.
   This is why chat must shard by channel, rate-limit, batch, and shed load.
Platform-wide chat: millions of msgs/min inbound, fanned to ~15M connected sockets.
Persistent connections: ~15M concurrent WebSockets. At ~100k sockets/edge box => ~150 edge chat servers minimum, realistically more for headroom.

Storage (VODs):

150,000 streams, avg 3 hours, say ~2,000 end per hour platform-wide at steady state.
VOD stored at ~3 Mbps (post-transcode, one main rendition kept hot + ladder):
   3 hrs * 3 Mbps = ~4 GB per VOD (single rendition); ~10-12 GB with the ladder.
Per day if ~1M streams end: 1M * ~8 GB = ~8 PB/day of new VOD if kept forever.
   => retention policy: keep recent VODs hot, expire old ones, keep clips longer.

The two takeaways that shape the whole design: transcode is the compute cost and must be allocated lazily and killed aggressively for unwatched streams; chat fanout on a hot channel is a billion sends/sec and must be a sharded pub/sub tree, not a database.

High-Level Design (HLD)

Three planes. The player is the only place they rejoin.

                         STREAMER (OBS / console)
                                 | RTMP / SRT push
                                 v
                   +-------------------------------+
                   |   INGEST EDGE (POP near user)  |  pick nearest, auth stream key
                   |   RTMP ingest, validates feed  |
                   +---------------+----------------+
                                   | source feed
                                   v
                   +-------------------------------+
                   |      TRANSCODE WORKERS         |  GPU/CPU pool, one job per stream
                   |  source -> 720/480/360/160     |  packages LL-HLS/CMAF segments
                   +---------------+----------------+
                                   | push segments + manifest
                                   v
                   +-------------------------------+
                   |   ORIGIN SHIELD / PACKAGER     |  holds last N segments per rendition
                   +---------------+----------------+
                                   | pull (cache-fill)
                                   v
            +----------------------------------------------------+
            |                CDN EDGE (global PoPs)              |  serves segments to viewers
            +--------------------------+-------------------------+
                                       |  LL-HLS / CMAF over HTTP
                                       v
   VIDEO PLANE  ------------------>  [ VIEWER / PLAYER ]  <------------------ CHAT PLANE
                                       ^            ^
                                       | WebSocket  | WebSocket
                                       |            |
   +-----------------------------+     |            |     +-----------------------------+
   |   EDGE GATEWAY (WS term)     |-----+            +-----|   EDGE GATEWAY (WS term)    |
   |  holds viewer sockets        |                        |                             |
   +--------------+--------------+                         +--------------+--------------+
                  |  subscribe channel_id                                 |
                  v                                                       v
          +-------------------------------------------------------------------+
          |              CHAT PUB/SUB BUS (sharded by channel_id)             |
          |   ingest -> rate-limit/moderation -> fan to subscribed gateways   |
          +-------------------------------------------------------------------+

   MONEY PLANE:  [ PLAYER ] --POST /donate|/subscribe--> [ PAYMENTS SVC ]
                 --idempotent charge--> [ Payment Provider ]
                 --on confirm--> emits event --> CHAT BUS (alert) + Subscription DB

Go-live path. OBS pushes RTMP/SRT to the nearest ingest PoP using a secret stream key. The PoP authenticates the key against the channel, accepts the feed, and hands it to a transcode worker. The worker produces the adaptive ladder and packages it as low-latency HLS (CMAF chunks). Segments and manifest are pushed to an origin/packager; the channel is marked live in the metadata store, which flips the channel page to “live.” Elapsed time to first viewable frame: a couple of seconds.

Watch path (video). Viewer opens the channel. The player fetches the channel’s playback manifest URL and connects to the nearest CDN edge. It pulls the LL-HLS manifest and the latest partial segments, starting playback 2-4 seconds behind the live edge. As bandwidth changes, the player switches renditions (ABR). For a 1M-viewer stream, all 1M players hit CDN edges; the edge has the segment cached (or pulls it once from origin shield), so origin sees one request per segment per PoP, not a million.

Watch path (chat). The same player opens a WebSocket to the nearest chat edge gateway and subscribes to channel_id. The gateway registers the socket as a subscriber of that channel in the pub/sub layer. When anyone posts, the message flows ingest -> rate-limit/moderation -> fan to every gateway that has subscribers for that channel -> each gateway writes to its local sockets. Sub-500ms end to end.

Money path. Viewer clicks donate/subscribe. The client POSTs to the payments service with an idempotency key. Payments charges the provider exactly once, writes the transaction, and on confirmation emits an event. That event does two things: creates a highlighted chat message / on-stream alert (published to the chat bus for that channel) and, for subscriptions, updates the subscription record that grants the badge and emotes. Money and video never share a code path.

Component Deep Dive

The two Twitch-specific monsters are live transcode economics and chat fanout. I also cover low-latency delivery and money idempotency because they are where correctness lives.

1. Live ingest and transcode: surviving the long tail

Naive approach. Every stream that goes live gets a dedicated transcode job on a fixed worker, producing the full ladder (720/480/360/160) from the moment it starts. Workers are statically provisioned for peak.

Where it breaks. You have 150,000 concurrent streams and most have zero or single-digit viewers. Transcoding a full ladder for a stream nobody watches is pure waste, and it is the single biggest line item in the budget - 150,000 transcode units, most doing work no one consumes. Static provisioning for peak means a huge fleet idle most of the day. And a worker crash kills a streamer’s broadcast outright, which violates the NFR.

The fixes, layered:

  • Transcode on demand (lazy ladder). When a stream goes live, only ingest the source passthrough (no transcode) and make it available. Spin up the transcoded renditions (720/480/360) only when a viewer actually requests a lower rendition. A stream with 3 viewers all on good connections watching source costs you ~nothing to transcode. The moment a viewer on 4G needs 480p, a transcode job starts for that rendition. The hot streams (which have viewers on every rendition) run the full ladder; the cold long tail runs source-only. This alone cuts transcode cost by a large multiple because cost now tracks watched renditions, not live streams.
  • GPU transcode pools with fast scheduling. Transcode is embarrassingly parallel per stream. Run a pool of GPU workers (NVENC-style) where one GPU handles many concurrent transcodes. A scheduler assigns (stream_id, rendition) jobs to workers, tracks health, and reassigns on failure. Autoscale the pool on queued-job depth, not a fixed size.
  • Ingest/transcode failover without dropping the broadcast. The ingest PoP buffers a few seconds of source. If the assigned transcode worker dies, the scheduler reassigns the job to a healthy worker and replays the buffered source into it; the viewer sees at most a brief stall, the streamer never gets disconnected. The streamer’s RTMP connection terminates at the ingest PoP, which is HA, not at the transcode worker.
  • Partner vs non-partner tiers. Big/partner channels get the full ladder pre-warmed and priority scheduling; the long tail gets source-only until demand appears. This is a deliberate QoS split that matches cost to value.
Stream goes live
   -> ingest PoP accepts source, marks channel live (source rendition ready)
   -> viewer requests source  => served, 0 extra transcode
   -> viewer requests 480p     => scheduler starts 480p transcode job on GPU pool
   -> transcode worker dies     => scheduler replays buffered source to new worker
   -> stream drops to 0 viewers => transcode jobs torn down after short grace period

2. Low-latency delivery: getting under 5 seconds

Naive approach. Use regular HLS with 6-second segments and a playlist that buffers 3 segments. That is the classic VOD/live-sports setup.

Where it breaks. Classic HLS latency is ~2-3 segments of buffer = 15-30 seconds behind live. For sports that is tolerable; for Twitch it kills the interaction loop - the streamer reacts to chat the viewer sent 25 seconds ago, which feels broken. You cannot have chat that is sub-second and video that is 25 seconds behind.

The fixes:

  • Low-Latency HLS (LL-HLS) with CMAF chunks. Keep segments short (2s) but split each segment into partial chunks (~200-500ms each, CMAF) that are published and fetchable the moment they are encoded, before the full segment is done. The player pulls partials and plays near the live edge, getting to 2-4 seconds glass-to-glass. Blocking playlist reloads let the player ask for the next partial and the server holds the request until it exists, avoiding poll latency.
  • WebRTC for true low-latency mode. For the tightest experience (sub-2s, e.g. interactive streams), deliver over WebRTC instead of HLS. WebRTC is harder to scale to 1M (it is peer-oriented and stateful) so it is reserved for smaller/interactive channels or a premium path; the 1M-viewer firehose stays on LL-HLS over CDN because HTTP segment caching is what makes a million viewers affordable.
  • The sync contract. Embed a stream timestamp / sequence in both the video segments and the chat events so the client can keep them loosely aligned. The product promise is not “zero latency,” it is “video and chat are close enough that reactions make sense,” which sub-5s video plus sub-500ms chat satisfies.
  • CDN tiering so origin never melts. Viewers hit edge PoPs; a PoP that misses pulls from a regional origin shield; the shield pulls from the packager. For a 1M-viewer stream, each new 500ms partial is fetched from the packager once per shield (a few hundred requests), cached, and served to a million players from the edges. Origin request volume is decoupled from viewer count - the core property that makes a hot stream survivable.

3. Chat fanout: one message to a million sockets

Naive approach. Chat messages are rows in a database. To display chat, each client polls GET /chat?channel=X&since=Y every second. New message = INSERT; everyone polls and reads the new rows.

Where it breaks. Polling at 1M viewers on one channel = 1M queries/sec hitting the database just to ask “anything new?”, almost always returning nothing. The database is incinerated. And a message posted now shows up a full second late (poll interval). This does not survive a channel with 10,000 viewers, let alone 1M. Chat is a push, pub/sub problem, not a database query problem.

The fixes:

  • Persistent WebSockets, not polling. Every viewer holds one WebSocket to a chat edge gateway. No polling. A gateway box holds ~100k sockets. 15M concurrent viewers => a fleet of gateways, each subscribing upstream only to the channels its sockets care about.
  • Pub/sub sharded by channel_id. Chat is organized as a topic per channel. When a message is posted, it is published to the channel’s topic. Every gateway that has at least one subscriber for that channel receives the message once and fans it out to its local sockets. So the fanout is a tree: publisher -> N gateways (not N viewers) -> each gateway -> its local sockets. For a 1M-viewer stream spread over, say, 200 gateways, the bus does 200 sends and each gateway does ~5,000 local socket writes. That is how a billion-sends-per-second problem becomes tractable: you never fan out at the bus level, only at the leaf.
  • A dedicated fanout layer for mega-channels. The hottest channels get a dedicated set of fanout nodes arranged in a tree (bus -> fanout tier -> edge gateways) so no single node fans to all 200 gateways. The message replicates down the tree, each level multiplying by a bounded factor.
  • Rate-limit, batch, and shed load. On a 1M-viewer channel, chat is unreadable anyway (thousands of messages/sec scroll past faster than a human reads). So:
    • Inbound rate-limit per user (e.g. 1 message / few seconds for non-mods) kills spam at the source.
    • Batch outbound. Instead of one WebSocket frame per message, each gateway batches messages over a ~100-200ms window and sends them as one frame. This cuts syscall/frame overhead by an order of magnitude on hot channels and the user cannot tell.
    • Sampling / load-shedding on mega-channels. Above a threshold, non-highlighted messages may be sampled (show a representative subset) so the socket write rate stays bounded. Highlighted messages (donations, subs, mod messages) are never dropped. This is an explicit, deliberate degradation: chat on a 1M channel is a vibe, not a transcript, and that is acceptable.
  • Moderation and ordering. Between ingest and fanout sits a moderation stage (banned words, banned users, slow-mode, sub-only mode). Ordering is per-channel best-effort: the bus assigns a per-channel sequence so clients can order within a channel; cross-channel order is irrelevant.
  • Persistence is a side effect, not the path. Messages are optionally written asynchronously to a store (for VOD chat replay and moderation audit) off the hot path. The live delivery path never touches a database.
Post message on channel X
  -> ingest -> rate-limit (drop spam) -> moderation (drop banned)
  -> assign channel seq -> publish to topic(X)
  -> fanout tree: bus -> fanout nodes -> gateways that subscribe X
  -> each gateway BATCHES 150ms -> one frame per socket
  -> (async) append to chat log store for VOD replay

4. Money: donations and subscriptions, exactly once

Naive approach. On “donate,” the client POSTs amount; the server calls the payment provider, then inserts a transaction row, then publishes the alert. Subscriptions: on “subscribe,” charge, insert, grant perks.

Where it breaks. Networks retry. The client times out and retries the donate; you charge twice. The server charges the provider, then crashes before writing the row; money left the viewer’s account but no record exists and no alert fires. Subscriptions get double-charged on a retried renewal. Every one of these is a real-money bug, which is the one thing the NFR says is unacceptable.

The fixes:

  • Idempotency key end to end. The client generates an idempotency key per donation attempt and sends it. The payments service stores (idempotency_key -> result); a retry with the same key returns the same result without charging again. The key is also passed to the payment provider (Stripe-style) so the provider itself dedupes the charge. Double-submit is now a no-op.
  • Transactional outbox. The charge result and the “emit alert” event are written in the same database transaction as the transaction row (an outbox row). A separate dispatcher reads the outbox and publishes the chat alert / updates subscription state at least once, retrying until acked. So “money recorded” and “alert will fire” are atomic; a crash after the charge cannot lose the alert because it is committed with the charge.
  • Provider webhook as source of truth for async confirmation. Card charges can be async. The provider sends a webhook on final confirmation; the payments service reconciles its pending transaction against the webhook. Only a confirmed transaction emits the on-stream alert and grants subscription perks. Webhooks are themselves idempotent (dedupe by provider event id).
  • Subscriptions are state, not events. A subscription is a row with status, current_period_end, renewal. A recurring charge job (or the provider’s billing) attempts renewal; success extends current_period_end, failure moves to a grace/expired state. Perks (badge, emotes, ad-free) are derived from status = active AND now < current_period_end, checked at read time, so there is no cron that must fire exactly on the boundary.
  • Why money is isolated. It has its own strongly-consistent store (a relational DB with real transactions), never touches the video or chat hot path, and only emits events into the chat bus. If chat is overloaded and sheds load, a donation alert is in the “never drop” class and is retried by the outbox dispatcher until delivered.

API Design & Data Schema

Three API surfaces, matching the three planes.

Video / stream control (HTTP/REST):

POST /v1/streams/key/rotate
   -> { stream_key }                      # secret RTMP key, per channel

(ingest is RTMP/SRT push to:  rtmp://ingest.<pop>.twitch/app/<stream_key> )

GET  /v1/channels/{channel_id}
   -> { channel_id, display_name, is_live, title, category,
        viewer_count, playback_url }

GET  /v1/channels/{channel_id}/manifest.m3u8   # LL-HLS master playlist
   -> HLS manifest listing renditions (source/720/480/360/160) + partial segments

POST /v1/channels/{channel_id}/clip
   { start_offset_ms, duration_ms } -> { clip_id, url }

Chat (WebSocket):

WS   /v1/chat
   -> client sends: { op: "SUBSCRIBE", channel_id }
   -> client sends: { op: "SEND", channel_id, body }     # subject to rate-limit/mod
   -> server pushes:{ op: "MSG", channel_id, seq, user, body, badges[], ts }
   -> server pushes:{ op: "ALERT", channel_id, kind: "sub"|"donation",
                      user, amount?, message, ts }        # never dropped
   -> server pushes:{ op: "VIEWERS", channel_id, count }  # periodic, approximate

Money (HTTP/REST, idempotent):

POST /v1/channels/{channel_id}/donate
   Headers: Idempotency-Key: <uuid>
   { amount_bits, message } -> 200 { transaction_id, status: "confirmed"|"pending" }

POST /v1/channels/{channel_id}/subscribe
   Headers: Idempotency-Key: <uuid>
   { tier } -> 200 { subscription_id, status: "active", current_period_end }

POST /v1/webhooks/payments      # from payment provider, idempotent by event_id
   { event_id, type, provider_charge_id, status } -> 200

Data stores - deliberately polyglot, one store per plane’s access pattern:

Channel / stream metadata - NoSQL (wide-column, e.g. Cassandra/DynamoDB-style), keyed by channel_id, read-heavy, eventually consistent, no joins:

Table live_channel (key: channel_id)
   channel_id        STRING  (partition key)
   streamer_id       STRING
   is_live           BOOL
   title             STRING
   category          STRING
   started_at        TIMESTAMP
   ingest_pop        STRING
   playback_url      STRING
   viewer_count      INT      (approximate, updated from presence aggregation)
   Index: GSI on (category, viewer_count) for the browse directory feed.

Chat log (replay/audit only, off hot path) - NoSQL time-series / wide-column, partitioned by (channel_id, time_bucket):

Table chat_log (partition: channel_id # hour_bucket, clustering: seq)
   channel_id   STRING
   seq          BIGINT   (per-channel monotonic)
   user_id      STRING
   body         STRING
   kind         STRING   (chat | alert)
   ts           TIMESTAMP
   TTL: 60 days for normal chat; alerts kept longer.

Subscriptions - SQL (relational, strong consistency), this is money-adjacent state:

Table subscription
   subscription_id    UUID PK
   user_id            BIGINT  (index)
   channel_id         BIGINT  (index)
   tier               SMALLINT
   status             ENUM(active, grace, expired, cancelled)
   current_period_end TIMESTAMP
   provider_sub_id    STRING
   UNIQUE (user_id, channel_id)   # one active sub per viewer per channel

Transactions (donations/charges) - SQL (relational, ACID), the exactly-once core:

Table txn
   transaction_id     UUID PK
   idempotency_key    STRING UNIQUE   # the dedupe guarantee
   user_id            BIGINT
   channel_id         BIGINT
   amount             BIGINT          # in smallest unit / bits
   status             ENUM(pending, confirmed, failed, refunded)
   provider_charge_id STRING
   created_at         TIMESTAMP

Table outbox
   outbox_id     BIGSERIAL PK
   txn_id        UUID
   event_type    STRING
   payload       JSONB
   dispatched    BOOL   (index on dispatched=false)   # transactional outbox

VOD metadata - NoSQL, keyed by vod_id, points at segment blobs in object storage (the YouTube pattern, reused). Live media segments are not in any of these - they live in CDN/origin caches and expire.

Why these choices: metadata and chat are high-volume, read-dominated, partition-friendly, and tolerant of eventual consistency, so NoSQL by channel_id. Money needs multi-row ACID transactions, a unique constraint for idempotency, and an atomic outbox write, so relational SQL. Using one database for all of it would force the money guarantees onto a store that cannot give them, or force the chat firehose onto a store that would die under it.

Bottlenecks & Scaling

Where it breaks first, in order, and the fix for each:

BottleneckWhere it breaksFix
Transcode compute150k live streams, most unwatched, full ladder eachLazy/on-demand transcode (source-only until a rendition is requested); GPU pools autoscaled on queue depth
Hot stream egress1M viewers x 2.5 Mbps = 2.5 Tbps on one channelCDN edge serving; origin shield pulls each segment once per PoP; origin load decoupled from viewer count
Chat fanout1M viewers, 1k msg/s = 1B sends/s on one channelPub/sub sharded by channel_id; fanout TREE (bus -> gateways -> sockets), batch 150ms, rate-limit, sample mega-channels
WebSocket connections15M concurrent socketsHorizontal fleet of edge gateways (~100k sockets each), gateways subscribe only to channels they hold
Viewer-count writes1M viewers each updating a counter = write stormApproximate counting: local per-gateway counts aggregated periodically into live_channel; never a per-viewer DB write
Money double-chargeClient/network retriesIdempotency key end to end + provider-level dedupe + transactional outbox
Latency creepClassic HLS = 15-30s behind liveLL-HLS with CMAF partials (2-4s); WebRTC for sub-2s interactive mode

Sharding keys, stated explicitly:

  • Chat / pub/sub: shard by channel_id. All of a channel’s chat lives on one logical topic, so a gateway subscribes once per channel regardless of viewer count. This is what keeps bus-level fanout = number of gateways, not number of viewers. The risk is a hot channel (1M viewers) overloading the nodes serving that topic - fixed by the dedicated multi-level fanout tree for mega-channels, not by adding more channels per node.
  • Chat log store: partition by (channel_id, time_bucket) so writes for a hot channel spread across time buckets and a single partition never gets a billion rows.
  • Metadata / VOD: partition by channel_id / vod_id - natural, high-cardinality, no hotspot except the live record of a hot channel, which is read from cache not the store.
  • Transactions: shard by user_id (money is user-centric; a user’s charges and idempotency keys colocate), with the idempotency_key unique index enforced per shard.

Hot keys and single points of failure:

  • The hot channel is the whole game. Every hot-path component (transcode, CDN, chat fanout) has a special case for “one channel has a million viewers.” The fixes are consistent: never fan out proportional to viewers at any single node - always insert a tree (CDN tiering for video, fanout tree for chat) so each node’s work is bounded.
  • Ingest PoP is the SPOF for a given broadcast; it is run HA with the streamer able to reconnect to a backup ingest endpoint, and the RTMP connection, not the transcode worker, is the thing kept alive.
  • Payments DB is a consistency SPOF; it is a replicated relational cluster with synchronous replication for the money rows, and the outbox guarantees no event is lost across a failover.
  • Async everywhere off the hot path: VOD transcode, chat persistence, viewer-count aggregation, moderation ML, analytics - all consume from queues, so a backlog in any of them degrades a non-critical feature (replay lag, stale count) rather than the live watch or chat experience.

Wrap-Up

The trade-offs that define this design:

  • Lazy transcode over always-on. I accept a brief delay when the first viewer requests a new rendition, in exchange for not paying to transcode 140,000 streams nobody watches. Cost tracks consumption, not existence.
  • Sub-5s LL-HLS over sub-second WebRTC for the masses. I give up the absolute lowest latency on the 1M-viewer path because HTTP segment caching is the only thing that makes a million viewers affordable; WebRTC is reserved for small/interactive streams where sub-2s matters more than scale.
  • Best-effort, load-shedding chat over perfect delivery. On a mega-channel I will sample and batch and drop ordinary messages, because a 1M-viewer chat is unreadable anyway - but donation and sub alerts are in a never-drop class routed through the transactional outbox.
  • Polyglot storage. NoSQL partitioned by channel_id for the read-heavy eventually-consistent planes (metadata, chat log, VOD); relational SQL for money, where idempotency keys, unique constraints, and the transactional outbox give exactly-once. One store cannot serve both without failing one of them.
  • Eventual everywhere except money. Viewer counts approximate, chat best-effort, video eventually at the edge - and a hard ACID exactly-once core for every rupee that moves.

One-line summary: Twitch is three independently-scaled real-time planes - on-demand live transcode behind a tiered CDN for sub-5s video, a channel-sharded pub/sub fanout tree for chat that batches and sheds load on hot streams, and an idempotent transactional-outbox money plane - joined only at the player and only by the constraint that chat and video must feel simultaneous.