Live comments look like a solved problem until you put a number on it. A celebrity goes live, a million people are watching, and comments are pouring in at ten thousand a second. Every one of those million viewers is supposed to see new comments appear the instant they are posted. Do the multiplication and the system dies on the spot: 10,000 comments/sec times 1,000,000 viewers is 10 billion messages per second if you naively push every comment to every viewer. No fleet on earth moves 10 billion messages a second for a single post.

So this is not really a “how do I store comments” problem. Storing them is easy. The entire design is about the fan-out: how one comment reaches a million screens without the multiplication blowing up, and how you do that when the write side and the read side are both concentrated on a single hot object. That single-object concentration is what makes it different from a chat system, where load spreads naturally across billions of independent conversations. Here everything piles onto one post. Let me build it properly.

Functional Requirements (FR)

In scope:

  • Post a comment. A viewer writes a comment on a live post/video. It is durably stored and gets an ordered position in that post’s comment stream.
  • Real-time delivery. Every viewer currently watching the post sees new comments appear within a second or two, without refreshing.
  • Initial load + live tail. When you open a post you see the most recent N comments (history), then the stream continues live from that point.
  • Ordering. Comments in a post render in a consistent, roughly chronological order for everyone. Two viewers watching the same post see the same comments in the same order.
  • Scroll back / pagination. A viewer can scroll up to load older comments.
  • Reactions / likes on comments. Counts update in near real time (an aggregate, not a per-event push).
  • Graceful degradation under a firehose. When comments arrive faster than a human can read, the system stays up and the UI stays legible instead of trying to render 10,000 lines/sec.

Explicitly out of scope (say this so you own the scope):

  • The video stream itself. Live video ingest, transcoding, and CDN delivery are a separate system. We handle only the comment overlay.
  • Ranking / relevance. Real Facebook ranks comments (top comments, friends first). I will assume mostly chronological with a hook for ranking, and not design the ML ranker.
  • Spam, abuse, moderation ML. I will note where a moderation hook sits but not design the classifier.
  • Rich media in comments, GIFs, mentions resolution. Treated as opaque payload.
  • Notifications (“someone replied to you”). Its own fan-out system.

The one decision that drives everything: this is a read-fan-out-heavy broadcast problem, not a point-to-point messaging problem. In chat, a message goes to one recipient (or a small group). Here one comment goes to a million anonymous viewers. The write rate is high but bounded; the delivery rate is the write rate times the audience, and that product is the beast we have to tame.

Non-Functional Requirements (NFR)

  • Scale: platform-wide ~2B DAU. The design target is the hot post: 1M concurrent viewers, 10K comments/sec sustained on a single post during a live event. Many such events run at once.
  • Latency: comment-to-screen p99 under ~2s for viewers of the same post. Posting a comment should ack the author in under ~200ms so the UI feels responsive; the author sees their own comment optimistically right away.
  • Availability: 99.9%+ on the read/delivery path. If a viewer briefly stops getting new comments, the video keeps playing and the stream reconnects. Comment delivery is important but not life-critical, so we favor availability over strict consistency.
  • Consistency: eventually consistent is fine and correct here. Not every viewer must see every comment (at 10K/sec no human can), and slight per-viewer differences in which comments render are acceptable. What must hold: comments a given viewer does see are in a sensible order, and the author’s own comment always appears for the author.
  • Durability: a posted comment must not be silently lost once acked. It is persisted before ack. But delivery to viewers is best-effort broadcast - a viewer who misses a comment in the live tail can scroll to find it via the durable store; we do not maintain a per-viewer guaranteed queue.
  • Legibility under load: the display rate is capped. Humans read maybe 5-10 comments/sec; the system must coalesce or sample the firehose down to a readable rate rather than trying to deliver all 10K/sec.

That last point is the unlock. Because delivery is best-effort and the display rate is capped, we are allowed to drop and coalesce, which is the only reason the numbers ever close.

Back-of-the-Envelope Estimation (BoE)

Real numbers, because they dictate the architecture.

The naive fan-out (why this is hard):

Hot post: 1,000,000 concurrent viewers
Write rate: 10,000 comments/sec on that one post
Naive push (every comment to every viewer):
  10,000 * 1,000,000 = 10,000,000,000 = 10 billion messages/sec

Ten billion messages/sec for ONE post is a non-starter. Every design decision below exists to kill this number.

First cut: cap the display rate. A human reads at most ~5-10 comments/sec. There is no point delivering more. Coalesce the incoming 10K/sec into a display budget of, say, 20 comments/sec (a mix of chronological + a few “top” ones), pushed in batched windows:

Batch window: 500ms  ->  2 pushes/sec per viewer
Each batch carries up to ~10 comments (20/sec / 2 pushes)
Per-viewer delivery: 2 pushes/sec  (not 10,000)
Post-wide delivery: 2 * 1,000,000 = 2,000,000 pushes/sec for the hot post

We just went from 10 billion to 2 million per second - a 5,000x reduction - purely by batching and capping display. 2M/sec is still large but now tractable across a fan-out fleet.

Two-level fan-out (kill the per-viewer target count):

Do not have one dispatcher push to 1M viewers. Push to the connection servers that hold those viewers, and let each server locally fan out:

1M viewers spread across ~1,000 connection servers  -> ~1,000 viewers/server
Dispatcher pushes each 500ms batch to ~1,000 servers = 2,000 pushes/sec (dispatcher side)
Each connection server local-fans-out to its ~1,000 local viewers:
  1,000 viewers * 2/sec = 2,000 local writes/sec per server for this post

The dispatcher’s job shrinks from 1M targets to ~1,000 targets. Each connection server does a bounded local fan-out. Nobody has to do a million-way anything.

Bandwidth (hot post):

Batch ~ 10 comments * ~300 bytes = ~3KB per push
2,000,000 pushes/sec * 3KB = ~6 GB/sec for one hot post
Spread across 1,000 connection servers = ~6 MB/sec each  -> trivial per box

Write path (ingestion):

10,000 comments/sec on one post.
Comment size ~ 300 bytes (id, post_id, author_id, text, ts).
Storage for one 2-hour live event: 10K * 7,200s * 300B ≈ 21.6 GB for one post.
Platform-wide comment writes (all posts): assume ~500K comments/sec average,
  peak ~1.5M/sec  ->  1.5M * 300B = ~450 MB/sec of writes  -> sharded store.
Daily comments platform-wide: ~500K/sec * 86,400 ≈ 43B comments/day
  * 300B ≈ 13 TB/day raw.

Concurrent connections:

Total concurrent viewers across all live posts: order 50-100M.
At ~500K connections per connection server (thin, IO-bound, SSE/WS):
  100M / 500K ≈ 200 connection servers to hold sockets platform-wide,
  scaled up for hot events.

Takeaways: the naive 10B/sec is defeated by display-rate capping + batching (5,000x) and two-level fan-out (1M targets -> 1,000 targets). Storage and raw connection counts are ordinary; the fan-out multiplication is the whole game.

High-Level Design (HLD)

Two planes. A write plane ingests comments, persists them, and publishes them into a per-post stream. A read/fan-out plane holds viewer connections and broadcasts each post’s stream to its subscribers. A pub/sub layer with one logical topic per post glues them: writers publish to topic:post_id, and the fan-out dispatchers subscribe on behalf of the connection servers that hold viewers.

   Comment authors                         Viewers (1M on hot post)
        │ POST /comments                        │ SSE / WS subscribe
        ▼                                        ▼
  ┌───────────────┐                       ┌──────────────────┐
  │  API Gateway  │                       │  Load Balancer    │
  └──────┬────────┘                       └────────┬──────────┘
         │                                          │
  ┌──────▼─────────┐                     ┌──────────▼───────────┐
  │ Comment Write  │                     │  Connection Servers   │ ~thin, IO-bound
  │   Service      │                     │  (hold SSE/WS sockets;│ each ~500K sockets
  │ (validate,     │                     │   local fan-out)      │
  │  id, persist)  │                     └──────────┬───────────┘
  └───┬───────┬────┘                                │ subscribe(post_id)
      │persist│publish                              │
      ▼       ▼                          ┌──────────▼───────────┐
 ┌─────────┐ ┌───────────────┐           │  Subscription        │
 │ Comment │ │  Pub/Sub bus   │◀──────────│  Registry            │
 │ Store   │ │  topic:post_id │  fan-out  │  post_id -> {conn    │
 │(Cassandra│ │ (Kafka/Redis  │  dispatch │  servers with subs}  │
 │ shard by │ │  streams)      │──────────▶└──────────────────────┘
 │ post_id) │ └───────┬───────┘  push batches to those conn servers
 └─────────┘         │
                     ▼
             ┌────────────────┐
             │ Fan-out Workers │  read topic, batch/coalesce per 500ms,
             │ (dispatchers)   │  push to subscribed conn servers
             └────────────────┘

Write flow (author posts a comment):

  1. Author sends POST /posts/{id}/comments with the text over normal HTTPS (writes are low-rate per user; no socket needed to send).
  2. Comment Write Service validates, runs a fast moderation hook, assigns a time-sortable comment_id (Snowflake: timestamp + shard + sequence), and persists to the Comment Store (sharded by post_id). Durability before ack.
  3. It acks the author (~200ms) with the canonical comment_id; the author’s client already showed the comment optimistically.
  4. It publishes the comment to the pub/sub topic topic:{post_id}. The write path is now done; delivery is asynchronous.

Read / fan-out flow (a comment reaches 1M viewers):

  1. On opening a post, a viewer does GET /posts/{id}/comments?limit=50 for the initial page, then opens a stream (SSE or WS) to a connection server, which subscribes the viewer to post_id and records in the Subscription Registry that “this connection server has viewers of post_id.”
  2. Fan-out workers consume topic:{post_id}. They do not push per comment. They batch a 500ms window and coalesce it down to the display budget (~20 comments/sec: newest chronological plus a few high-signal ones).
  3. For each batch, a worker looks up the Subscription Registry: which connection servers currently hold viewers of this post? It pushes the batch once to each such server (~1,000 of them for the hot post), not to each viewer.
  4. Each connection server receives the batch and locally fans it out down every open socket subscribed to that post. That is the only place a million-ish writes actually happen, and it is split across ~1,000 boxes so each does ~1,000.
  5. Viewers append the batch to their comment view. When a viewer scrolls away or closes, the connection server unsubscribes them; when a server’s last viewer of a post leaves, it drops out of that post’s registry entry.

The key insight: the fan-out is a tree, not a star. Writer -> pub/sub topic -> ~1,000 connection servers -> ~1,000 viewers each. No single component ever addresses a million endpoints, and the display-rate cap means we push ~2/sec per viewer instead of 10,000.

Component Deep Dive

The hard parts, naive-first then evolved: (1) the fan-out explosion, (2) ingesting 10K writes/sec onto one hot object, (3) connection + subscription management, (4) ordering, late joiners, and the display firehose.

1. The fan-out explosion (the real problem)

Naive approach: comment triggers a direct push to every viewer. The write service, after saving a comment, loops over the 1M viewers of the post and writes it to each socket.

on new comment c for post p:
    for viewer in viewers_of(p):     # 1,000,000 iterations
        push(viewer.socket, c)       # x 10,000 comments/sec = 10^10/sec

Where it breaks:

  • 10 billion pushes/sec for one post. The number is absurd on its face. Even at 1KB each that is terabytes/sec of a single post’s traffic.
  • The write service becomes a fan-out bottleneck. It cannot both ingest 10K/sec and do a million-way loop per comment. The two responsibilities fight.
  • It delivers more than anyone can read. Pushing 10K comments/sec to a screen is pointless; the human sees a blur. You are spending the entire budget to render illegible noise.

First evolution: pub/sub with a topic per post. Decouple write from delivery. The write service just publish(topic:p, c). A separate fan-out tier subscribes and delivers. This is the right decoupling but does not by itself fix the multiplication - a dumb subscriber still faces 1M targets times 10K/sec.

Second evolution: two-level fan-out (the tree). Do not address viewers directly. Address the connection servers that hold them. The Subscription Registry maps post_id -> {set of connection servers with >=1 viewer of this post}. The dispatcher pushes each update once per connection server (~1,000), and each connection server does the final local fan-out to its own sockets. The million-way work is partitioned across the fleet, ~1,000 per box.

dispatcher:  servers = registry.servers_for(p)   # ~1,000, not 1,000,000
             for s in servers: push(s, batch)     # 1,000 pushes
conn server: for sock in local_subs[p]: write(sock, batch)   # ~1,000 local

Third evolution: batch + coalesce to the display budget (the multiplier killer). Even 1,000 servers times 10K comments/sec is 10M pushes/sec of tiny messages - wasteful and pointless because nobody can read 10K/sec. The fan-out worker holds a 500ms window per post, collects the ~5,000 comments that arrived, and coalesces them to a display budget of ~20 comments/sec (10 per 500ms batch): the newest chronological ones plus a few flagged high-signal (many reactions, verified author). Everything else is dropped from the live tail - it still lives in the durable store for anyone who scrolls.

Before: 10,000 comments/sec  * 1,000 servers = 10,000,000 pushes/sec (illegible)
After:  2 batches/sec        * 1,000 servers =     2,000 pushes/sec
        each batch ~10 comments, ~3KB        -> readable + 5,000x cheaper

This is the crux and it is only legal because the NFRs say delivery is best-effort and eventually consistent. We are allowed to not deliver every comment to every viewer, because no human could consume them anyway, and the durable store is the fallback for anyone who wants the full history.

Stage Pushes/sec (hot post) Why it fails / works
Naive per-viewer 10,000,000,000 10B/sec, impossible
Pub/sub, dumb subscriber ~10,000,000,000 decoupled but still 1M targets
Two-level fan-out (tree) ~10,000,000 1,000 targets, but still all comments
+ batch/coalesce to budget ~2,000 readable, durable-store fallback

2. Ingesting 10K writes/sec onto one hot object

Naive approach: one row per post you update, or a per-post counter with a lock. Store a comment_count on the post row and UPDATE posts SET comment_count = comment_count + 1 per comment, and insert into a comments table keyed by post with an auto-increment sequence.

Where it breaks:

  • Row-level lock contention. 10K/sec incrementing one posts.comment_count row serializes on that row’s lock. A single row cannot absorb 10K writes/sec; you get lock queues and latency spikes.
  • Single-partition hot spot. All 10K/sec land on the partition that owns this post_id. In a naively sharded store this one partition’s node is saturated while the rest of the fleet idles.
  • Monotonic sequence generation. A per-post AUTO_INCREMENT or a “get next seq” transaction is a serialization point - the same contention in a different place.

The fix: append-only writes, decentralized IDs, and sharded counters.

  • Append-only, no update-in-place. Comments are immutable inserts, never updates. There is no row being contended; each comment is a fresh key. Writes never block each other.
  • Snowflake IDs, not a central sequence. Each write service instance mints a comment_id = [41-bit ms timestamp | 10-bit instance id | 12-bit per-ms sequence]. IDs are globally unique and time-sortable without any coordination - no lock, no round-trip to a sequence service. Ordering is derived from the ID’s timestamp bits, giving a total order good enough for display (ties broken by instance id).
  • Sub-shard the hot partition. Sharding purely by post_id still concentrates a hot post on one partition. Add a bucket to the key: partition = (post_id, comment_id_time_bucket) or (post_id, hash(comment_id) % N). The hot post’s writes now spread across N physical partitions instead of hammering one. On read, scatter-gather the N buckets for a time range and merge by comment_id (cheap; they are already time-sortable).
  • Sharded / approximate counters. The visible “comment count” uses M counter shards per post (count:{post_id}:{0..M}), each incremented by a random shard, summed on read. This turns one hot counter into M cool ones. The count can be slightly approximate/eventually consistent, which the NFRs allow.
write(post p, text):
    id = snowflake()                       # no coordination
    bucket = hash(id) % N
    store.insert((p, bucket), id, text)     # append-only, spread across N
    counter.incr("count:%s:%d" % (p, id % M))   # sharded counter
    publish("topic:%s" % p, {id, text})     # into fan-out

The write path is now free of single-row locks and single-partition hot spots: every comment is an independent append with a coordination-free ID, spread across N buckets and M counter shards.

3. Connection and subscription management

Naive approach: WebSockets for everyone, one giant subscription table viewer -> post. Every viewer holds a bidirectional WS; a central table lists every viewer’s subscription; to fan out you scan it.

Where it breaks:

  • Full-duplex is overkill for a mostly-read stream. Comments flow server->client 99% of the time; a viewer posts rarely. Paying for a stateful bidirectional WS per viewer, plus its keepalive complexity, is more than the read-heavy shape needs.
  • A per-viewer subscription table is 1M rows for one post and the fan-out has to scan or index a million rows per push. That is the star topology again, just in a table.

The fix: SSE for the read stream, HTTP POST for writes, and a coarse post -> server registry.

  • Server-Sent Events (SSE) for delivery. SSE is a single long-lived HTTP response the server streams events down. It is unidirectional (server->client), auto-reconnects with a Last-Event-ID header (built-in resume), and is far lighter than WS for a broadcast read stream. Viewers post comments over ordinary POST requests. (Use WS only if you also need low-latency client->server signaling like live typing; for comments, SSE is the better, simpler fit.)
  • The subscription registry is coarse, keyed by post, valued by server - not by viewer. A connection server, when it gains its first viewer of post p, registers itself: subs:{p} gains server_id. When it loses its last viewer of p, it removes itself. The registry is post_id -> small set of connection server ids (hundreds to low thousands for a hot post), not a million viewer rows. The per-viewer subscription lives locally on the connection server in an in-process map post_id -> [sockets], where the final fan-out is a cheap local loop.
Registry (Redis):   subs:{post_id} -> SET{conn_server_id, ...}   # coarse
Connection server:  local map  post_id -> [socket, socket, ...]  # in-process

on viewer subscribe(p) at server S:
    S.local[p].append(socket)
    if len(S.local[p]) == 1: registry.sadd("subs:%s"%p, S.id)   # first here
on viewer leave:
    S.local[p].remove(socket)
    if empty: registry.srem("subs:%s"%p, S.id)                  # last one gone

dispatcher push(p, batch):
    for sid in registry.smembers("subs:%s"%p):   # hundreds/low-thousands
        send_to_server(sid, p, batch)

Connection servers are thin and IO-bound (epoll/edge-triggered, Go/Rust/BEAM), holding ~500K sockets each with a few KB per connection. They do no business logic - hold sockets, keep the local subscription map, run the local fan-out loop. If one crashes, its viewers’ SSE clients auto-reconnect (with jitter) to another server via the LB, which re-subscribes them and resumes from Last-Event-ID.

4. Ordering, late joiners, and the display firehose

Naive approach: push comments in whatever order they finish writing, and assume everyone is watching from the start. No sequence discipline; new viewers just start seeing the live tail.

Where it breaks:

  • Out-of-order rendering. Two comments written on different shards can arrive at a viewer out of timestamp order, so the thread looks scrambled and two viewers see different orders.
  • Late joiners see a void. Someone opening the post mid-event with only the live tail sees comments referencing things they never got. They need the recent history first.
  • The firehose is unreadable. Even coalesced to 20/sec, a wall of comments scrolling by is noise; and a viewer who scrolls up to read wants the stream to pause, not yank them back down.

The fix: order by comment_id, seed with history then attach the live tail, and manage the tail on the client.

  • Total order from time-sortable IDs. Every comment carries its Snowflake comment_id. Both the initial page and the live batches are sorted by comment_id. The client merges incoming batches into its list by id, so regardless of network arrival order the render order is stable and identical across viewers (for the comments they share). This is eventual, per-viewer-consistent ordering without any global lock.
  • Initial load + live tail handoff. On open: GET /posts/{id}/comments?limit=50 returns the newest 50 with a cursor = the highest comment_id returned. The client then opens the SSE stream with Last-Event-ID = that cursor. The connection server delivers only comments with id > cursor, so there is no gap and no duplication between the history page and the live tail. A late joiner gets the same seamless seed-then-stream.
  • Client-side tail control. The client caps its own render rate and buffers. If the user scrolls up to read older comments, the client pauses auto-scroll and shows a “N new comments” pill, buffering the live batches; on scroll-to-bottom it flushes. This keeps the UI legible no matter how hard the firehose blows, and it means a slow client naturally sheds load (it just drops buffered batches beyond a cap - fine, because the durable store has them).
  • Backpressure / slow consumers. A connection server bounds each socket’s send buffer. If a viewer’s connection is slow and the buffer fills, the server drops the oldest batches for that socket rather than blocking the whole local fan-out. One slow phone never stalls delivery to the other 999 viewers on that box. Best-effort delivery makes this drop legal.

The mental shift for the whole read side: the live tail is a lossy, coalesced broadcast; the durable store is the source of truth. Anything dropped in the interest of legibility or backpressure is recoverable by a range read. That is what lets every layer shed load without violating a guarantee.

API Design & Data Schema

Writes and history are REST; the live tail is a streaming SSE endpoint.

REST + streaming endpoints

POST /api/v1/posts/{post_id}/comments
  body: { "text": "...", "client_dedup_id": "c-abc" }
  -> validate, moderate, persist, publish
  Response 201: { "comment_id": "0x7f3a...", "created_at": 1721... }

GET /api/v1/posts/{post_id}/comments?limit=50&before=<comment_id>
  -> cursor-paginated history (scroll back or initial load)
  Response 200: { "comments": [...], "next_cursor": "<comment_id>" }

GET /api/v1/posts/{post_id}/comments/stream        (Server-Sent Events)
  headers: Last-Event-ID: <comment_id>             (resume point; optional)
  -> text/event-stream; server pushes coalesced batches:
     event: batch
     id: 0x7f3a...            (highest comment_id in this batch -> resume cursor)
     data: { "comments": [ {id, author, text, ts, reactions}, ... ] }

POST /api/v1/comments/{comment_id}/reactions   { "type": "like" }
  -> increments a sharded reaction counter (aggregate, not per-event push)

Pagination is cursor-based on comment_id, never offset - comments arrive constantly, so offsets would skip or duplicate. Because comment_id is time-sortable, the cursor is a stable position in the stream and doubles as the SSE resume point.

Data store choices

1. Comment Store - NoSQL wide-column (Cassandra / ScyllaDB / DynamoDB). The access pattern is “append a comment” and “range-scan a post’s recent comments by id,” at up to 10K writes/sec on one post and millions/sec platform-wide, with no joins and no cross-row transactions. That is exactly a distributed wide-column store: high write throughput, horizontal scale, tunable consistency. A single relational DB cannot take 10K writes/sec on one logical object without lock contention.

Table: comments
  post_id     STRING   PARTITION KEY (part 1)   -- co-locate a post's comments
  bucket      INT      PARTITION KEY (part 2)   -- hash(comment_id) % N, spreads hot post
  comment_id  BIGINT   CLUSTERING KEY DESC      -- Snowflake, time-sortable order
  author_id   BIGINT
  text        TEXT                              -- <= a few hundred bytes
  created_at  TIMESTAMP
  status      TINYINT                           -- visible / hidden (moderation)
  Shard by:   (post_id, bucket)
  Read:       scatter-gather N buckets for "post p, id < cursor", merge by comment_id
  Write:      append-only insert, no update-in-place

Sharding by (post_id, bucket) keeps a post’s comments together and spreads a hot post’s write load across N physical partitions, defeating the single-hot-partition problem. Reads scatter-gather the N buckets and merge - cheap because rows are already time-ordered by comment_id.

2. Comment / reaction counters - sharded counters (Redis or a counter table). M shards per post, summed on read, eventually consistent. Avoids one hot counter row.

count:{post_id}:{0..M}   -> integer, incr a random shard, SUM on read
react:{comment_id}:{0..M} -> integer, same pattern

3. Subscription Registry - Redis (in-memory, replicated). subs:{post_id} -> SET{conn_server_id} with a heartbeat TTL per member so a crashed connection server’s entry self-expires. Small, hot, ephemeral, rebuildable (servers re-register on reconnect) - the perfect in-memory fit.

4. Pub/Sub bus - Kafka or Redis Streams, one topic (or partitioned topic) per hot post. Durable enough to let fan-out workers batch/replay a window; partitionable so a single hot post’s stream can be split across multiple workers.

Why NoSQL + Redis + a log over SQL everywhere: the workload is partition-scoped, append-only, transaction-free, write-heavy, and read-fanned-out, demanding horizontal scale to billions of rows and an in-memory tier for routing. SQL’s joins and ACID transactions are features we do not need, and its single-hot-row/single-writer ceiling is exactly the wall a hot post would hit. There is no money ledger here to justify strong relational consistency.

Bottlenecks & Scaling

Where it breaks first, in order, and the fix for each:

1. Fan-out multiplication (breaks first, and hardest). 10K comments/sec x 1M viewers = 10B pushes/sec naive. Fix: the tree, not the star. Pub/sub topic per post -> ~1,000 connection servers (two-level fan-out) -> ~1,000 local sockets each. Plus batch a 500ms window and coalesce to a ~20 comments/sec display budget, turning 10M/sec of pointless pushes into ~2,000/sec of readable batches. This is a 5,000x cut and it is only legal because delivery is best-effort with the durable store as fallback.

2. Hot write partition on the post. All 10K writes/sec land on one post_id partition. Fix: sub-shard the key (post_id, bucket = hash(comment_id) % N) so writes spread across N partitions; append-only inserts (no row locks); Snowflake IDs (no central sequence); sharded M-way counters for the visible count. Reads scatter-gather the N buckets and merge by time-sortable id.

3. Hot topic on the pub/sub bus. A single Kafka partition for the hot post caps throughput and one fan-out worker cannot serve 1M viewers. Fix: partition the post’s topic across P partitions and run P fan-out workers, each owning a slice of the subscribed connection servers. The topic’s ordering-within-partition is fine because the client re-sorts by comment_id anyway. This shards the fan-out tier itself.

4. Connection scale and the event-start thundering herd. A million viewers try to connect in the same few seconds when the stream goes live. Fix: thin IO-bound connection servers (~500K sockets each), horizontal scale, and jittered reconnect/connect backoff so clients do not synchronize a herd. The LB spreads new connections; SSE’s Last-Event-ID makes reconnect cheap (resume, not full reload).

5. Slow consumers / backpressure. A slow phone’s send buffer fills on a connection server. Fix: bounded per-socket buffers; drop oldest batches for that one socket rather than blocking the local fan-out. One slow client never stalls the other 999 on the box. Best-effort delivery permits the drop; the client can scroll to backfill from the store.

6. Late joiners and gaps. Someone opens the post mid-event. Fix: history page (GET ...?limit=50) seeds the newest comments and returns a cursor; the SSE stream resumes from that cursor (id > cursor), so there is no gap and no duplicate between history and live tail. Same path handles reconnects.

7. Hot key on the counter. The visible comment/reaction count on a viral post. Fix: M-way sharded counters summed on read; the count is eventually consistent and may be shown approximately (“12K comments”), which is fine.

8. Shard keys, stated plainly. Comment Store -> (post_id, bucket) (co-locate a post, spread its hot writes). Pub/Sub -> partitioned topic:{post_id} (split the hot fan-out). Subscription Registry -> post_id (point lookup of the server set). Sharding comments by created_at alone would be the classic mistake - it piles all current writes onto the “now” shard, a permanent moving hot spot. Keying by (post_id, bucket) spreads writes while keeping a post co-located and time-ordered.

9. Single points of failure. Connection servers are interchangeable (viewer state is a reconnect away; the registry, not the box, tracks who has whom). Write service and fan-out workers are stateless behind the LB / consumer groups. Store, bus, and registry are replicated. Losing any one box costs a slice of viewers a jittered reconnect and resync, not lost comments (those are persisted before ack).

10. Multi-region for a global audience. A live event has viewers worldwide. Fix: connection servers and fan-out workers per region; the write service publishes into a regional bus that replicates the post’s stream across regions asynchronously. Each region fans out to its own viewers locally, so a comment crosses the ocean once (region-to-region replication) instead of a million times. Because ordering is per-post-eventual (clients sort by comment_id) and delivery is best-effort, async cross-region replication is exactly right - no global consensus needed.

Wrap-Up

The trade-offs that define this design:

  • Best-effort, coalesced broadcast over guaranteed per-viewer delivery. We accept that not every viewer sees every comment - no human could read 10K/sec anyway - and cap the display to ~20/sec in 500ms batches. That single decision turns 10 billion pushes/sec into ~2,000, and it is safe because the durable store backs every comment for anyone who scrolls.
  • A fan-out tree, not a star. Writer -> pub/sub topic -> ~1,000 connection servers -> ~1,000 local sockets each. No component ever addresses a million endpoints; the million-way work is partitioned across the fleet.
  • Append-only writes with coordination-free IDs and sub-sharded hot partitions. Snowflake ids give order without a central sequence, (post_id, bucket) spreads a hot post’s 10K writes/sec across N partitions, and sharded counters kill the hot-count row - so the single-hot-object write side never contends.
  • SSE + coarse post -> server registry over WS + per-viewer tables. The stream is read-heavy and unidirectional, so SSE with Last-Event-ID resume is lighter than WebSockets, and a registry keyed by post (valued by server) keeps the fan-out lookup at hundreds of entries instead of a million.
  • Eventual consistency and async cross-region replication. Clients re-sort by comment_id, so we never need a global lock or consensus; a comment crosses regions once and each region fans out locally.

One-line summary: a read-fan-out-heavy broadcast system that beats the 10-billion-pushes-a-second wall by coalescing the firehose to a legible display budget and delivering it through a pub/sub-fed fan-out tree (topic per post -> ~1,000 thin connection servers -> ~1,000 SSE sockets each), while the write side ingests 10K comments/sec onto one hot post via append-only, coordination-free Snowflake ids and sub-sharded partitions, with the durable Cassandra store as the source of truth behind an intentionally lossy, best-effort live tail.