Design Discord (Voice Chat with Low Latency) - System Design

People hear “build Discord voice” and reach for the same WebRTC tutorial they would use for a video call: two peers, an SDP exchange, media flows, done. Discord voice is a different animal from Zoom, and the differences are what make the design interesting. There is no “meeting” you schedule and join. There is a persistent voice channel sitting inside a server, always there, and users drop in and out of it the way you walk into a room. The latency budget is tighter - voice-only, interactive, gaming-adjacent, so the target is sub-100ms mouth-to-ear, not Zoom’s 200ms. And the pathological case is not a 1000-person webinar with one speaker; it is a hot channel with 10,000+ people who can all unmute and talk, a community stage or a raid, where the fan-out is genuinely many-to-many.

I have written about WebRTC peer-to-peer elsewhere, and about designing Zoom. This is deliberately a different cut: voice-only, persistent channels, a stricter budget, and a fan-out pattern driven by servers and channels rather than ad-hoc meetings. The core problem is moving a lot of small, latency-critical audio packets through a channel of N speakers under 100ms, while the channel is a long-lived object that most of the time has 3 people in it and occasionally has 10,000.

Let me do it properly.

Functional Requirements (FR)

In scope:

• Join/leave a voice channel. A user clicks a voice channel inside a server and is immediately connected: sending their mic audio and hearing everyone else’s. No scheduling, no lobby, no host. The channel persists empty; joining is instant and cheap because it happens constantly.
• Real-time voice. Every connected user publishes their microphone (Opus audio) and receives the audio of the other connected users, mixed or forwarded, at conversational latency. This is the hot path.
• Mute / deafen / push-to-talk. Client-side and server-enforced mute (others stop hearing you), deafen (you stop hearing others, mic also cut), and push-to-talk (only transmit while a key is held). These are control operations plus a gate on the media path.
• Speaking indicators. The UI shows who is currently speaking (the green ring). Driven by voice activity, pushed to all channel members in near real time.
• Server/channel structure. Voice channels live inside servers (Discord calls them guilds). A user is a member of many servers; each server has many channels; permissions (who may join/speak) are per channel/role.
• Voice regions. A channel is pinned to or auto-selects a geographic region so all its participants route through nearby media servers, keeping latency low.
• Large stage channels. A one-to-many mode (stages/town halls) where a few speakers broadcast to thousands of listeners, and listeners can be promoted to speakers.

Explicitly out of scope (state this to bound the problem):

• Text chat, message history, attachments. A separate, durable, write-heavy system (store-and-fan-out of messages) that shares the gateway but not the media path. Mentioned, not deep-dived.
• Video and screen share (Go Live). Real in Discord and essentially the Zoom SFU problem; here I focus on voice, which is the stricter-latency, higher-concurrency core. I note where video bolts on.
• Server/guild management, roles, invites, moderation. CRUD and policy in front of the voice system; assume an authorized user_id with resolved permissions reaches us.
• Full auth/identity. Assume authenticated gateway sessions carry a resolved user_id.
• End-to-end encryption of voice. Discord has shipped E2EE voice (DAVE); the default I design is DTLS/SRTP hop-by-hop so the media server can route and mix. I note the E2EE trade-off.

The decision that drives everything: voice does not flow peer-to-peer; each user sends one Opus stream over UDP to a nearby media server (a selective forwarder), which forwards the active speakers’ audio to everyone else. Persistent channels, voice regions, and the 10K-speaker case all hang off that primitive.

Non-Functional Requirements (NFR)

• Scale: 150M monthly active users. Assume ~15M concurrent users at peak, and a large fraction of them in voice at any time (Discord is voice-first). Call it several million concurrent users in voice channels across millions of active channels. The distribution is extreme: the vast majority of channels hold 2-10 people, but the long tail has channels with 10,000+ concurrent participants, and that tail is where the design is tested.
• Latency: mouth-to-ear under ~100ms for interactive voice. Humans notice voice lag fast, and Discord’s gaming use means a 100ms target, not 200ms. After network RTT to a nearby region (single-digit to low-tens of ms each way), the media server has only a few tens of ms to receive, select, and forward. This is a hard real-time budget.
• Availability: 99.99%+. Voice dropping mid-conversation is a severe, highly visible failure. The media plane must degrade gracefully (drop a late packet, never the whole channel) and reconnect fast on node failure.
• Consistency: voice is best-effort and lossy. A late audio packet is useless and is discarded, never retransmitted; audio is UDP/SRTP, not TCP. The only strongly-consistent state is channel membership and voice control (who is connected, who is muted, who may speak), which is small and lives in the gateway/state store. Media has no consistency requirement beyond “recent.”
• Durability: live voice is disposable; nothing is stored by default. The durable state is small: server/channel structure, membership, permissions. (Text messages are durable but out of scope.) No voice recording in the default design.

Back-of-the-Envelope Estimation (BoE)

Voice is about packet rate and bitrate, not request QPS. Let me ground it.

Users and channels:

Monthly active users:        150,000,000
Peak concurrent users:       ~15,000,000   (assume ~10% online at peak)
Fraction in voice at peak:   assume ~40%   -> ~6,000,000 concurrent in voice
Avg voice channel size:      ~5 participants
Concurrent active voice channels: 6M / 5 ≈ 1,200,000 channels
Hot channels (1K-10K+):      a tiny fraction of channels, each a heavy unit

Per-stream audio (the fundamental unit):

Opus voice:        ~40-64 Kbps  (call it ~50 Kbps)
Packetization:     20 ms frames -> 50 packets/sec per publisher
Packet size:       ~150-200 bytes each (payload + SRTP + UDP/IP headers)

The packet rate matters as much as the bitrate. 50 packets/sec/publisher means a media node handling thousands of streams is processing hundreds of thousands of small packets per second - it is packet-processing-bound (SRTP decrypt, header parse, forward, re-encrypt), not compute-bound, as long as we do not transcode.

Ingress (uplink into the media server) - one copy per publisher:

Each publisher uploads ONE Opus stream, not N copies.
  6M voice users * ~50 Kbps ≈ 300 Gbps of audio ingress at peak
  (small compared to video; voice is cheap to ingest)

Egress (downlink out) - where fan-out bites:

In a channel of N where M are actively speaking, each of the N users receives
the M active streams. Total egress ≈ N * M streams.

Small channel (N=5, M=2):      5 * 2 = 10 forwarded streams. Trivial.
Hot channel (N=10,000):        if all 10K could be heard at once it would be
   10,000 * 10,000 = 100,000,000 forwards per channel. Impossible and pointless -
   no human parses 10,000 simultaneous voices.

That last line is the whole ballgame for the hot case: you never forward 10,000 voices; you forward the top few active speakers. Opus audio is naturally sparse (people take turns, and silence/voice-activity gating means most mics send nothing most of the time), so the effective M is small even when N is huge. More in the deep dive.

Media server capacity:

A voice media node (selective forwarder) handles roughly ~1-2 Gbps and, more to
the point, a few hundred thousand packets/sec. At ~50 Kbps/stream that is on the
order of ~20,000-40,000 forwarded streams per node, packet-rate permitting.
To carry millions of concurrent voice users with regional placement:
  thousands of media nodes globally, placed in edge regions near users.

Gateway / signaling load:

Join/leave/mute/speaking are tiny control messages over a persistent WebSocket.
  Each of 15M concurrent users holds one gateway WebSocket.
  Voice join/leave is extremely frequent (people hop channels constantly):
  assume ~1M voice-channel transitions/min at peak ≈ ~17K/sec, 5x bursts.
  Speaking-indicator events: voice-activity on/off, coalesced, per speaker.

The numbers say three scaling problems: a persistent gateway/state plane (15M WebSockets, tiny messages, strongly-consistent per-server channel state), a voice media plane (UDP/SRTP selective forwarders, edge-deployed, 100ms-bound, packet-rate-bound), and the hot-channel fan-out problem (a single channel with 10K+ members that no single node and no naive forwarding can serve). We design each, and the hot channel is where the interview is won.

High-Level Design (HLD)

Discord voice is two planes that must stay separate: a gateway/control plane (the persistent WebSocket that carries presence, channel membership, mute state, and the handshake that tells a client where to send voice) and a voice media plane (the actual Opus packets over UDP/SRTP through edge selective forwarders). The gateway is TCP/WebSocket and cares about correctness and ordering of control events; the media plane is UDP and cares about microseconds. Mixing them is the classic mistake.

   ┌──────────────────────────────────────────────────────────────────┐
   │  Clients (desktop / mobile / web)                                 │
   │   - ONE persistent Gateway WebSocket (presence, channel state)    │
   │   - UDP/SRTP voice to an assigned Voice Media node (per channel)  │
   └───────┬───────────────────────────────────────┬───────────────────┘
           │ (A) gateway: WSS                       │ (B) voice: UDP/SRTP
           ▼                                        ▼
   ┌────────────────────┐   voice-server-update   ┌────────────────────────────┐
   │   Gateway Service   │──(token + endpoint)────▶│   Voice Media Fleet         │
   │  (stateful WS fan-  │◀───speaking/stats───────│  (per-region selective      │
   │   out per guild)    │                         │   forwarders; no transcode) │
   │  - presence         │                         └───────┬────────────┬────────┘
   │  - channel members  │                                 │            │
   │  - mute/deafen       │   channel routing              │ node <->   │ voice
   └───────┬─────────────┘                                 │ node relay │
           │ guild/channel state                           │ (hot chan) ▼
           ▼                                        ┌─────────────┐  ┌──────────────┐
   ┌────────────────────┐                           │ Media node   │  │ Voice region  │
   │  Guild State Store  │                           │ <-> Media    │  │ selector /    │
   │  (per-guild shard:  │   Routing directory       │ node mesh    │  │ RTC endpoint  │
   │  members, channels, │   (Redis): channel ->      │ (fan-out     │  │ discovery     │
   │  roles, voice state)│   region + media nodes      │  tree)       │  └──────────────┘
   └────────────────────┘                           └─────────────┘

Request flow - joining a voice channel and getting audio (the hot path):

1. The client already holds a gateway WebSocket (opened at app start; carries presence for all the user’s servers). To join voice, it sends VOICE_STATE_UPDATE {guild_id, channel_id} over the gateway.
2. The gateway validates permissions against the guild state (may this user join/speak in this channel?), updates the channel’s voice membership, and selects a voice region for the channel (its pinned region, or auto-selected from where the first joiners are). It picks a media node in that region (least-loaded, tracked in the routing directory).
3. The gateway replies with VOICE_SERVER_UPDATE {endpoint, token} - the address of the assigned media node and a short-lived token. This is the handoff from control plane to media plane.
4. The client opens a separate voice connection to that media node: a small UDP handshake plus IP discovery (so the server learns the client’s reflexive address through NAT), then DTLS to establish SRTP keys. Now there is an encrypted UDP media session between client and media node.
5. The client publishes its mic as one Opus stream (20ms frames, voice-activity-gated) to the media node. The node receives each publisher once.
6. The media node selectively forwards audio: it forwards the top few active speakers’ Opus packets to every other member of the channel, without decoding or re-encoding (just route packets). For a small channel that is everyone; for a hot channel it is a bounded active-speaker set.
7. The media node detects voice activity per stream (reading the Opus/RTP level, no decode) and reports speaking state back to the gateway, which fans out SPEAKING {user_id, speaking} events to the channel’s members so the UI lights the green ring.

The key split: the gateway (WebSocket, TCP, strongly-consistent per-guild channel state, tolerant of tens of ms) is completely separate from the voice media plane (UDP/SRTP, best-effort, sub-100ms, edge-deployed). The gateway tells the client where to send voice and who is in the channel; the media node moves the actual packets. Each scales and fails on its own terms.

Component Deep Dive

Four hard parts: (1) persistent voice channels and the gateway state model - why “channel” is cheap and always-on, and how per-guild sharding holds membership; (2) the voice media path and the sub-100ms budget - jitter, loss, voice activity, and why we forward rather than mix; (3) the hot channel with 10K+ speakers - where naive forwarding and a single node both break, and the fan-out tree that fixes it; (4) voice regions and node failure - keeping latency low globally and surviving a node death. Each goes naive -> where it breaks -> the fix.

1. Persistent Voice Channels and Gateway State

Unlike a Zoom meeting, a Discord voice channel is a long-lived object attached to a server. Most of its life it is empty. Joining and leaving happen constantly and must be nearly free.

Naive: treat each voice channel like a meeting room you allocate on join

Create per-channel server-side state only when someone joins, tear it down when it empties, and keep a global table of “active channels.”

Where it breaks: two problems. First, a global active-channel table is a hot, contended write path at Discord’s join/leave rate (tens of thousands of transitions/sec, bursting on raids and events) - every hop touches a shared structure. Second, the real cost is not the voice channel itself but presence fan-out: when a user joins a channel, every other member of that server should see the voice state change (who is in which voice channel is shown in the whole member list). A server (guild) can have hundreds of thousands of members. Naive fan-out of one join to all of them is an explosion.

Better: shard state by guild, keep voice state tiny, fan out only to interested sessions

• The guild is the unit of state and sharding. All of a server’s channels, members, roles, and voice state live together on a guild process/shard (Discord famously runs this as stateful Elixir/Erlang processes, one per guild). Sharding by guild_id means a channel join is a local state mutation on that guild’s shard, not a global write. Channels are just entries in the guild’s state; a voice channel is “cheap” because it is a list of member ids plus a region pointer, created with the server, not allocated on join.
• Voice state is small and lives with the guild. The authoritative record is voice_states[guild_id][channel_id] -> {user_id, muted, deafened, self_mute, media_node}. A join appends; a leave removes. This is in-memory on the guild shard, backed by a durable store for the channel/role structure (which rarely changes) but not for ephemeral presence (rebuildable).
• Fan-out is bounded by subscription, not membership. A 400,000-member server does not fan a voice join to 400,000 sockets. The gateway tracks which sessions are actually watching the guild right now (the client has the server open, or is a member whose client subscribed to presence). Large guilds use lazy member lists - the client only subscribes to the slice of the member list it is viewing - so a voice-state change fans out to the thousands of currently-watching sessions, not every member. This is the single most important scaling trick on the gateway side: decouple “who is a member” (huge, durable) from “who needs this event now” (bounded, in-memory).
• The gateway WebSocket is shared across all the user’s servers. One client holds one gateway socket carrying events for every server it is in. Joining voice is just another event type on that socket; it does not open a new control connection. Only the media connection (UDP) is per-voice-session.

guild shard (sharded by guild_id):
  channels:    [#general-voice, #afk, #stage-townhall, ...]   (static structure)
  voice_states: { #general-voice: [{u1,mute}, {u2}], ... }    (ephemeral membership)
  watchers:     set of session_ids currently viewing this guild (bounded)

join voice:  mutate voice_states locally -> fan event to `watchers` only
             -> return VOICE_SERVER_UPDATE (media endpoint) to the joiner

Verdict: shard all state by guild so a voice join is a cheap local mutation; keep voice state as a tiny in-memory membership list; and fan voice-state events only to currently-watching sessions via lazy member lists, never to the full membership. Persistent, always-on channels are only affordable because the channel is just a field in the guild’s state and joining touches one shard.

2. The Voice Media Path and the Sub-100ms Budget

Audio is cheap to move but unforgiving on latency. 100ms mouth-to-ear, over lossy consumer networks, with no retransmission.

Naive: mix all voices on the server into one stream per listener (MCU)

The media server decodes every publisher’s Opus, sums the waveforms into one mixed audio track, re-encodes it, and sends each listener a single mixed stream.

Where it breaks: three ways. Latency - decode + mix + re-encode adds a transcode hop (buffering 20ms frames, running the codec twice) that eats a big slice of a 100ms budget. CPU cost - transcoding is the most expensive thing in media; doing it per listener (each listener needs a mix that excludes their own voice) for millions of concurrent users needs a fortune in hardware. Inflexibility - a server-side mix cannot do per-user spatial audio, per-user volume, or client-side ducking, and it means every packet of loss corrupts the whole mix rather than one voice.

Better: selective forwarding (SFU) of Opus packets, with voice activity and client-side jitter buffering

• Forward, do not transcode. The media node receives each publisher’s Opus packets and forwards them to the other channel members unchanged. No decode, no mix, no re-encode on the live path. This keeps the server packet-bound (cheap) and removes the transcode latency hop. The client decodes and mixes the handful of streams it receives locally, which is trivial for a few Opus streams and enables per-user volume and spatial audio.
• Voice-activity gating slashes the real stream count. Opus has discontinuous transmission (DTX): a silent mic sends almost nothing. The client only transmits when the user is actually speaking (and push-to-talk makes this explicit). So even in a channel of N, the number of actively flowing streams M is small - the people currently talking. The media node forwards those; silent publishers cost nothing. This is why voice fan-out is far cheaper than it looks from N alone.
• The jitter buffer lives on the client, sized for the budget. Packets arrive out of order and with jitter. The client holds a small adaptive jitter buffer (tens of ms) to reorder and smooth before decoding, trading a little latency for continuity. It is kept as short as the network allows to protect the 100ms budget - voice favors low latency over perfect smoothness, the opposite of video streaming.
• Loss is concealed, not retransmitted. A lost Opus packet is never resent (a resent voice packet arrives too late to matter). Instead Opus packet loss concealment and forward error correction (Opus in-band FEC carries a low-bitrate copy of the previous frame) reconstruct a plausible frame. UDP, not TCP; best-effort by design.
• Congestion control adapts bitrate, not reliability. The client watches loss/RTT and can drop Opus bitrate (e.g., 64 -> 32 Kbps) to fit a degrading link, prioritizing getting some intelligible audio through over fidelity.

publisher -> media node:  Opus packets, 20ms frames, sent only while speaking (DTX/PTT)
media node:               select active speakers -> forward packets unchanged (no decode)
subscriber:               jitter buffer (short, adaptive) -> Opus decode (PLC/FEC on loss)
                          -> mix the few received streams locally -> play out
   One upload per speaker, local mix, no server transcode, sub-100ms protected.

Verdict: forward Opus packets through a selective forwarder, never mix server-side; let voice-activity/DTX keep the active stream count small; put a short adaptive jitter buffer and Opus PLC/FEC on the client to ride out loss without retransmission; adapt bitrate under congestion. This is what fits conversational voice inside 100ms over consumer networks.

3. The Hot Channel: 10,000+ Concurrent Participants

A community event, a stage, or a raid puts 10,000+ people in one voice channel. This is the case that breaks the simple design.

Naive: one media node forwards every active speaker to all 10,000 members

Put the whole channel on one media node; it forwards each active speaker’s packets to all N-1 others.

Where it breaks: two walls. Fan-out on one node - even with only M active speakers, forwarding M streams to 10,000 members is M * 10,000 packet-forwards every 20ms. At M=10 that is 100,000 forwards per 20ms = 5,000,000 packet-sends/sec from a single node, plus SRTP encryption per recipient. One node cannot sustain the packet rate or the bandwidth for a single channel. Many-to-many chaos - if the channel let all 10,000 unmute, M could be huge, and 10,000 people talking at once is not communication, it is noise, and it is unservable.

Better: cap concurrent speakers, split listeners across a fan-out tree of media nodes

• Bound the number of simultaneous speakers (M). No channel needs 10,000 simultaneous voices. Enforce an active-speaker cap - forward only the top-K loudest/most-recent speakers (small K for a conversation; for a stage, only the designated speakers can transmit at all). The media plane selects the top-K by voice-activity energy (read from the packet, no decode) with hysteresis so the set does not flap. This turns the forwarded stream count from O(N) down to O(K), a small constant, regardless of how big the channel gets.
• Stage channels make the speaker set explicit. For town halls, the model is one-to-many: a handful of speakers, thousands of listeners who cannot transmit until promoted. Promotion is a gateway permission change (listener -> speaker) that opens their publish path. This is the natural shape of a 10K channel and it makes M tiny by design.
• Split the listeners across a tree of media nodes, not one node. A channel with 10,000 listeners is spread across many media nodes in the region. The speakers’ K streams enter at an origin node, which relays them once to each of a set of fan-out nodes, each of which serves a slice of the listeners. So the origin sends K streams to, say, 50 fan-out nodes (K * 50 forwards), and each fan-out node serves ~200 listeners (K * 200 forwards). No single node fans out to 10,000. The tree can be deeper for truly massive channels.

speakers (K active) ─▶ [origin media node]
                           │  relay K streams once per fan-out node
          ┌────────────────┼────────────────┐
          ▼                ▼                ▼
   [fan-out node 1]  [fan-out node 2] ... [fan-out node 50]
     ~200 listeners    ~200 listeners       ~200 listeners
   Origin: K * 50 forwards. Each fan-out: K * 200 forwards. No node does K * 10,000.

• Node-to-node relay carries one copy per stream per link. The expensive thing is avoided: a speaker’s stream crosses the origin-to-fanout link once, then is duplicated locally at the fan-out node near its listeners. This is the same cascade idea as multi-region meetings, applied within a region to beat single-node fan-out limits.
• Speaking indicators also fan out through the tree, coalesced. With thousands of listeners, you do not push every speaking on/off to all of them individually from one place - the gateway fans speaking events to currently-watching sessions (lazy lists again), and the set of speakers is small, so the event volume is bounded by K, not N.

Verdict: a 10K channel is only possible because (a) the number of concurrent speakers is capped to a small K (explicitly, via stage/speaker roles, or implicitly, via top-K active-speaker selection) and (b) the listeners are spread across a fan-out tree of media nodes so no single node forwards to thousands. Cap the sources, tree the sinks.

4. Voice Regions and Node Failure

Sub-100ms is a physics problem, and media nodes die. Both must be handled without dropping the conversation.

Naive: one global pool of media nodes, assign any free node to a channel

Assign each channel to whatever media node has capacity, anywhere.

Where it breaks: the speed of light and the blast radius. A channel whose members are in Europe but whose node is in the US pays a transatlantic RTT on every packet - well over 100ms mouth-to-ear before the server does anything. And a node that dies takes every channel on it offline with no fast recovery path. Global-random placement breaks both latency and availability.

Better: region-pinned placement, nearest-region selection, and fast re-handshake on failure

• Channels are placed in a voice region. Each voice channel resolves to a geographic region (an explicit admin choice, or auto-selected from the first joiners’ latencies). All members route to media nodes in that region, keeping client-to-node RTT in the single-digit to low-tens of ms and protecting the budget. A channel full of European users sits on European nodes.
• Region selection is latency-probed. On first join the client can measure RTT to candidate regions (or the gateway infers from IP geo) and pick the lowest-latency region that satisfies everyone, not just the host. Discord surfaces this as the channel’s region setting.
• Placement is least-loaded within the region. The routing directory (Redis) tracks media_node -> {region, load, packet_rate}. The gateway places a new channel (or a new fan-out node for a hot channel) on the least-loaded node in the target region, spreading load and avoiding hot nodes.
• Media state is soft; recovery is a re-handshake, not a failover of media. A media node holds no durable state - just live SRTP sessions and the channel’s current membership/routing, all rebuildable. When a node dies, its clients detect the media stall (no packets / RTCP-style timeout within a second or two), go back to the gateway, which re-selects a healthy node in the same region and issues a fresh VOICE_SERVER_UPDATE. The clients re-run the quick UDP+DTLS handshake and rejoin. The gap is a second or two of silence; the lost audio is just discarded packets (nothing to recover, because voice is disposable). No media-state replication is needed - the gateway is the authority for membership and simply re-points clients.
• The gateway/guild plane is the part that needs real failover. The guild shard holding voice state must survive node loss: Discord runs guild processes with supervision and can migrate/restart a guild shard, rebuilding ephemeral presence from reconnecting clients while the durable channel/role structure comes from the backing store. The gateway is stateful but its state is cheap to reconstruct from client reconnects.

Verdict: pin channels to the nearest voice region and place on the least-loaded node there to hold sub-100ms; keep media nodes stateless so a node death is recovered by the client re-handshaking to a sibling node via the gateway in a second or two; and put the real durability effort on the guild/gateway plane that owns membership, since media itself is disposable.

API Design & Data Schema

Gateway API (WebSocket messages, client <-> gateway)

WSS /gateway                                   # one persistent socket per client, all servers

client -> server:
  IDENTIFY          { token, intents }                  # auth the session
  VOICE_STATE_UPDATE{ guild_id, channel_id|null,        # join (channel_id) / leave (null)
                      self_mute, self_deaf }
  REQUEST_MEMBERS   { guild_id, range }                 # lazy member-list slice (fan-out scope)

server -> client:
  READY             { session_id, guilds:[...] }
  VOICE_STATE       { guild_id, channel_id, user_id,    # someone's voice membership changed
                      mute, deaf, self_mute }
  VOICE_SERVER_UPDATE { guild_id, endpoint, token }     # media node + token: go connect voice
  SPEAKING          { guild_id, user_id, speaking }     # green-ring indicator

Voice media handshake (client <-> media node, over UDP after VOICE_SERVER_UPDATE)

voice connection (separate UDP transport, keyed by the gateway-issued token):

  SELECT_PROTOCOL  { codec: "opus", mode: "srtp" }
  IP_DISCOVERY     { }                                  # server echoes client's reflexive ip:port (NAT)
  DTLS handshake   -> establishes SRTP keys
  then: Opus/RTP packets over SRTP, 20ms frames, DTX-gated

  (RTCP-style) feedback: receiver stats, bitrate hints, keepalive

Control-plane REST API (structure/permissions, out-of-band of media)

POST /v1/guilds/{guild_id}/channels                     # create a voice channel
  Body: { name, type:"voice"|"stage", region?, user_limit?, bitrate? }
  201: { channel_id }

PATCH /v1/guilds/{guild_id}/channels/{channel_id}        # change region/limit/permissions
GET   /v1/guilds/{guild_id}/voice-states                 # current voice membership (debug/admin)

Design notes:

• The gateway is one shared WebSocket; voice is a separate per-session UDP transport handed off via VOICE_SERVER_UPDATE. The REST API only touches structure and permissions, never media.
• VOICE_STATE_UPDATE with channel_id=null is leave. The same message type joins, moves between channels, and leaves - a single idempotent state write on the guild shard.
• Permissions are resolved on the guild shard (role/channel overwrites) before any join or speaker promotion is allowed; this is the strongly-consistent state.

Data schemas

Guild / channel structure (SQL) - durable, rarely-changing control state:

guilds
  guild_id      BIGINT PK
  name          TEXT
  owner_id      BIGINT
  region_hint   TEXT                       -- default voice region
  created_at    TIMESTAMP
  INDEX (owner_id)

channels
  channel_id    BIGINT PK
  guild_id      BIGINT                      -- FK, shard key
  type          ENUM                        -- text | voice | stage
  name          TEXT
  region        TEXT NULL                   -- override guild region_hint
  user_limit    INT NULL
  bitrate       INT                         -- channel audio bitrate cap
  position      INT
  INDEX (guild_id)                          -- list a guild's channels

channel_permissions
  channel_id    BIGINT
  role_id       BIGINT
  allow         BIGINT                      -- bitmask: CONNECT, SPEAK, etc.
  deny          BIGINT
  PRIMARY KEY (channel_id, role_id)

Why SQL here: guild/channel/permission structure is relational, correctness-critical (permission checks on every join and speaker promotion), and low-write (structure changes rarely). It is sharded/partitioned by guild_id and cached hot in the guild shard’s memory.

Live voice state (in-memory on the guild shard, mirrored to Redis for routing):

voice_states[guild_id][channel_id] = [
   { user_id, session_id, media_node_id, mute, deaf, self_mute, speaking, joined_at },
   ...
]
   Authoritative in the guild process (Erlang/Elixir); ephemeral; rebuilt from
   client reconnects on shard restart. NOT in SQL - it churns far too fast.

Routing directory (Redis) - the fast live index for media placement:

KEY  channel:{channel_id}     -> { region, media_nodes:[origin, fanout...], member_count }
KEY  media_node:{node_id}     -> { region, load_packets_per_sec, bandwidth, healthy }
KEY  session:{session_id}     -> { guild_id, channel_id, media_node_id, ttl }
   Used by the gateway to place joiners, build the fan-out tree for hot channels,
   and re-point clients on node failure. Ephemeral, rebuildable, sub-ms reads.

Why NoSQL/in-memory here: routing state is ephemeral, high-churn (every join/leave/hop), read on every voice transition and reconnect, and needs sub-millisecond lookups with no durability requirement. Redis plus the guild-process memory, not SQL.

Media itself has no schema and no storage - it is SRTP Opus packet flows, disposable, never persisted in the default design (no recording). The durable footprint of a voice system is tiny (structure + permissions); the expensive part is moving disposable packets fast and near the user.

Bottlenecks & Scaling

Where it breaks first, in order, and the fix for each:

1. Presence fan-out on large guilds (breaks first). A voice join naively fanned to every member of a 400K-member server is an explosion on the gateway. Fix: lazy member lists + subscription-scoped fan-out - events go only to sessions currently watching the guild (the slice of the member list they have open), not to the full membership. Decouple “is a member” (huge, durable) from “needs this event now” (bounded, in-memory).

2. Global write contention on voice transitions. Tens of thousands of join/leave/hop per second against shared state. Fix: shard all state by guild_id onto stateful guild processes. A voice transition is a local mutation on one shard, not a global write. This is the foundational sharding decision.

Shard/placement key choice: the guild is the control-plane shard key (all of a server’s channels/members/voice-state on one process), and the channel is the media-placement unit (placed in a region, onto one or more media nodes). Sharding control by guild keeps a server’s state coherent on one process; placing media by channel+region keeps fan-out local. Sharding by user would scatter a channel’s membership across unrelated shards and break the fan-out model.

3. Single media node fan-out in a hot channel. One node forwarding K active speakers to 10,000 listeners exceeds its packet rate and bandwidth. Fix: cap concurrent speakers to a small K (stage/speaker roles, or top-K active-speaker selection with hysteresis) and spread listeners across a fan-out tree of media nodes, where the origin relays each stream once per fan-out node and each fan-out node serves a few hundred listeners. Cap the sources, tree the sinks.

4. Latency on geographically spread channels. A badly placed node adds transoceanic RTT and blows the 100ms budget. Fix: region-pinned channels + nearest-region, least-loaded placement. Members route to nearby media nodes; region is probed/auto-selected so everyone, not just the host, is close.

5. Media node death. A node failure drops every channel on it. Fix: stateless media nodes + fast client re-handshake. Clients detect the stall, return to the gateway, get re-pointed to a sibling node in the same region, and re-run the quick UDP/DTLS handshake. One to two seconds of silence, no media-state replication needed (voice is disposable). Durability effort goes to the guild/gateway plane, which is supervised and rebuilds ephemeral state from reconnects.

6. Hot key: a viral server or a raid. A single guild suddenly spikes to hundreds of thousands watching and a channel to 10K+. Fix: the guild shard must be sized/able to migrate, and the channel uses the fan-out tree; lazy lists keep gateway fan-out bounded; place new fan-out nodes on least-loaded nodes in-region tracked by Redis. The tree deepens as the channel grows.

7. Gateway WebSocket fleet at 15M concurrent. Control sockets and join storms. Fix: a horizontally scaled gateway fleet fronting the guild shards, with jittered reconnect on node death (a thundering reconnect herd after a deploy or outage is a real failure mode - Discord stages reconnects). Messages are tiny, so the gateway scales far more easily than the media plane - the hard scaling is in media fan-out, not control.

8. NAT and UDP-blocked clients. Corporate/firewalled networks block UDP, and NAT hides client addresses. Fix: IP discovery in the voice handshake resolves NAT reflexive addresses for most clients; a fleet of relay/TURN-style servers (media over TCP/443 when UDP is blocked) handles the minority behind strict firewalls, placed at the edge near clients. Most pay only the direct UDP path; only the blocked minority pay the relay hop.

9. Speaking-indicator thrash. Rapid talk-start/stop could spam speaking events to thousands of watchers. Fix: debounce/coalesce speaking state (short hold before flipping off, read from packet energy with no decode) and fan only to watching sessions. The speaker set is small (K), so event volume is bounded by K, not N.

E2EE trade-off (noted): running voice end-to-end encrypted (DAVE-style) means the media node can no longer read audio energy for active-speaker selection or do any server-side processing; selection and key management move client-side (sender-side keys, MLS-style group key agreement), and the server becomes a pure encrypted-packet forwarder. It is the right default for privacy but it constrains server-side features - the classic E2EE-versus-server-smarts trade.

Wrap-Up

The trade-offs that define this design:

• Persistent channels are cheap because the guild is the unit of state. A voice channel is just a field in a per-guild stateful process; joining is a local mutation on one shard, not a global allocation. Sharding by guild is the foundational decision that makes always-on, constantly-hopped channels affordable.
• Fan-out is bounded by subscription, not membership. A 400K-member server does not fan a voice join to 400K sockets; lazy member lists mean events reach only the thousands currently watching. Decoupling “is a member” from “needs this event now” is the gateway’s whole scaling story.
• Forward Opus packets, never mix server-side. Selective forwarding keeps the media node packet-bound and cheap, removes the transcode latency hop, and lets clients mix locally for per-user volume and spatial audio. Voice-activity/DTX keeps the active stream count small, Opus PLC/FEC rides out loss without retransmission, and a short client jitter buffer protects the 100ms budget. UDP, best-effort, disposable.
• The 10K channel is cap-the-sources, tree-the-sinks. Concurrent speakers are capped to a small K (explicitly via stage roles or implicitly via top-K selection), and listeners are spread across a fan-out tree of media nodes so no single node forwards to thousands. Neither naive forwarding nor a single node survives the hot case.
• Region-pin for latency, stateless media for availability. Channels route to nearby least-loaded nodes to hold sub-100ms; media nodes carry no durable state, so a node death is a one-to-two-second client re-handshake to a sibling, not a dropped conversation. The real durability effort is on the guild/gateway plane that owns membership.

One-line summary: shard all control state by guild into stateful processes and fan voice-state events only to currently-watching sessions; hand each joiner off to a nearby, least-loaded, stateless media node that selectively forwards the top-K active speakers’ Opus packets without transcoding; and for hot channels cap the speakers and spread listeners across a fan-out tree of nodes - so a 3-person channel, a 10,000-person stage, constant channel-hopping, and node failures all hold under 100ms at 150M monthly users.