Everyone thinks the screen limit is a counter. “Each plan allows N streams, keep an integer per account, increment when a play starts, decrement when it stops, reject when it hits N. Done.” Then the interviewer asks the questions that decide whether it actually works: what happens when the player crashes and never sends the decrement, so the counter leaks upward until a paying family can never watch anything again; what happens when two devices in the same house both hit play in the same 50 milliseconds on the last free slot; how do you make that allow/deny decision fast enough that nobody notices it before their video starts; and how do you do all of it for 250 million subscribers with 100 million concurrent streams at peak. Now it is a real problem.

The screen limit is deceptive because the naive counter is correct in exactly the world where nothing ever fails. In the real world, players crash, phones lose signal in tunnels, apps get force-killed, networks partition, and the internet drops packets. Every one of those events is a decrement that never arrives. So the whole design is really about one thing: counting active streams when you cannot trust anyone to tell you they stopped. That single constraint - never trust a clean shutdown - drives every decision below.

Let me do it properly.

Functional Requirements (FR)

In scope:

  • Enforce a per-account concurrent-stream cap. Each subscription plan allows N simultaneous streams (Basic = 1, Standard = 2, Premium = 4). At any instant, the number of actively playing streams on an account must not exceed N.
  • Block the N+1th stream at play-start, in real time. When a device presses play and the account is already at N active streams, that play attempt is rejected before the video starts, with a clear “too many devices” message and the option to see which devices are streaming.
  • Release a slot promptly when a stream ends - whether it ends cleanly (user hits stop/back), or dirtily (app killed, device dies, network drops). A leaked slot that is never released is the single worst failure: it silently shrinks the family’s plan.
  • Handle graceful stop. Pause, stop, back-to-browse, and app-close should free the slot quickly so the next device is not blocked for long.
  • Consistent across all client types. TV, phone, tablet, web, console - all share the same account-wide count.

Explicitly out of scope (say this out loud so you control the scope):

  • Video encoding, CDN delivery, adaptive bitrate. The actual bytes of video are a separate system (Open Connect / CDN); we only gate the right to start a stream.
  • Authentication, login, and profile management. We assume the play request already carries a resolved account_id and device_id.
  • Billing, plan upgrades, and payment. We read the plan’s N; we do not sell plans.
  • Household / geo-location enforcement (“password sharing” crackdown by IP). That is a related but distinct policy layer. We enforce concurrency, not who is in the house.
  • DRM and license issuance. Related (a play needs a license) but separate; we focus on the concurrency gate.

The one decision that drives everything: we cannot rely on a stop signal to free a slot. Clients disappear without saying goodbye all the time. So the count of “active” streams must be derived from something the client keeps proving - a lease it must renew - not from a start/stop pair we hope both arrive.

Non-Functional Requirements (NFR)

  • Scale: 250M subscribers. Peak concurrency of ~100M simultaneous streams globally. Play-start events (the write that matters) peak well above the steady state because of prime-time and big-launch spikes - budget for ~300K-500K play-starts/sec at peak. Heartbeats from 100M active streams are the dominant background load.
  • Latency: the allow/deny check sits between “user pressed play” and “video appears.” It must add under ~50ms at p99. Users tolerate a couple of seconds of buffering; they do not tolerate a spinner that is our fault. The common case (account well under its limit) must be fast and cheap.
  • Availability: 99.99%+. But note the asymmetry: this is a soft gate on a paid product. If the concurrency service is degraded, the correct business choice is usually to fail open - let the customer watch. Blocking a paying subscriber’s movie because our counter service hiccuped is a far worse outcome than briefly allowing an N+1th stream. This is the opposite of a security system.
  • Consistency: we need strong consistency only on the contended edge - the moment two devices race for the last slot on one account. Everywhere else, eventual is fine. Crucially, the state is partitioned by account: there is no cross-account transaction, ever. So we get “strong within an account, independent across accounts,” which is the ideal sharding story.
  • Durability: low. Active-stream state is ephemeral, second-to-minute-lived, and self-correcting via lease expiry. If we lose the live state on a crash, the worst case is a brief window where the caps are not enforced (streams re-register on their next heartbeat). We never need this data on durable disk for its own sake; we keep it in memory with replication for availability, not for a permanent record.

Back-of-the-Envelope Estimation (BoE)

Let me ground the design in numbers.

Concurrency and play-start rate (the writes that matter):

Subscribers:                 250,000,000
Peak concurrent streams:     100,000,000
Average session length:      ~1 hour of continuous viewing

Play-starts/sec (steady):
  100M streams / 3600s avg session
  ≈ 27,000 new streams/sec just from natural churn
Peak play-starts (prime time + launches):
  budget 10-20x steady for spikes  ≈ 300,000-500,000/sec

Heartbeat load (the real firehose):

Every active stream renews its lease on an interval.
Choose heartbeat interval = 30s.

100,000,000 active streams / 30s
  = 3,333,000 heartbeats/sec  (~3.3M QPS)

That 3.3M heartbeats/sec is the dominant load in the whole system, and it is a pure key-touch (refresh a TTL). It tells us immediately: heartbeats must be dirt cheap - a single in-memory key write, no fan-out, no downstream. The heartbeat interval is the master knob: halve it and you halve leak-detection time but double the firehose.

Storage for live state:

Per active stream we store a small record:
  account_id (8B) + device_id (16B) + stream_id (16B)
  + last_heartbeat_ts (8B) + plan_slot metadata (~16B)
  ≈ 64 bytes of payload
  + store overhead (index, key) ≈ ~150-200 bytes/stream

100M streams * ~200 bytes ≈ 20 GB of live state.

Twenty gigabytes. The entire live-concurrency state of Netflix fits in the RAM of a modest cluster many times over. This is emphatically not a storage problem - it is a coordination-and-liveness problem. That reframing is the whole point: do not reach for a giant database, reach for a fast, partitioned, in-memory store with atomic per-account operations.

Account-state size (the unit of contention):

Per account we hold: plan N (1 byte) + a small set of active streams.
Even a Premium household rarely exceeds 4-8 device records.
Per-account state ≈ a few hundred bytes. Tiny.

Bandwidth: each play-start check and each heartbeat is a few hundred bytes. 3.3M heartbeats/sec * ~300 bytes ≈ ~1 GB/sec of internal traffic spread across the cluster. Trivial next to the actual video (which we do not touch). This confirms it: the challenge is op-rate and liveness detection, not bytes.

High-Level Design (HLD)

The screen limit is not something the video path routes through; it is a gate consulted once at play-start and then a lease the client keeps renewing for the life of the stream. The video bytes flow from the CDN completely independently. Here is the end-to-end picture: a stateless Concurrency Service in front of a per-account, in-memory, atomically-updated store, with lease expiry as the mechanism that reclaims leaked slots.

        ┌──────────────────────────────────────────────┐
        │   Clients: TV / phone / tablet / web / console │
        └───────────────┬──────────────────────────────┘
                        │  (1) POST /streams  (play-start)
                        │  (3) heartbeat every 30s
                        │  (5) DELETE /streams (graceful stop)
                        ▼
        ┌──────────────────────────────────────────────┐
        │              API Gateway / LB                  │
        │   routes by account_id (sticky to shard owner) │
        └───────────────┬──────────────────────────────┘
                        │
        ┌───────────────▼──────────────────────────────┐
        │        Concurrency Service (stateless)         │
        │   - resolves plan N for account                │
        │   - runs atomic acquire/renew/release          │
        │   - decides allow / deny (block N+1th)          │
        └───────┬──────────────────────────┬────────────┘
                │ atomic per-account op     │ read plan
                ▼                           ▼
   ┌────────────────────────────┐   ┌──────────────────────┐
   │  Live-State Store (Redis)   │   │  Plan/Subscription DB │
   │  sharded by account_id      │   │  account -> plan -> N │
   │  key: acct:{id} -> stream    │   │  (SQL, cached hot)    │
   │  set with per-stream TTL     │   └──────────────────────┘
   │  primary + replica per shard│
   └──────────────┬─────────────┘
                  │  TTL expiry reclaims leaked slots
                  ▼
   ┌────────────────────────────────────────────────┐
   │  Reaper / lazy expiry: a stream whose lease is   │
   │  older than 2-3 missed heartbeats is dead -> free │
   └────────────────────────────────────────────────┘

   Video bytes flow separately:
   Client <====== Open Connect / CDN (not on this path) ======>

Request flow (the hot path):

  1. A device presses play. The client sends POST /streams with account_id, device_id, and the title. The gateway routes it to a Concurrency Service instance; the request is keyed by account_id.
  2. The service looks up the plan’s N (cached in memory, refreshed rarely) and runs a single atomic acquire against that account’s shard: “remove expired streams, count what remains, and if count < N, add this stream and return a lease; else return deny.”
  3. If allowed, the client gets a stream_id and a lease_ttl, then goes to the CDN and starts playing. It now must heartbeat every 30s (PUT /streams/{id}/heartbeat), which just refreshes that stream’s TTL.
  4. If denied (already at N), the client shows “You are watching on too many devices” plus the list of active devices, and offers to stop one.
  5. On a graceful stop, the client sends DELETE /streams/{id}, which frees the slot immediately.
  6. On a dirty stop (crash, killed app, dead network), no delete arrives - but the heartbeats also stop. After 2-3 missed heartbeats the lease TTL expires and the slot is reclaimed automatically. This is the safety net that makes the whole thing robust.

The key architectural insight is step 3 and 6 together: the slot is held by a lease, not by a start/stop pair. A stream is “active” only as long as it keeps proving it is alive. That inverts the fragile “hope the stop signal arrives” model into a robust “assume dead unless proven alive” model.

Component Deep Dive

Four hard parts: (1) how to count active streams without trusting a stop signal, (2) how to make the acquire atomic so the last-slot race is correct, (3) where to run the state so 100M streams and 3.3M heartbeats/sec are cheap, and (4) how the client and reaper cooperate to reclaim leaked slots fast. I will walk each from naive to scalable.

1. Counting Active Streams: Start/Stop Counter -> Leases

Approach A: An integer counter with increment/decrement (the naive instinct)

Keep one integer per account. Increment on play, decrement on stop, reject when it reaches N.

def on_play(account, N):
    if counter[account] >= N:
        return DENY
    counter[account] += 1
    return ALLOW

def on_stop(account):
    counter[account] -= 1     # <-- this is where it all falls apart

Dead simple, O(1) memory. Where it breaks: the decrement is not guaranteed to run. The player crashes, the phone dies in a tunnel, the OS force-kills the backgrounded app, the Wi-Fi drops mid-episode - in every one of those cases on_stop is never called. The counter is now permanently one too high. It only ever leaks upward. After a few crashes, a Premium family that paid for 4 streams sees “too many devices” with zero devices actually playing. The naive counter turns transient client failures into a permanent, silent downgrade of the plan. There is no client behavior you can add to fix this, because by definition the failure is the client not getting to run its cleanup code. And it gets worse: which decrement belongs to which stream? A raw counter cannot tell, so you cannot even show the user which device to stop.

Approach B: A set of stream records, each with a lease/TTL (the fix)

Do not store a count; store the identity of each active stream, and make each one carry a lease that must be renewed. The active count is simply “how many leases are currently unexpired.”

account state = set of active streams, e.g.
  acct:42 -> {
     stream_A: { device: "living-room-TV", expires_at: T+90s },
     stream_B: { device: "moms-phone",     expires_at: T+90s },
  }

count = number of streams whose expires_at > now

The client renews its lease with a heartbeat every 30s, which pushes expires_at forward. If the client dies, it stops renewing, the lease expires on its own, and the slot frees itself with no cooperation from the (dead) client required. This flips the model from trust the stop to assume dead unless heartbeating. A leaked slot now self-heals within one lease window instead of leaking forever.

Two knobs define the behavior:

KnobSmall valueLarge value
Heartbeat interval (e.g. 30s)Faster leak detection, more QPSCheaper, slower to reclaim
Lease TTL (e.g. 90s = 3 intervals)Reclaims fast, risks killing a live-but-laggy stream on a couple dropped beatsTolerant of blips, holds leaked slots longer

Choosing TTL = ~3 * interval tolerates a couple of dropped heartbeats (a brief network blip should not kill an active stream) while still reclaiming a truly dead stream within ~90s. This is the honest trade: you cannot distinguish “crashed” from “temporarily unreachable” instantly, so you wait a few missed beats. Storing identities (not a bare number) also directly powers the “here are your active devices, stop one” screen, since we know exactly which device holds which slot.

Verdict: leases, not counters. The count is derived from live leases, so it self-corrects. This is the single most important decision in the design.

2. The Last-Slot Race: Making Acquire Atomic

Leases fix leaks, but they do not fix concurrency. Two devices in the same household pressing play at nearly the same instant, on an account with exactly one free slot, is the core correctness case.

Naive: read-count-then-write from the application

streams = store.get(account)                 # read
active  = count_unexpired(streams)            # count
if active < N:                                # check
    store.add(account, new_stream)            # write  <-- race window here
    return ALLOW
return DENY

Where it breaks: between the read/count and the write, another Concurrency Service instance handling the other device does the same read, sees the same active < N, and also writes. Both think they got the last slot. The account is now at N+1. This is a classic read-modify-write race (lost update), and with two TVs in one living room hitting play off the same “Continue Watching” row, it is not a rare edge case - it is Tuesday night. Even worse if you sharded the count across nodes: no single node sees the true total.

Better: one atomic acquire per account

The read-remove-expired-count-check-add must be a single indivisible operation on the account’s state. Because all state for one account lives on one shard, and that shard is single-threaded per key, we can push the entire logic into one atomic server-side script. Redis executes a Lua script start-to-finish with no interleaving:

-- acquire_slot.lua   KEYS[1] = account state key (a sorted set)
-- ARGV: N (plan limit), now, ttl, stream_id, device_meta
local N        = tonumber(ARGV[1])
local now      = tonumber(ARGV[2])
local ttl      = tonumber(ARGV[3])
local stream   = ARGV[4]
local device   = ARGV[5]

-- 1. reclaim expired leases: drop members whose score (expiry) < now
redis.call("ZREMRANGEBYSCORE", KEYS[1], 0, now)

-- 2. is this device already streaming? (renew, do not double-count)
local existing = redis.call("ZSCORE", KEYS[1], stream)

-- 3. count live streams
local active = redis.call("ZCARD", KEYS[1])

if existing == false and active >= N then
  return {0, active}                 -- DENY: at capacity
end

-- 4. add / renew this stream with a fresh expiry, atomically
redis.call("ZADD", KEYS[1], now + ttl, stream)
redis.call("HSET", KEYS[1] .. ":meta", stream, device)
redis.call("PEXPIRE", KEYS[1], (ttl + 60) * 1000)   -- whole set self-cleans if idle
return {1, redis.call("ZCARD", KEYS[1])}

Storing streams in a sorted set scored by expiry timestamp is the trick that makes this cheap: expired leases are removed with one ZREMRANGEBYSCORE, the live count is one ZCARD, and add/renew is one ZADD. The whole acquire is O(log n) on a set of ~4-8 members - effectively free. Because the script runs atomically on the one shard that owns this account, the two-TVs race is resolved deterministically: one ZADD runs fully before the other starts, the second sees active >= N, and it is cleanly denied. No lost update, no N+1.

Heartbeat is the same script with a shortcut: the stream already exists, so it just re-ZADDs a new expiry (renew) and returns allow. That keeps the 3.3M heartbeats/sec on the exact same cheap atomic path.

Verdict: one atomic per-account script on a single-owner shard. Partitioning by account means these atomics never coordinate across accounts, so there is no global lock and no cross-shard transaction - the contention is naturally confined to one household’s handful of devices.

3. Where the State Lives: Scaling to 100M Streams and 3.3M Heartbeats/sec

The atomic-per-account model is correct on one node. Now make it hold 100M leases and 3.3M renews/sec.

Naive: one database, one row per account, updated on every heartbeat

Put the active-stream state in the main SQL database, one row per account, and UPDATE it on every play and every heartbeat.

Where it breaks: 3.3M UPDATEs/sec of durable, disk-backed, transactional rows is absurd. Each heartbeat is a tiny TTL refresh, but a relational engine makes it a WAL write, a lock, an fsync-eventually - orders of magnitude more work than the operation deserves. You would need a monstrous, expensive database doing nothing but rewriting the same rows every 30 seconds, and heartbeats would contend with the play-start writes on the same hot rows. The durability you are paying for is worthless here: this data is throwaway, self-correcting, sub-minute state. Using a durable transactional DB for lease heartbeats is paying for a bank vault to store sticky notes.

Better: sharded in-memory store, partitioned by account

Keep the live state in an in-memory key-value store (Redis), sharded by account_id. Each account’s sorted set lives on exactly one shard.

shard = hash(account_id) % num_shards

Load spread:
  3.3M heartbeats/sec + ~0.5M play-starts/sec  ≈ ~4M ops/sec total
  A Redis shard handles ~100K-200K atomic-script ops/sec comfortably.
  4M / 150K ≈ 27+ shards to hold the op rate; run more for headroom
             and failover (primary + replica per shard).

Why this is the right shape:

  • Account is the perfect shard key. Every operation - acquire, renew, release - is scoped to one account. There is never a query that spans accounts. So each op is a single-shard, single-key atomic script. Load spreads evenly because account IDs are high-cardinality and uniformly hashed; no household is big enough to be a hot shard on its own.
  • Heartbeats become a single in-memory key touch. A renew is one ZADD with a new score. No fan-out, no downstream write, no durability cost. That is what makes 3.3M/sec affordable.
  • Memory is a non-issue. ~20 GB across the cluster (from the BoE), trivially held in RAM with replicas.
  • The Concurrency Service stays stateless. All state is in Redis; service instances are interchangeable and scale horizontally behind the load balancer. Route by account_id so a given account’s requests tend to hit warm connections, but correctness never depends on stickiness because the atomic lives in Redis, not in the service.

Durability posture: we run primary + replica per shard with fast failover (Redis Cluster / Sentinel) for availability, not because we need the data forever. If a shard is lost entirely, the affected accounts’ streams simply re-register on their next heartbeat (within 30s), and worst case a few accounts briefly run over their cap. That is an acceptable failure for a soft gate on a paid product - which loops back to the fail-open NFR.

4. Reclaiming Leaked Slots Fast: Client Cooperation + Reaper

Lease expiry guarantees a leaked slot eventually frees. Two refinements make “eventually” fast and make normal stops instant.

Graceful stop: the explicit release (the happy path)

When the user hits stop/back/close, the client fires DELETE /streams/{id}, which is one ZREM - the slot is free immediately, no waiting for a TTL. Most stops are graceful, so most slots free instantly. Crucially, the app should also send this release on lifecycle events the OS does give us warning for: onPause/onStop/onDestroy on mobile, beforeunload/visibilitychange on web, backgrounding on TV apps. A best-effort release on background covers a large fraction of “dirty-looking” stops and makes the system feel snappy.

Dirty stop: lease expiry as the backstop (the safety net)

For true crashes where no release fires, the missed heartbeats let the lease expire. Two ways the expired lease actually gets removed:

  • Lazy expiry (primary mechanism). We never need a background sweeper to hunt for dead leases, because the acquire script always runs ZREMRANGEBYSCORE(key, 0, now) as its first step. So the moment the next device on that account tries to play, all that account’s stale leases are purged inline, and the count is correct. Expiry is paid for exactly when it matters - at contention time - and costs nothing when the account is quiet. This is the elegant part: the same atomic that enforces the limit also does the garbage collection.
  • Active reaper (optional, for the UI). The “which devices are active” screen should not show a device whose lease expired 20s ago even though nobody has pressed play since. A lightweight background job (or Redis key-expiry notifications) can periodically prune expired members so read views are clean. This is a nicety for display accuracy, not a correctness requirement - the acquire script already guarantees the count is right.

Tuning the reclaim window

heartbeat interval = 30s
lease TTL          = 90s  (tolerate 2 missed beats)

Dirty-stop reclaim time:
  best case  : graceful/background release  -> instant
  crash case : up to 90s until the slot frees automatically

A 90s worst case for a hard crash is a fine tradeoff: it is rare, it self-heals, and shrinking it means more heartbeat load and more risk of killing a live-but-laggy stream on a transient blip. If a user is actively blocked and impatient, the “stop a device” screen lets them force-release a specific stream immediately (an explicit ZREM on the chosen stream_id), so they are never actually stuck waiting for a TTL.

API Design & Data Schema

Client-facing API

POST /v1/streams                         # play-start: acquire a slot
  Body: { account_id, device_id, profile_id, title_id, device_meta }
  200 ALLOW:
     { stream_id, lease_ttl_sec: 90, heartbeat_interval_sec: 30 }
  409 DENY (at capacity):
     { error: "concurrent_stream_limit",
       limit: 2,
       active: [ {stream_id, device_meta:"Living Room TV", started_at},
                 {stream_id, device_meta:"Mom's iPhone",   started_at} ] }

PUT /v1/streams/{stream_id}/heartbeat    # renew the lease every 30s
  Body: { account_id }
  200: { ok: true, lease_ttl_sec: 90 }
  410: { error: "lease_expired", action: "re_acquire" }   # was reaped; try POST again

DELETE /v1/streams/{stream_id}           # graceful stop: free the slot now
  Body: { account_id }
  200: { ok: true }

GET  /v1/accounts/{account_id}/streams   # list active devices (for the "stop one" UI)
  200: { limit: 4, active: [ {stream_id, device_meta, started_at, last_seen} ] }

POST /v1/accounts/{account_id}/streams/{stream_id}/force-stop
  # user chose to kick a device; explicit ZREM, frees the slot immediately
  200: { ok: true }

Design notes:

  • POST /streams is the only place the limit is enforced. Heartbeat renews an existing slot and never needs to re-check N (a live stream that briefly exceeds due to a race was already caught at acquire).
  • The 410 on heartbeat handles the case where a real stream got reaped after a long blip: the client transparently re-acquires, and if the account is now full it gets a clean deny - correct behavior.
  • All endpoints carry account_id so the gateway can route to the owning shard without a lookup.

Data store choice: in-memory KV (Redis) for live state, SQL for plans

Live-stream state is tiny, ephemeral, self-correcting, and hammered with 4M atomic ops/sec keyed by a single id. No joins, no range scans, no durability requirement, but we need atomic read-modify-write and per-member TTL semantics. That is the exact profile for an in-memory KV store (Redis), sharded. Plan/subscription data is the opposite: small, read-heavy, changes rarely, must be correct and durable - a natural fit for SQL, heavily cached in the Concurrency Service.

Live-stream state (Redis, per account)

Key:    acct:{account_id}                 -- e.g. "acct:8675309"
Type:   Sorted Set (ZSET)
  member = stream_id
  score  = lease_expiry_epoch_ms          -- expiry is the score; ZREMRANGEBYSCORE reaps
Sidecar:
Key:    acct:{account_id}:meta            -- Hash
  field  = stream_id
  value  = { device_id, device_meta, profile_id, started_at }
TTL:    whole key PEXPIRE'd to (lease_ttl + slack) so idle accounts self-evict
Access: single atomic Lua script (reap-expired + count + check + add/renew)
Shard:  hash(account_id) % num_shards

Why a sorted set: scoring members by expiry turns “reclaim dead leases” into one range delete and “count live” into one ZCARD, both cheap. That single data-structure choice is what makes lazy expiry free.

Plan / subscription (SQL, cached)

account_id        BIGINT   PK
plan              ENUM     -- basic | standard | premium
max_streams (N)   INT      -- 1 | 2 | 4
status            ENUM     -- active | paused | churned
updated_at        TIMESTAMP
  INDEX (account_id)   -- point lookups only

account_id -> N is read on every play-start, so it is cached in-process in the Concurrency Service with a short TTL and invalidated on plan change (plan changes are rare and can tolerate a few seconds of staleness). We never hit SQL on the hot path once the account’s plan is warm in memory.

Bottlenecks & Scaling

Where it breaks first, in order, and the fix for each:

1. Leaked slots from dirty client shutdowns (breaks first, and worst). The original sin: a decrement that never arrives shrinks the plan permanently. Fix: leases, not counters. A stream holds its slot only while it keeps heartbeating; a dead client stops renewing and its lease expires. Lazy expiry inside the acquire script reclaims the slot at the next contention with zero background work. This is the design’s foundation, not an add-on.

2. The last-slot race (two devices, one free slot). Read-count-then-write lets both win, pushing the account to N+1. Fix: a single atomic Lua script per account (reap + count + check + add) on a single-owner shard. Because Redis is single-threaded per key, the two acquires serialize and exactly one wins.

3. Heartbeat firehose (3.3M ops/sec). This is the dominant steady-state load and would crush any durable database. Fix: keep live state in a sharded in-memory store; a heartbeat is one ZADD (TTL refresh), no fan-out, no downstream. Tune the heartbeat interval to trade reclaim speed against QPS; 30s balances a ~90s worst-case reclaim against ~3.3M/sec.

4. Single store node as bottleneck / SPOF. One node cannot hold 4M ops/sec, and its loss would blind those accounts. Fix: shard by account_id (hash(account_id) % num_shards), ~27+ shards for op rate plus headroom. Each account lives on one shard, so every op is single-shard and single-key - no cross-shard coordination ever. Run primary + replica per shard with fast failover. Account IDs are uniformly hashed and no single household is large, so there is no natural hot shard.

Shard key choice: account_id. Every acquire/renew/release is scoped to exactly one account, and the limit is a per-account invariant, so the account is the natural partition. Sharding by anything else (region, device, plan) would split one account’s streams across shards and reintroduce the cross-node counting problem the atomic was meant to kill.

5. Plan lookup on the hot path. Hitting SQL for N on every play-start would bottleneck on the database. Fix: cache account_id -> N in the stateless service with a short TTL, invalidated on the (rare) plan change. Play-starts read N from memory.

6. Store outage blocking paying customers. If the concurrency store is unreachable and we fail closed, a Redis blip stops people watching a product they paid for. Fix: fail open. On store failure, allow the play (optionally with a coarse local cap) and log it. A brief over-limit window is a far better business outcome than blocking subscribers. This is the deliberate inversion versus a security gate - the whole system is a soft nudge, not a lock. Fail-open plus replicated shards keeps the actual outage window tiny.

7. Clock skew across service nodes. Lease expiry math uses timestamps; if service clocks disagree, a stream could be reaped early or held late. Fix: compute now and expiry using the Redis server’s clock (via TIME inside the Lua script) as the single authority for the atomic path, so all expiry math references one monotonic source rather than each service node’s local clock. Keep nodes NTP-synced for good measure.

8. Thundering re-acquire after a shard failover. If a shard fails over, its accounts’ state is briefly empty and many clients re-register at once. Fix: this is self-correcting and benign - clients re-acquire on their next heartbeat (spread across the 30s interval, not all at once), the empty state means everyone is briefly allowed (fail-open-ish), and the caps re-establish within one heartbeat window. Jitter the heartbeat interval per client (e.g. 30s +/- a few seconds) so renewals do not synchronize into a spike.

Wrap-Up

The trade-offs that define this design:

  • Leases over a start/stop counter. We refuse to trust that a stop signal ever arrives, because in the real world it frequently does not (crashes, kills, dead networks). The active count is derived from unexpired leases, so a leaked slot self-heals within one lease window instead of permanently shrinking the plan. This one inversion - assume dead unless heartbeating - is the heart of the design.
  • Atomic per-account script over application-side read-modify-write. The last-slot race between two devices in one household is the core correctness case, and a single-threaded, single-shard atomic (reap + count + check + add, over a sorted set scored by expiry) resolves it deterministically with no global lock.
  • Sharded in-memory store over a durable database. The state is 20 GB of throwaway, self-correcting, sub-minute data hit at 4M ops/sec. In-memory KV sharded by account makes heartbeats a single cheap key touch and keeps every operation confined to one household’s handful of streams. We replicate for availability, not for a permanent record.
  • Lazy expiry over a background sweeper. The same atomic that enforces the limit reaps that account’s dead leases as its first step, so garbage collection is paid for exactly at contention time and is free when the account is idle.
  • Fail open, not fail closed. This is a soft gate on a paid product, the opposite of a security system. If the counter service is degraded, letting a customer watch beats blocking a paying subscriber over an N+1th stream.

One-line summary: enforce the screen limit as a per-account lease, acquired at play-start via one atomic reap-count-check-add script on an account-sharded in-memory store, renewed by cheap 30s heartbeats and reclaimed automatically by lease expiry, so the cap holds correctly at 100M concurrent streams while a crashed player’s slot heals itself instead of leaking forever.