“23 people are looking at this hotel right now.” It is one line of text, a soft nudge that the room might not be here tomorrow. It looks trivial. It is not. To render that number you have to answer, for every one of a million product pages, a question that is genuinely hard at scale: how many distinct humans are looking at this page right now, updated live, refreshed as people arrive and leave, across a global fleet.

Do the naive multiplication and you can see the trap. If every viewer’s browser pings “I am still here” every few seconds, and you write each ping to a counter in a database, you are doing tens of millions of writes per second, most of them onto a few thousand hot pages, and every read of “how many are here” is a COUNT(*) over a churning set. The count is also not a stored fact - it is a windowed aggregate over a set of live sessions that are constantly expiring. That is the whole problem: this is a counting-and-expiry system, not a storage system. Nobody is asking you to durably persist that 23; if you lose it, you recompute it in seconds. What you cannot do is take a write storm on a hot key. Let me build it properly.

Functional Requirements (FR)

In scope:

  • Live viewer count per page. For any product page (a hotel, a flight, a listing), return an approximate count of distinct viewers currently on it: “23 people are looking right now.”
  • Presence tracking. A viewer opening a page is counted; a viewer who closes the tab, navigates away, or goes idle stops being counted within a bounded window (seconds to a minute).
  • Live updates. The number the viewer sees updates without a full page reload - it ticks up and down as others arrive and leave.
  • De-duplication. One person with the page open in three tabs, or refreshing, should count roughly once, not three times. The count is distinct humans, not raw hits.
  • Bounded staleness. The count may lag reality by a few seconds. It must never be wildly wrong (showing 500 when 5 are present) or stuck (frozen at a number after everyone left).
  • Graceful degradation. If the counting tier is degraded, the widget hides or shows a coarse bucket (“Popular today”) rather than a wrong precise number. It must never take down the product page.

Explicitly out of scope (own the scope):

  • The product page itself. Catalog, pricing, availability, and booking are separate systems. We only compute and serve the count.
  • Exact analytics. This is not the web-analytics pipeline that must count every session to the unit for billing. We want a good-enough live number, and we are allowed to approximate.
  • Per-user identity / who is viewing. We count distinct sessions, not named people. No “your friend Alice is looking.”
  • Fraud / bot filtering beyond basic dedup. We note where a bot filter hooks in but do not design it.
  • Historical time series. “How many looked at this page last Tuesday” is an offline analytics job, not this real-time widget.

The one decision that drives everything: the count is a derived, approximate, ephemeral aggregate over expiring presence sessions - not a durable stored value. That reframing is what makes the numbers close. We never need a transaction, never need durability, and we are allowed to be a few percent off. Every design choice below cashes in on that freedom.

Non-Functional Requirements (NFR)

  • Scale: 1M active product pages, 10M concurrent viewers globally at peak. The load skews hard: a few thousand pages (a viral deal, a city during a holiday) hold a large fraction of viewers, while the long tail has one or two viewers each.
  • Latency: the count on the page should render fast (part of the page paint budget, under ~100ms from cache) and update live with a p99 freshness of a few seconds. A heartbeat-to-count-change latency of ~2-5s is fine.
  • Availability: 99.9%+ on the read path. The widget failing must be invisible - the page renders without it. Writes (heartbeats) are fire-and-forget; dropping some is acceptable.
  • Consistency: eventually consistent and approximate. Two viewers of the same page can briefly see 22 vs 23. That is not a bug. What must hold: the number trends correctly and expires when viewers leave.
  • Durability: none required. Presence is inherently transient. If a counting node dies, its viewers re-heartbeat and the count rebuilds within one window. We deliberately keep this state in memory.
  • Accuracy target: within roughly +/-5-10% of true concurrency for hot pages, and “small integer that is basically right” for cold pages. Precision matters more when the number is small (showing 3 when it is 3), and approximation is fine when it is large (showing “120+” when it is 118).

That accuracy target is a subtle unlock: we can be sloppier as the number grows. Nobody cares if a page shows 340 when it is truly 355, but everyone notices if it shows 8 when it is 2. This lets us use exact counting for small pages and probabilistic counting for hot ones.

Back-of-the-Envelope Estimation (BoE)

Real numbers, because they dictate the architecture.

The naive write storm (why this is hard):

Concurrent viewers globally: 10,000,000
Heartbeat interval (browser says "still here"): every 10s
Naive: each heartbeat is a DB write to a per-page counter
  Heartbeat write rate = 10,000,000 / 10s = 1,000,000 writes/sec

A million presence writes per second, and they are not spread evenly - they pile onto the hot pages. If the top 1,000 pages hold 40% of viewers, that is 400K writes/sec landing on 1,000 keys, i.e. hundreds of writes/sec onto a single counter row. That is the classic hot-key write storm, and a single row cannot take it.

Read side:

Page views/sec (fresh page loads) globally: assume ~200,000/sec at peak
Each page load reads the count for its page: 200,000 reads/sec
Plus live-update polls/pushes for viewers already on a page.

Storage of live presence (if we stored sessions):

10,000,000 live sessions
Per session: session_id + page_id + last_seen  ~ 40 bytes
  = 10M * 40B = ~400 MB of live presence state
Fits comfortably in memory across a small Redis/in-memory fleet.

That 400MB number is the key insight on the storage side: all live presence fits in RAM. There is no reason to touch disk. Presence is small and transient.

Count cache:

1,000,000 active pages, each with one small integer count + a few metadata bytes
  ~ 1M * 50B = ~50 MB of "current count per page"
Trivially cacheable in memory, fully replicable to every read node.

Bandwidth of live updates:

If we push count changes: a change is tiny (~20 bytes: page_id + new count).
A hot page changing count ~1/sec, pushed to N viewers -> but we do NOT push
per-viewer per-change. We let viewers poll a cached number every few seconds,
or push coalesced updates. Poll: 10M viewers / 5s = 2M count-reads/sec,
each served from an in-memory cache -> cheap, and CDN/edge-cacheable for hot pages.

Takeaways: raw storage is nothing (400MB of presence, 50MB of counts, all in RAM). The entire difficulty is (a) absorbing ~1M heartbeat writes/sec without a hot-key storm, (b) expiring presence so the count drops when people leave, and (c) serving ~2M count-reads/sec cheaply. Every technique below - local aggregation, sharded counters, probabilistic counting, edge caching - exists to defeat one of those three.

High-Level Design (HLD)

Three planes. An ingest plane takes heartbeats from browsers and aggregates them before they ever hit a shared counter. A counting plane maintains per-page live counts with automatic expiry. A read plane serves the cached number to page loads and pushes live updates. The trick throughout: never let a single heartbeat become a single write to a shared hot key.

   Viewers' browsers (10M)                     Page loads (read count)
        │ heartbeat every 10s                        │ GET count
        │ (page_id, session_id)                      ▼
        ▼                                     ┌──────────────────┐
  ┌──────────────┐                            │  Edge / CDN cache │  hot pages
  │ Load Balancer │                           │  (count TTL ~2-5s)│
  └──────┬───────┘                            └────────┬──────────┘
         │                                             │ miss
  ┌──────▼───────────────┐                    ┌────────▼──────────┐
  │  Presence Ingest      │  local, in-memory  │  Count Read API    │
  │  Nodes                │  aggregation       │  (serves cached    │
  │  - dedup by session   │  per node          │   per-page count)  │
  │  - local sliding      │                    └────────┬──────────┘
  │    window per page    │                             │ reads
  │  - emit DELTAS only    │                            ▼
  └──────┬───────────────┘                     ┌────────────────────┐
         │ periodic delta flush                │  Count Store        │
         │ (page_id -> local live count)       │  (in-memory, per    │
         ▼                                      │   page current cnt, │
  ┌──────────────────────┐   aggregate         │   sharded by page)  │
  │  Aggregation /         │─────────────────▶ │  Redis / in-mem     │
  │  Counting Service      │  sum node-deltas   └─────────┬──────────┘
  │  (per page: sum of      │  per page                    │ pub/sub
  │   per-node live counts) │◀─────────────────────────────┘ count changed
  └──────────────────────┘                              push to read nodes

Write flow (a viewer’s presence is counted):

  1. On page open, the browser generates a session_id (stable per tab/session) and starts sending a lightweight heartbeat POST /presence with {page_id, session_id} every ~10s. It also sends a leave beacon on tab close (best-effort navigator.sendBeacon).
  2. The LB routes the heartbeat to any Presence Ingest node. Crucially, a given session_id is routed to the same ingest node (sticky by session hash) so that node owns that session’s liveness.
  3. The ingest node maintains, in memory, a per-page sliding window of live sessions it has seen: page_id -> {session_id -> last_seen}. A heartbeat refreshes last_seen. This is pure local state, no shared write.
  4. Periodically (every ~2s) the ingest node computes, for each page it holds sessions for, its local live count (sessions whose last_seen is within the window) and flushes that as a delta or absolute-per-node value to the Counting Service. One node emits one number per page, not one write per heartbeat.

Count flow (the number per page):

  1. The Counting Service keeps, per page, the set of per-node contributions: page_id -> {node_id -> node_local_count}. The page’s live count is the sum across nodes. When a node reports a new value for a page, the service updates that node’s slot and recomputes the sum.
  2. The summed count is written to the Count Store (in-memory, sharded by page_id) and, if it changed meaningfully, published so read nodes and edge caches refresh.

Read flow (a page shows its number):

  1. A page load calls GET /pages/{id}/viewer-count. For hot pages this is served straight from the edge/CDN cache with a short TTL (2-5s). On miss, the Count Read API returns the value from the in-memory Count Store.
  2. Viewers already on the page get live updates by either polling that cached endpoint every few seconds, or via a lightweight push (SSE) that sends the new number when it changes. Because the number is small and cached, either is cheap.

The core insight: aggregate at the edge of the write path, not at the center. Heartbeats collapse into per-node counts inside the node that receives them, so the shared counter only ever sees one number per node per page (a few hundred writes/sec globally per hot page at most), never one write per heartbeat (hundreds/sec per page). The million-writes-per-second storm is absorbed locally and never reaches shared state.

Component Deep Dive

The hard parts, naive-first then evolved: (1) absorbing the heartbeat write storm, (2) expiry so the count falls when people leave, (3) counting hot pages without exact per-session sets, (4) dedup and the read/update path.

1. Absorbing the heartbeat write storm

Naive approach: every heartbeat increments a counter. Browser pings “still here,” the server does INCR count:{page_id} (or UPDATE pages SET viewers = viewers + 1). To handle leaves, decrement on tab close.

on heartbeat(page_id, session_id):
    INCR count:{page_id}          # 1,000,000 INCRs/sec globally
on leave(page_id, session_id):
    DECR count:{page_id}          # unreliable - browsers do not always fire it

Where it breaks:

  • Hot-key write storm. A viral page with 5,000 viewers heartbeating every 10s is 500 INCRs/sec onto one key. Redis can do a lot, but concentrate hundreds of hot pages and the shard owning them saturates. A relational row is far worse - lock contention serializes it.
  • Double counting. INCR on every heartbeat counts the same session every 10s. You would count one viewer dozens of times over their visit. You would need INCR-only-on-first-heartbeat, which needs a “have I seen this session” check - another read/write per heartbeat.
  • Leaves are unreliable. Browsers do not reliably fire an unload beacon (crashes, killed tabs, dead networks). If you rely on DECR-on-leave, the count only ever goes up and gets stuck. This alone kills the naive design.

First evolution: count via presence + expiry, not increment/decrement. Do not maintain a running counter. Maintain a set of live sessions per page where each session expires if it stops heartbeating. The count is the size of the live set. Leaves become automatic: stop heartbeating, and within one window you fall out of the set. No unreliable DECR needed. This fixes correctness but a naive shared set per page still takes a write per heartbeat.

Second evolution: local aggregation on the ingest node (the storm killer). Route each session_id stickily to one ingest node. That node holds the session’s liveness in its own memory and refreshes last_seen on each heartbeat - a local map write, not a shared-store write. Every ~2s the node emits one summary per page: “node 7 currently sees 812 live sessions for page P.” The shared counter now takes one write per (node, page) per 2s, not one per heartbeat.

ingest node (local, in-memory):
  live[page_id][session_id] = last_seen        # local map, no shared write
  every 2s, for each page_id held:
      n = count sessions with (now - last_seen) < WINDOW
      report(node_id, page_id, n)              # ONE number to Counting Service

Do the math on the reduction. A hot page with 5,000 viewers spread over, say, 50 ingest nodes: each node holds ~100 of its sessions and reports once per 2s. The shared counter sees 50 nodes / 2s = 25 writes/sec for that page instead of 5,000 heartbeats / 10s = 500 writes/sec, and those writes are tiny idempotent overwrites, not read-modify-write increments. Globally, 1M heartbeats/sec collapse to (num_ingest_nodes * num_pages_per_node) / 2s node-reports - orders of magnitude smaller, and each report is an absolute value so lost reports self-heal on the next flush.

Before: 1,000,000 heartbeat writes/sec onto shared counters (hot-key storm)
After:  node-level reports: a few thousand small overwrites/sec, spread by page
        each report is idempotent absolute value -> lossy-safe, self-healing

This is the crux. The write path never touches shared mutable state per heartbeat; it collapses locally first.

2. Expiry - making the count fall when people leave

Naive approach: a background job scans all sessions and deletes stale ones. A cron every minute walks the presence store and removes sessions older than the window, then recomputes counts.

Where it breaks:

  • Scan cost and lag. Walking 10M sessions every minute to find expired ones is expensive, and the count lags by up to the scan interval - a page shows 23 for a full minute after everyone left.
  • Thundering deletes. A big event ending means millions of sessions expire at once; the scan-and-delete spikes hard.

The fix: TTL-based sliding windows, no scan. Two complementary mechanisms:

  • Local sliding window on the ingest node. When a node computes its per-page count, it only counts sessions with last_seen within the last WINDOW (say 30s). Expired sessions are simply not counted - no deletion needed at count time; they are lazily evicted from the local map on the next touch or by a cheap periodic sweep of only that node’s small map. Because the node recomputes and reports every 2s, the reported number naturally decays as heartbeats stop.
  • TTL on the shared per-node contribution. Each node’s reported value for a page is stored with a TTL slightly longer than its report interval (e.g. report every 2s, TTL 6s). If a node dies or stops reporting a page (its last viewer of that page left), its contribution auto-expires and drops out of the sum. The Counting Service never has to detect “node went away” - the TTL does it.
Count Store (Redis), per page:
  HSET pagecount:{page_id}  node_7  812   EX 6   # node contribution + TTL
  HSET pagecount:{page_id}  node_9  431   EX 6
  live_count(page) = SUM of non-expired node fields   # dead nodes fall off

Window choice:
  WINDOW  (session liveness)   = 30s   (2-3 missed heartbeats tolerated)
  report interval              = 2s
  contribution TTL             = 6s    (3 missed reports -> node drops)

The whole expiry model is “freshness by TTL, not deletion by scan.” A viewer stops heartbeating; within WINDOW the owning node stops counting them; the node’s next report reflects the lower number; and if a node dies entirely, its whole contribution ages out. There is never a scan of 10M rows, and the count decays smoothly within seconds. This directly implements the NFR that the count must never get stuck.

3. Counting hot pages without exact per-session sets

Naive approach: keep an exact set of session_ids per page and take its size. SADD viewers:{page_id} session_id per heartbeat, SCARD to read.

Where it breaks:

  • Memory and network for hot pages. A page with 50K viewers holds a 50K-element set; SADD per heartbeat is the write storm again, and merging sets across nodes to dedup a session seen by two nodes is expensive.
  • Cross-node dedup is hard. If sticky routing occasionally sends a session to two nodes (rebalancing, reconnect), exact counting double counts unless you union the actual id sets across all nodes - defeating local aggregation.

The fix: exact counting for small pages, probabilistic (HyperLogLog) for hot ones. Recall the accuracy NFR: precision matters when the number is small, approximation is fine when it is large. Exploit exactly that.

  • Small pages (the long tail): exact. Below a threshold (say < 100 viewers), keep the exact session set locally and sum small integers. Showing 3 when it is 3 matters, and the sets are tiny, so exactness is cheap.
  • Hot pages: HyperLogLog (HLL). For pages above the threshold, each ingest node maintains a HyperLogLog sketch of the session_ids it has seen live for that page instead of a full set. An HLL estimates cardinality of a multiset in a fixed ~12KB with ~1-2% error, and - critically - HLL sketches merge losslessly. The Counting Service merges the per-node HLLs for a page and reads the cardinality of the union. This solves cross-node dedup for free: a session_id seen by two nodes appears once in the merged sketch.
per node, per hot page:
  hll[page_id].add(session_id)           # ~12KB fixed, O(1) add
  every 2s: report(node_id, page_id, serialize(hll[page_id]))

Counting Service per hot page:
  merged = HLL.merge(all live node sketches)   # dedup across nodes, lossless
  live_count = merged.cardinality()            # +/-1-2% error, fine when big

This gives us cross-node de-duplication and bounded memory in one mechanism. A page with 100K viewers costs ~12KB per node regardless of viewer count, the count is within a couple percent (invisible at that scale), and sessions seen by multiple nodes are unioned correctly. The window is applied by rolling the HLL: nodes keep short-lived HLL slices (e.g. one per 10s bucket) and merge only the last WINDOW’s buckets, so expiry works on sketches too.

Page size Method Memory/node Error Why
< 100 viewers Exact session set tiny (KBs) 0 small numbers must be exact
100 - 100K HyperLogLog ~12KB fixed ~1-2% dedup + bounded memory
> 100K HyperLogLog + coarse display ~12KB fixed ~1-2% show “100K+” anyway

4. De-duplication and the read/update path

Naive approach: count every heartbeat source as a viewer, and have every page load hit the counting service. No session identity; each connection is a viewer. Every render reads through to the store.

Where it breaks:

  • Multi-tab / refresh inflation. One human with three tabs = three counts. Refreshes spawn new counts. The number lies high.
  • Read hot spot. 200K page loads/sec plus 10M viewers polling every few seconds = millions of reads/sec, all hitting the counting service on the hot pages.

The fix: stable session id for dedup, and a heavily cached read path.

  • Stable session_id. The browser derives a session_id that is stable across tabs and refreshes for the same user session (e.g. a first-party cookie or localStorage value, hashed). All tabs send the same id; the HLL/set counts it once. A logged-in user can key on user_id hashed with a salt, so multiple devices still dedup where desired. Bots and headless clients that never persist the cookie get a fresh id each time - a cheap first-line filter, with a proper bot check hooking in here.
  • Cache the count aggressively. The count for a page is a single tiny integer that changes slowly relative to how often it is read. Serve it with a short TTL from an edge/CDN cache for hot pages (a viral page’s count read millions of times is served from the edge, not origin), and from an in-memory replica on each read node for warm pages. A 2-5s TTL means the count is at most a few seconds stale - within the NFR - while collapsing millions of reads into one origin read per page per TTL.
GET /pages/{id}/viewer-count
  1. Edge cache (hot pages, TTL 3s) -> hit ~99% for viral pages
  2. Read node in-memory replica (warm pages)
  3. Count Store (origin)  -> only on cold miss
  Response: { "page_id": "...", "count": 23, "bucket": "20-30", "ts": ... }
  • Live updates without a push storm. Viewers already on a page need the number to tick. Two options, both cheap because the payload is tiny: (a) poll the cached endpoint every ~5s - 10M viewers / 5s = 2M reads/sec, almost all edge/replica hits; or (b) SSE push the new number only when it changes meaningfully (crossing a bucket boundary or +/-N), coalesced. For a widget this soft, polling a cached value is usually the simpler, cheaper choice; push is worth it only if you want sub-second liveness.
  • Display smoothing. To avoid a jittery number flickering 22/23/22, the client (or read node) smooths: round to buckets for large counts (“120+”, “50+”), and rate-limit visible changes. This also hides the last few percent of HLL error.

The read side mantra: the count is small, slow, and approximate, so cache it hard and never read through per viewer. One cheap integer, served from the edge, updated a few times a second - that is all any of the 10M viewers ever pulls.

API Design & Data Schema

Heartbeats are fire-and-forget writes; counts are cached reads; updates are poll or SSE.

Endpoints

POST /api/v1/presence/heartbeat
  body: { "page_id": "hotel_8821", "session_id": "s-3f9a...", "ts": 1721... }
  -> routed sticky by session_id; refreshes local last_seen; no response body needed
  Response 204 (fire-and-forget; client ignores failures)

POST /api/v1/presence/leave           (best-effort, navigator.sendBeacon on unload)
  body: { "page_id": "hotel_8821", "session_id": "s-3f9a..." }
  -> hint to drop session early; NOT relied upon (TTL expiry is the real mechanism)
  Response 204

GET /api/v1/pages/{page_id}/viewer-count
  -> served from edge/replica cache (TTL ~3s), origin on miss
  Response 200: { "page_id": "hotel_8821", "count": 23, "bucket": "20-30",
                  "approx": false, "ts": 1721... }

GET /api/v1/pages/{page_id}/viewer-count/stream      (optional SSE live updates)
  -> text/event-stream; server pushes { count } only when it changes meaningfully
     event: count
     data: { "count": 24, "bucket": "20-30" }

GET /api/v1/pages/viewer-count:batch?ids=hotel_8821,hotel_9002,...
  -> batch read for a search-results grid showing counts on many cards at once
  Response 200: { "hotel_8821": 23, "hotel_9002": 5, ... }

The heartbeat is deliberately 204, tiny, and safe to drop - losing one heartbeat means the session is still covered by the window (which tolerates 2-3 misses). The batch endpoint matters in practice: a search results page shows counts on 25 hotel cards at once, so one batched, cached read beats 25 round-trips.

Data stores

1. Local presence state - in-process memory on ingest nodes (no external store). This is the highest-churn state and it lives only in the node that owns the session. It is intentionally not persisted.

Ingest node (in-memory):
  # small pages: exact
  live[page_id] = { session_id -> last_seen_ms }         # evicted past WINDOW
  # hot pages: probabilistic, bucketed for windowing
  hll[page_id][time_bucket] = HyperLogLog(~12KB)          # merge last WINDOW buckets
Rebuildable: if a node dies, its sessions re-heartbeat elsewhere within WINDOW.

2. Count Store - Redis (in-memory, sharded by page_id). Holds each page’s per-node contributions with TTLs, and the derived current count. Chosen because the data is small, hot, ephemeral, and needs atomic field ops with TTL - exactly Redis’s shape. No SQL: there is nothing relational, no transactions, no durability need, and a single hot row is precisely the wall SQL would hit.

Redis, sharded by page_id (consistent hashing):
  # per-node contributions (exact small pages)
  HSET nodecount:{page_id}  {node_id} {n}   EX 6      # TTL auto-drops dead nodes
  live_count(page) = SUM of hash fields

  # per-node HLL sketches (hot pages) - stored/merged in the aggregation tier,
  # only the merged cardinality lands here:
  SET count:{page_id}  {merged_count}  EX 6

  # served count (smoothed, bucketed)
  SET display:{page_id} {count}|{bucket}  EX 4

3. Aggregation / Counting Service - stateful shards, keyed by page_id. Each aggregation shard owns a slice of pages (by hash(page_id)), collects the per-node reports/sketches for its pages, computes the sum or merged-HLL cardinality, and writes the display count. Keeping a page’s aggregation on one shard means all of its node-reports converge in one place cheaply.

4. Edge / CDN cache - for the read path. The viewer-count GET is cacheable with a short TTL keyed by page_id. Hot pages are served entirely from the edge.

Why in-memory + probabilistic over a database everywhere: the workload is a high-churn, approximate, ephemeral aggregate with zero durability requirement and a punishing hot-key profile. A durable database is the wrong tool on every axis - it adds persistence we do not want, transactions we do not need, and a single-hot-row ceiling that is exactly our failure mode. RAM, TTLs, sharded contributions, and HLL sketches match the problem precisely.

Bottlenecks & Scaling

Where it breaks first, in order, and the fix for each:

1. Heartbeat write storm (breaks first, and hardest). 10M viewers heartbeating = ~1M writes/sec, concentrated on hot pages. Fix: local aggregation on sticky-routed ingest nodes. A heartbeat is a local in-memory last_seen update, never a shared write. Each node emits one absolute count (or HLL) per page every ~2s. The shared store sees thousands of idempotent overwrites/sec, not a million read-modify-writes, and because reports are absolute, lost ones self-heal.

2. Hot key on the shared counter for a viral page. Even node-reports for one mega-page converge on one Redis key. Fix: the per-page state is a hash of per-node fields (HSET nodecount:{page} field per node), so writes spread across fields, and reads are one HGETALL+sum. For extreme pages, further shard the page’s contributions across K Redis keys (nodecount:{page}:{0..K}) and sum. The display count is written once and read from cache, so reads never hit this key directly.

3. Read hot spot on viral pages. Millions of viewers reading one page’s count. Fix: edge/CDN cache with a 2-5s TTL absorbs essentially all reads for hot pages; warm pages hit an in-memory replica on read nodes. Origin sees ~1 read per page per TTL. The batch endpoint collapses grid reads. Live updates are poll-a-cached-value or coalesced SSE, both tiny.

4. Expiry lag / stuck counts. Count must fall when viewers leave, without scanning 10M sessions. Fix: TTL-based sliding windows, not scans. Local windows stop counting stale sessions within WINDOW; per-node contributions carry a TTL so a dead or departed node’s slot ages out. The count decays smoothly in seconds; nothing is scanned.

5. Sticky routing failures / rebalancing double-count. If a session lands on two nodes, exact counting inflates. Fix: HLL merge dedups across nodes for free on hot pages (a session in two sketches unions to one); for small exact pages the double-count is at most a couple of sessions - negligible and invisible after bucketed display. Sticky routing (rendezvous/consistent hash on session_id) keeps this rare.

6. Ingest node loss. A node holding a slice of sessions crashes. Fix: stateless-ish and rebuildable. Its sessions re-heartbeat (every 10s) and get re-routed to a surviving node, rejoining the count within WINDOW. Its contributions in the Count Store TTL out. Momentary small dip in the count for its pages, then recovery - acceptable for an approximate widget.

7. Skew: a few pages hold most viewers. The long tail is one viewer; the head is 100K. Fix: the exact-vs-HLL threshold adapts per page - cheap exact sets for the millions of cold pages, fixed-size HLL for the hot few. Memory is bounded either way. Aggregation shards can rebalance hot pages onto dedicated shards if one page alone saturates a shard.

8. Shard keys, stated plainly. Ingest routing -> hash(session_id) (so one session is owned by one node -> local dedup and liveness). Count Store and Aggregation -> hash(page_id) (so a page’s contributions converge on one shard/key for summing). Read cache -> keyed by page_id. Routing heartbeats by page_id instead of session_id would be the classic mistake - it piles all of a hot page’s heartbeats onto one ingest node, recreating the hot spot on the ingest tier. Route writes by session (spread + dedup), aggregate by page (converge).

9. Single points of failure. Ingest nodes are interchangeable and rebuildable (presence re-heartbeats). Aggregation shards are replicated; a failed shard’s pages briefly stop updating (count freezes for seconds, then a replica takes over) - and freezing an approximate number for a few seconds is invisible. The Count Store is replicated Redis. Nothing here is durable, so there is no data to lose - only a few seconds of freshness to recover.

10. Multi-region for a global audience. Viewers of one page are worldwide. Fix: ingest and aggregate per region (a viewer heartbeats to their nearest region), then merge per-region sketches/counts into a global count. HLL merges losslessly across regions, so global concurrency for a page is cardinality(merge(region sketches)), computed by async cross-region exchange of ~12KB sketches per hot page every few seconds - a page’s count crosses the ocean as one small sketch, not as a million heartbeats. Because the number is approximate and eventually consistent, async regional merge is exactly right - no global coordination needed.

Wrap-Up

The trade-offs that define this design:

  • An approximate, ephemeral aggregate over expiring sessions, not a stored counter. We never persist the number and never increment a shared counter per heartbeat. The count is derived from live presence and decays by TTL, which makes leaves reliable (stop heartbeating and you age out) and makes the whole system rebuildable in one window with zero durability.
  • Aggregate at the edge of the write path, not the center. Sticky-routed ingest nodes collapse a heartbeat into a local memory write, and emit one number per page every couple of seconds. That single move turns ~1M writes/sec of hot-key contention into a few thousand idempotent, self-healing overwrites.
  • Exact for small, HyperLogLog for hot. Precision where it is noticed (small pages, exact sets) and fixed-size probabilistic counting where it is not (hot pages, ~12KB HLL with 1-2% error), with HLL’s lossless merge giving cross-node and cross-region de-duplication for free.
  • Cache the number hard. One tiny, slowly-changing integer served from the edge with a short TTL absorbs millions of reads per second; viewers poll or get coalesced pushes of a value that is at most a few seconds stale - well within an intentionally soft freshness budget.
  • TTL-driven expiry, never a scan. Local sliding windows plus TTL’d per-node contributions make the count fall smoothly as people leave and make dead nodes drop out automatically, with no 10M-row sweep.

One-line summary: a presence-counting system that beats the million-heartbeats-per-second hot-key storm by collapsing heartbeats into per-node in-memory counts (exact for small pages, HyperLogLog for hot ones), summing/merging those per page into an approximate live count that decays by TTL instead of by scan, and serving that one tiny number to 10M viewers from an aggressively cached, edge-served read path - approximate, ephemeral, and rebuildable by design, because “23 people are looking right now” only ever needs to be roughly right and never needs to be durable.