An unknown number rings your phone and, before you pick up, the screen says “Rajesh Kumar - Reported as spam 240 times.” You have never saved that number. Your phone has never seen it. So where did the name come from? The trick behind Truecaller is deceptively simple to state and genuinely hard to build: millions of other people have that number saved in their address book under some name, and Truecaller has quietly aggregated all of those private address books into one giant global reverse phone directory. You uploaded your contacts once; so did 350M other people; the union of everyone’s contacts is the product.

The core question this design has to answer is: given a phone number I have never seen, return the single most likely human name for it, plus a spam signal, in under 200ms, drawn from a consensus over billions of crowdsourced contact records. That one sentence hides every hard problem in the system. How do you ingest billions of contact rows from 350M phones without melting a database? How do you turn “1,900 people saved this number, 1,400 of them as some spelling of Rajesh, 300 as ‘Plumber’, 200 as ‘Do Not Pick’” into one clean display name? How do you serve that lookup fast enough that it beats the phone’s own ring animation? And how do you honour a person’s right to not be in a directory they never consented to join? Let me build it properly.

Functional Requirements (FR)

In scope:

  • Caller ID on an incoming call. When a number that is not in my local contacts rings, the app shows the crowdsourced name (or business name) for that number, ideally before I answer.
  • Manual number search. I can type a number into the app and look up who it belongs to.
  • Contact upload / sync. With my consent, the app reads my phone’s address book and uploads (phone_number, name_I_saved_it_as) pairs. Re-syncs periodically to pick up new contacts. This is the raw material for everyone else’s caller ID.
  • Spam identification. Users can tag a number as spam / telemarketer / scam. A number that is reported by many users is flagged, with a report count and category.
  • Name resolution by consensus. For a number saved under many different names by many people, the system resolves one canonical display name (and detects businesses vs individuals).
  • Reverse-lookup unlisting / privacy. A person can request that their number be removed from the searchable directory (right to be forgotten).

Explicitly out of scope (so I own the scope):

  • Placing or carrying the actual call. The telephony (VoIP, PSTN, dialer) is the OS / carrier’s job. We are an overlay that identifies the number; we do not route voice.
  • Full-text messaging / chat. Truecaller has an SMS and chat product; here I focus on the distinctive part, the crowdsourced caller-ID directory, not a messaging backend.
  • The identity / auth service. Sign-up, phone-number verification via OTP, and session tokens are a standard auth service I consume, not design.
  • Payments / UPI. A separate product surface.
  • Deep ML fraud modelling. I will build the spam aggregation and scoring pipeline, but the adversarial ML model that classifies novel scam patterns is its own problem; I treat the model as a scorer I call.

The one decision that drives everything: this is a read-heavy lookup service backed by a massive, continuously-rebuilt reverse index, where the write path (contact ingestion) is an enormous but latency-insensitive batch/stream problem, and the read path (caller ID at ring time) is a tiny-payload, single-key, ultra-low-latency point lookup. The two paths have almost nothing in common and must be designed separately. The heart of the system is the aggregation that sits between them: turning billions of noisy, conflicting, private contact rows into one authoritative number -> {name, spam_score} record.

Non-Functional Requirements (NFR)

  • Scale: 350M registered users. Average address book ~500 contacts, so raw contact records aggregated worldwide are on the order of 350M * 500 = 175 billion rows before dedup, collapsing to a few billion distinct phone numbers in the reverse index. Caller-ID lookups: every incoming call any user receives is a potential lookup, on the order of a few billion lookups/day.
  • Latency: the caller-ID lookup must return in well under 200ms p99 - the app has a very short window between the OS signalling an incoming call and the user seeing the screen. Ideally the common case is served from an on-device cache in single-digit ms with zero network. Contact upload latency is irrelevant to the user; it can take minutes.
  • Availability: 99.9%+ on the read/lookup path. If a lookup fails, the app degrades to showing just the raw number, which is the pre-Truecaller experience, so a miss is tolerable but should be rare. The ingestion path can be down for an hour and no user notices.
  • Consistency: eventual consistency is fine and expected. A new name for a number surfacing an hour or a day late is acceptable; the directory is a slowly-evolving aggregate, not a bank ledger. The one thing that must propagate promptly is a privacy unlisting (a person who opted out must stop appearing quickly) and a strong spam signal (a live scam campaign should flag fast).
  • Durability: the raw contact uploads and the derived reverse index are durable and replicated. Losing ingested contacts means degrading names until users re-sync, so we persist, but the derived index is always rebuildable from the raw store.
  • Privacy / legal: this is the defining non-functional constraint. We aggregate data about people who never signed up. That demands per-number unlisting, honouring regional data law (GDPR-style deletion), never exposing who saved a number under what, and only ever returning an aggregate. Privacy is not a feature bolted on; it shapes the schema.

Back-of-the-Envelope Estimation (BoE)

Real numbers with the arithmetic, because they set the architecture.

Contact ingestion (the write firehose, but latency-insensitive):

350M users, avg 500 contacts each, re-syncing.
Initial upload backlog = 350M * 500 = 175,000,000,000 raw contact rows.
Steady state: assume each user re-syncs deltas ~weekly, avg 20 changed
contacts/user/week.
= 350M * 20 / (7 * 86400)
= 7,000,000,000 / 604,800
≈ 11,600 contact upserts/sec average, call it ~15k/sec with peaks.

So the steady-state ingest is a modest ~15k rows/sec, but the initial and re-onboarding backlog is 175 billion rows that must be processed as bulk. The design must handle both a bulk backfill and a steady delta stream. Crucially, none of this is latency-sensitive.

Caller-ID lookups (the read path, latency-critical):

Assume an active user receives ~10 calls/day that trigger a lookup.
350M * 10 = 3,500,000,000 lookups/day
= 3.5B / 86400
≈ 40,500 lookups/sec average.
Peak (business hours, multiple time zones overlapping) ~3x
≈ 120,000 lookups/sec.

But a very large fraction of these never hit the backend at all, because the app caches results on-device and pre-fetches the user’s own contacts. Realistically maybe 30-50% are true cache misses that reach the service:

Backend lookup QPS ≈ 120,000 * 0.4 ≈ 48,000 QPS at peak.

A single-key point lookup at ~50k QPS is very tractable with the right store; the challenge is the p99 latency and the sheer size of the keyspace, not the QPS.

Storage:

Raw contact store (who-saved-what, needed for consensus + dedup + deletion):
  175B rows initial * ~64B/row (hashed number, hashed name, saver id, ts)
  ≈ 175B * 64B ≈ 11.2 TB, growing. Sharded, compressed, cold-ish.

Derived reverse index (the hot product):
  Distinct phone numbers ~2-3B worldwide with any presence.
  Per number: number(8B) + display_name(~40B) + spam_score(4B)
             + report_count(4B) + name_variants(compact, ~60B)
             + flags/meta(~20B) ≈ ~140B.
  2.5B * 140B ≈ 350 GB.

That derived index at ~350GB is the key insight: the thing users actually query is only a few hundred GB, small enough to shard across a modest fleet of in-memory / SSD key-value nodes and even to push high-value slices to the device. The 11TB of raw contacts is the messy backing store we compute from, never the thing we serve.

Bandwidth:

Ingest: 15k upserts/sec * ~64B ≈ 1 MB/sec steady (backfill is bulk, offline).
Lookup egress: 48k QPS * ~150B response ≈ 7 MB/sec. Trivial.

The takeaways: a bulk 175B-row backfill plus a modest ~15k/sec steady contact stream on the write side; a latency-critical ~50k QPS single-key lookup on the read side; an 11TB raw store we aggregate from; and a ~350GB derived reverse index that is the actual served product, small enough to cache aggressively and even ship slices to the device. The aggregation pipeline that turns raw into derived is the spine; the read path is a cache problem and the write path is a batch problem.

High-Level Design (HLD)

The architecture splits cleanly into three planes: an ingestion plane (phones upload contact books; we normalize, hash, dedup, and land raw rows), an aggregation plane (a pipeline that reads raw rows and computes, per number, a consensus name and a spam score into the derived reverse index), and a serving plane (a low-latency lookup service, fronted by on-device and edge caches, that answers “who is this number” at ring time). Privacy actions (unlisting, deletion) cut across all three.

   ┌───────────────┐   contact sync (consented, hashed)   ┌────────────────┐
   │  User phone    │ ───────────────────────────────────▶ │ Ingestion API  │
   │  - address book│                                       │ (normalize,    │
   │  - on-device   │ ◀──── caller-ID lookup / prefetch ─── │  hash, auth)   │
   │    cache       │                                       └───────┬────────┘
   └──────┬─────────┘                                               │ produce
          │ lookup(number) at ring time                    ┌───────▼────────┐
          │                                                 │  Kafka (raw     │
   ┌──────▼───────────────┐                                 │  contact events)│
   │   Lookup Service      │                                 └───────┬────────┘
   │  (edge, read-only)    │                                         │ consume
   │   ┌───────────────┐   │                                 ┌───────▼────────┐
   │   │ Reverse-index  │◀──┼──── rebuilt periodically ────── │  Raw Contact   │
   │   │ cache (hot)    │   │                                 │  Store (11 TB, │
   │   └───────┬───────┘   │                                 │  sharded KV)   │
   └───────────┼───────────┘                                 └───────┬────────┘
               │ miss                                                │
   ┌───────────▼─────────────┐    Aggregation Pipeline       ┌───────▼────────┐
   │  Reverse Index Store      │◀── (Spark/Flink jobs:        │  Consensus +   │
   │  (number -> {name,        │     dedup, vote, spam score) │  Spam Scorer   │
   │   spam_score, ...})       │◀────────────────────────────│  (batch+stream)│
   │  sharded by number        │                              └────────────────┘
   └───────────┬───────────────┘
               │
   ┌───────────▼───────────┐   ┌────────────────────┐   ┌────────────────────┐
   │ Unlist / Privacy       │   │ Spam Report Ingest  │   │ Business / Verified │
   │ Service (opt-out,      │   │ (user tags a number │   │ Directory (self-    │
   │ GDPR delete, blocklist)│   │  as spam)           │   │ registered names)   │
   └────────────────────────┘   └────────────────────┘   └────────────────────┘

Contact ingestion flow (write path):

  1. With explicit consent, the app reads the phone’s address book and computes, for each contact, a normalized number and the name the user saved it under. It uploads deltas (new/changed/removed contacts), not the whole book each time.
  2. The Ingestion API authenticates the user, normalizes each number to E.164, and lands a raw event {number, saved_name, saver_id, action, ts} onto a Kafka topic keyed by number.
  3. A consumer writes each event into the Raw Contact Store, a sharded KV/wide-column store partitioned by number, which holds “who saved this number as what.” This store exists for three reasons: recomputing consensus, deduping repeat uploads, and honouring deletion requests (we must know which raw rows to purge).

Aggregation flow (the spine):

  1. The Aggregation Pipeline (batch Spark jobs for the periodic full rebuild, plus a streaming Flink job for near-real-time deltas) reads raw contact rows grouped by number.
  2. For each number it runs consensus name resolution (deep dive 2) and spam scoring (deep dive 3), producing one record {number -> display_name, name_variants, spam_score, report_count, category, is_business}.
  3. It writes that record into the Reverse Index Store, sharded by number. This is the authoritative derived product.

Lookup flow (read path):

  1. An unknown number rings. The app first checks its on-device cache (its own synced contacts plus recently-seen numbers). Hit -> shown instantly, zero network.
  2. Miss -> the app calls the Lookup Service at the nearest edge. The Lookup Service checks its hot reverse-index cache; on a cache miss it reads the Reverse Index Store shard for that number.
  3. It returns {display_name, spam_score, category, is_business}; the app renders the caller-ID card and caches the result on-device with a TTL.

Privacy flow (cross-cutting): an unlisting or deletion request goes to the Privacy Service, which (a) tombstones the number in the Reverse Index Store so lookups stop returning a name, (b) schedules purge of the person’s raw rows, and (c) pushes an invalidation to the caches.

The key insight: the write path is a giant, latency-tolerant aggregation, and the read path is a tiny, latency-critical point lookup, connected by a derived index we rebuild continuously. We never compute consensus at read time; we precompute it and serve a single flat record per number.

Component Deep Dive

The hard parts, naive-first then evolved: (1) ingesting and deduping billions of contact records, (2) consensus name resolution - the core problem, (3) spam aggregation and scoring, (4) the sub-200ms lookup path and its caching, plus (5) privacy and unlisting.

1. Ingesting and deduping billions of contact records

Naive approach: on each sync, the app uploads the entire address book, and the server inserts a row per contact into a big contacts table.

Where it breaks:

  • Re-upload amplification. A user with 500 contacts syncing weekly re-uploads all 500 rows even if nothing changed. 350M * 500 weekly = 175B rows/week of mostly-duplicate writes, an enormous waste that buries the store in redundant data.
  • No delta awareness. You cannot tell an added contact from an unchanged one, so you cannot cheaply know that a number stopped being saved by someone (which matters for deletion and for decaying stale names).
  • Relational insert throughput. Bulk-inserting billions of rows into a relational table with indexes is hopeless; index maintenance and WAL dominate.
  • Storing raw names in the clear is a privacy and legal liability of the worst kind.

First evolution: client-side delta detection + hashing. The app keeps a local fingerprint of the last synced book and uploads only the diff (added/changed/removed contacts). It normalizes numbers to E.164 and hashes both the number and the saved name before upload where possible, so the backend stores tokens, not plaintext. This cuts steady-state volume from 175B/week to the ~15k upserts/sec computed in BoE. Better, but we still need a store that ingests fast and dedups per number.

The answer: an append-only event log into a number-sharded wide-column store, with idempotent upserts keyed by (number, saver).

  • Normalize then key by number. Every incoming contact is normalized to E.164 (country code inferred from the uploader’s region and the number format). The event is produced to Kafka keyed by number, so all contact events for a given number land on the same partition and are processed in order.
  • Raw Contact Store keyed by number, clustered by saver_id. The row key is the number; within it, one entry per saver: raw[number][saver_id] = {saved_name_hash, updated_at, action}. An upsert from the same saver overwrites in place, so re-syncs are idempotent and natural dedup falls out - we store one current row per (number, saver) pair, not per upload.
  • Dedup and count fall out of the schema. “How many distinct people saved this number, and under what names” is a single-partition scan of raw[number], which is exactly the input the consensus step needs. We never scan a global table.
  • Removals are first-class. When a user deletes a contact, the delta carries a REMOVE action; we drop that (number, saver) row. This lets a name decay when the people who saved it stop saving it, and it is essential for deletion compliance.
Ingestion:
  normalize number -> E.164
  event = { number, saver_id, saved_name_hash, action: ADD|UPDATE|REMOVE, ts }
  produce to Kafka(topic=contacts, key=number)

Consumer -> Raw Contact Store (wide-column, partition key = number):
  raw[number] = { saver_id -> { saved_name_hash, updated_at } , ... }
  upsert (number, saver_id) is idempotent   # re-syncs dedup naturally
  REMOVE deletes the (number, saver_id) entry

Why the Raw Contact Store must exist and cannot be skipped: consensus and spam scoring both need the full multiset of who-saved-what per number, we must dedup repeat uploads, and deletion requires knowing exactly which raw rows to purge. The derived index alone cannot satisfy any of those, so we keep both: an 11TB raw store we aggregate from, and a 350GB derived store we serve.

2. Consensus name resolution (the core problem)

This is the distinctive Truecaller problem. A single number is saved by thousands of people under many different, noisy, conflicting names. We must resolve one canonical display name. Consider a real spread for one number:

raw[number] name multiset (after dedup by saver):
  "Rajesh Kumar"      x 812
  "Rajesh"            x 410
  "Rajesh Plumber"    x 190
  "Plumber Rajesh"    x  95
  "Do Not Pick"       x  60
  "Spam"              x  40
  "Kumar Rajesh ji"   x  22
  ...long tail of one-offs...

Naive approach: return the single most-frequent exact string. Count exact name strings, pick the top one. Here that returns “Rajesh Kumar” (812), which happens to be fine.

Where it breaks:

  • Spelling and formatting fragmentation. “Rajesh Kumar”, “Rajesh kumar”, “rajesh kumar”, “Rajesh Kumar .” are different strings but the same person. Exact counting splits the vote and can let a smaller-but-consistent variant win, or produce an ugly cased result.
  • Labels are not names. “Do Not Pick”, “Plumber”, “Spam”, “Block This” are how people label a number, not the person’s name. Naive top-count can surface “Plumber” or, worse, “Do Not Pick” as the caller ID, which is both wrong and embarrassing.
  • Sybil / manipulation. If 500 fake accounts all save a number as “Best Bank Official”, exact counting hands attackers the display name - a phishing vector.
  • Businesses vs individuals. “HDFC Bank”, “HDFC”, “HDFC Customer Care” should resolve to a business entity, not be treated like a personal name.

First evolution: normalize and cluster before counting. Lowercase, trim, collapse whitespace, strip punctuation and honorifics (“ji”, “sir”, “bro”), then group near-duplicate strings (edit distance / token-set similarity) into clusters and count clusters, not raw strings. Now “Rajesh Kumar” / “Rajesh kumar” / “Kumar Rajesh ji” collapse into one cluster with ~834 votes, and we pick a canonical, nicely-cased representative from the cluster. This fixes fragmentation but not the label problem or manipulation.

The answer: cluster + classify + weighted consensus with a confidence threshold.

  • Token normalization and clustering as above: fold case, whitespace, punctuation, honorifics; cluster variants by token-set similarity into candidate name groups, each with a vote count and a canonical display form (most common well-cased variant).
  • Classify each cluster as name vs label vs business. A dictionary + light classifier tags clusters like “do not pick”, “spam”, “plumber”, “block”, “unknown” as labels, not personal names. Labels are excluded from the name competition but feed spam/category signals (many “spam”/“do not pick” labels raise the spam score). Business clusters (matching a business dictionary / verified directory, or with commercial tokens like “bank”, “customer care”, “delivery”) route to the business path.
  • Weighted votes to resist Sybil. Not every save counts equally. Weight each vote by trust signals: account age, verified phone, whether the saver themselves is well-attested, and diversity of the saving population (500 saves from accounts created yesterday on one IP block count for far less than 500 saves from long-lived, geographically spread accounts). This blunts manipulation.
  • Pick the winner only above a confidence threshold. The top name cluster wins only if it clears both an absolute floor (a handful of independent savers) and a relative margin over the runner-up. Below threshold we show the number with no name (honest “unknown”) rather than a low-confidence guess. This is the difference between “we know” and “we are guessing,” and guessing is what produces embarrassing or dangerous caller IDs.
  • Recency decay. Recent saves weigh more than years-old ones, so a reassigned number (numbers get recycled) gradually sheds its old identity as new people save it under a new name.
resolve(number):
  entries = raw[number]                       # {saver -> saved_name_hash}
  # (aggregation runs on de-hashed/tokenized names inside the secure pipeline)
  clusters = cluster_by_similarity(normalize(names))
  for c in clusters:
     c.type  = classify(c)                     # NAME | LABEL | BUSINESS
     c.score = sum(weight(saver) * recency(ts) for saver in c)
  name_clusters = [c for c in clusters if c.type == NAME]
  top = argmax(name_clusters, key=score)
  if top.score >= FLOOR and top.score >= MARGIN * second(name_clusters).score:
      display_name = top.canonical_form
  else:
      display_name = None                      # honest unknown, show number only
  is_business = any(c.type == BUSINESS and c.score high)
  spam_signal = aggregate(LABEL clusters like "spam"/"do not pick")  # feeds deep dive 3

The elegant part: we treat the crowd’s saved names as noisy weighted votes, cluster away spelling noise, throw out labels that are not names, discount manipulation, and only commit to a name when the consensus is confident - otherwise we say nothing rather than something wrong. This runs in the offline pipeline, not at read time, so we can afford the clustering and weighting; the result is one flat record the lookup path serves in one hop.

3. Spam aggregation and scoring

Naive approach: a number is spam if the raw count of “report spam” taps exceeds a fixed threshold, say 100.

Where it breaks:

  • No time sensitivity. A scam campaign burns a fresh number for a day; by the time it crosses a lifetime threshold of 100 the campaign has moved to a new number. A flat lifetime count is too slow for the exact case that matters most.
  • Popularity != spam volume. A legitimate high-traffic number (a bank, a delivery service) receives many calls and a few mistaken “spam” taps; a raw count punishes popular-but-legitimate numbers.
  • Retaliation / brigading. Coordinated false reports can blacklist a rival’s number.
  • No categories. “Telemarketer”, “scam”, “robocall”, and “OTP fraud” are very different; a single boolean loses that.

The answer: a rate-and-ratio spam score over a sliding window, with weighting and category aggregation, computed in the streaming pipeline.

  • Report events stream in. Each spam tag is an event {number, reporter_id, category, ts} on a Kafka topic, landed and aggregated by number.
  • Score on rate and ratio, not raw count. Maintain, per number over a sliding window (say 24h and 7d), the count of distinct reporters and, where known, the ratio of reports to total call volume for that number. A spam score combines: report velocity (reports/hour spiking), reporter diversity (many distinct, trusted reporters, not one ring), the reports-to-calls ratio, and category consensus. A number ringing thousands of strangers and getting reported by 2% of them within an hour scores high fast; a bank getting 50 mistaken taps against millions of calls does not.
  • Weight reporters the same way we weight name votes: trusted, long-lived, diverse reporters count; a cluster of new accounts from one source is discounted, defeating brigading.
  • Category by majority. The displayed category (“Telemarketer” / “Scam” / “Robocall”) is the majority label among recent reports.
  • Fast path for live campaigns. Because scoring runs in the streaming job over a short window, a number that suddenly spikes crosses the threshold within minutes and is flagged well before a lifetime-count approach would react. The streaming score is written straight to the Reverse Index Store’s spam fields, bypassing the slower full name-consensus rebuild.
  • Verified business allowlist protects legitimate high-volume numbers: numbers in the verified business directory get a much higher spam threshold and a “Verified Business” badge, so a delivery service is not blacklisted by stray taps.
spam_score(number) = f(
    velocity   = distinct_reporters_last_1h,
    diversity  = trust_weighted_distinct_reporters,
    ratio      = reports / max(call_volume, 1),
    recency    = decay(old reports),
) with verified-business override raising the threshold.
category = majority(recent report categories)

The point: spam is a time-sensitive, rate-based, weighted signal, not a lifetime counter - so we score it in a streaming window that reacts in minutes, weight reporters to defeat brigading, and use ratios so popularity is not mistaken for spam.

4. The sub-200ms lookup path

Naive approach: at ring time, the app calls a backend that runs the consensus query over the raw contact rows for that number and returns the name.

Where it breaks:

  • Consensus at read time is far too slow. Clustering and weighting thousands of raw rows per lookup, 50k times a second, blows the 200ms budget by orders of magnitude and re-does identical work on every call.
  • Cold network on every call. A round trip to a distant region can itself eat 150ms+ before any compute; the app must often show the card before the network even returns.
  • The keyspace is huge. A few billion numbers means the store must be fast on a single-key point read across a large keyspace.

The answer: precompute the flat record, serve it from a number-sharded KV store behind layered caches, and push high-value slices to the device.

  • Nothing is computed at read time. The lookup is a single point read of the precomputed reverse_index[number] record. All the hard work (consensus, spam scoring) happened offline/streaming.
  • Reverse Index Store sharded by number. A distributed KV / SSD-backed store (think Cassandra or a RocksDB-based service) partitioned by number gives O(1) single-key reads across billions of keys. At ~350GB it is small enough to keep hot data in memory across the fleet.
  • On-device cache first. The app caches (a) the user’s own synced contacts and (b) results of recent lookups, with a TTL. A large fraction of incoming calls hit this and return in single-digit ms with zero network - the ideal, because it beats the ring animation and works offline. Truecaller famously works even with the phone offline for numbers already on device.
  • Edge lookup service + hot cache. True cache misses go to the nearest edge Lookup Service, which fronts the KV store with an in-memory LRU of hot numbers (spam numbers and businesses are looked up disproportionately, so they cache extremely well). Regional replicas keep the round trip short.
  • Bloom-filter negative cache. Most random unknown numbers have no directory entry. A bloom filter (or the on-device slice) lets the app and edge answer “definitely no data” without a full lookup, saving a store read for the common empty case.
  • Optional on-device slice. For markets where numbers are dense, the app can carry a compressed local slice of the highest-value entries (top spam numbers, local businesses), so even first-time lookups for those are offline.
lookup(number) on incoming call:
  1. on-device cache?          -> hit: render in ~ms, no network. done.
  2. on-device spam/biz slice? -> hit: render offline. done.
  3. edge Lookup Service:
        hot LRU cache? -> return
        bloom says absent? -> return "no data" (show number only)
        else read reverse_index[number] from KV shard -> return + cache
  4. app caches result on-device with TTL; renders caller-ID card.

This is the standard read-heavy lesson: precompute the answer, serve it as a single flat record from a sharded KV store, and cache aggressively at the edge and on the device so the hot and empty cases never touch the store. The 200ms budget is met because at ring time we do a lookup, never a computation.

5. Privacy, unlisting, and deletion

Because we hold data about people who never signed up, this is not optional. A person must be able to say “remove my number from your directory.”

  • Unlist request (from an unlisting web form, verified by OTP to prove control of the number): the Privacy Service writes a tombstone for that number in the Reverse Index Store so lookups return “no data” regardless of how many people have it saved, and pushes a cache invalidation so on-device and edge caches drop it.
  • Deletion of a user’s contributed data (GDPR-style): purge every raw row where saver_id = that user across the Raw Contact Store, so their uploaded contacts no longer feed anyone’s consensus. Because raw rows are keyed by number and clustered by saver, this is a targeted delete, though it fans across many number partitions and runs as a background job.
  • Blocklist for numbers, allowlist for verified businesses, both consulted by the pipeline.
  • We only ever return an aggregate. A lookup returns a consensus name, never “you were saved as X by person Y.” Individual attribution never leaves the pipeline; the serving record is anonymized by construction.
  • Fail toward privacy. A tombstone always wins over a computed name, and an unlisting propagates ahead of the slower name rebuild via direct cache invalidation.

API Design & Data Schema

The lookup path is a tiny read; ingestion and privacy are separate surfaces.

Client-facing REST

POST /api/v1/contacts/sync
  body: { deltas: [ { number, saved_name, action:"ADD|UPDATE|REMOVE" }, ... ],
          device_id }
  -> { accepted: 1820, rejected: 0, next_full_sync_after: "7d" }

GET  /api/v1/lookup?number=+919812345678
  -> { number:"+919812345678",
       name:"Rajesh Kumar", is_business:false,
       spam_score:0.07, spam_category:null, report_count:3,
       confidence:"high" }
  -> (unlisted/absent) { number:"+91...", name:null, no_data:true }

POST /api/v1/spam/report
  body: { number, category:"telemarketer|scam|robocall|other" }
  -> { number, spam_score:0.83, report_count:241 }

POST /api/v1/privacy/unlist          // verified by OTP on that number
  body: { number, otp_token }
  -> { number, status:"unlisted" }

POST /api/v1/privacy/delete-my-data   // remove everything I uploaded
  -> { status:"scheduled", eta:"72h" }

Data stores - the right tool per plane

1. Raw Contact Store - wide-column NoSQL, partition key number, clustering key saver_id. Holds who-saved-what, the input to consensus, and the target of deletion. Huge (11TB+), write-heavy, no cross-row transactions, needs a single-partition “all savers of this number” read - a textbook wide-column (Cassandra / HBase / Bigtable) fit. Relational buys nothing and could not ingest the volume.

Table: raw_contacts        (Cassandra-style)
  number        TEXT      PARTITION KEY      -- E.164
  saver_id      BIGINT    CLUSTERING KEY
  saved_name_h  BLOB                          -- tokenized/hashed name
  updated_at    TIMESTAMP
  Query: all savers of a number = one partition scan (feeds consensus).
  Delete-my-data: delete where saver_id = X across partitions (bg job).
  Shard by: number.

2. Reverse Index Store - sharded KV, key number. The precomputed served product. Point reads on a single key across billions of keys, ~350GB, read-heavy, eventually consistent - a KV / SSD store (Cassandra, or a RocksDB-backed service) sharded by number. This is the only store on the hot read path.

Table: reverse_index       (key-value, key = number)
  number        TEXT   PRIMARY KEY
  display_name  TEXT                 -- null if below confidence or tombstoned
  name_variants LIST<{name,votes}>   -- top clusters, for debugging/UX
  is_business   BOOL
  verified_biz  BOOL                 -- from verified directory
  spam_score    FLOAT
  spam_category TEXT
  report_count  INT
  confidence    TEXT                 -- high | low
  tombstoned    BOOL                 -- unlisted; overrides display_name
  updated_at    TIMESTAMP
  Index: primary key only; lookups are single-key point reads.
  Shard by: number.

3. Spam Report Store / stream state - streaming aggregate keyed by number. Sliding-window counters (reporters, categories, velocity) maintained by the Flink job; the derived spam fields are written into the reverse index. Backed by a KV store for durability of the window state.

spam_state[number] = { reporters_1h: HLL/set, reporters_7d, categories: map,
                        call_volume_est, last_report_ts }

4. Verified Business Directory - relational, small. Self-registered / verified businesses with authoritative names, logos, and higher spam thresholds. Modest volume, strong consistency, joins to billing - relational is right here. Feeds the pipeline’s business classification and the allowlist.

5. Privacy / tombstone + blocklist - KV, keyed by number, replicated for fast propagation. Consulted by both the pipeline and the Lookup Service; a tombstone here always overrides a computed name.

Why NoSQL/KV on the hot paths and relational only for the small business directory: the two large stores have no transactional entity per request, no joins, and shard cleanly on number; they need massive write ingest (raw) and massive single-key read throughput (reverse index), which is exactly what wide-column and KV stores deliver and relational does not. The only place with real relational needs - verified businesses with billing and authoritative joins - is small and slow-changing, so it lives in Postgres.

Bottlenecks & Scaling

Where it breaks first, in order, and the fix for each:

1. Consensus at read time (breaks first if you let it). Computing the name from raw rows per lookup is orders of magnitude too slow at 50k QPS. Fix: precompute. The offline/streaming aggregation writes one flat record per number; the read path is a single point read. Never cluster or vote at ring time.

2. Reverse-index read hot spots. Spam numbers and popular businesses are looked up far more than random numbers. Fix: layered caching - on-device cache, edge LRU, and a bloom-filter negative cache. Hot spam/business entries live entirely in memory; the empty-number case (the majority) is answered by the bloom filter without a store read. The store only sees genuine, cold, present-number misses.

3. The bulk backfill and full re-aggregation. Recomputing consensus over 175B raw rows for a few billion numbers is a massive batch job. Fix: incremental aggregation. The full Spark rebuild runs infrequently (weekly); a Flink streaming job updates only the numbers whose raw rows or spam reports changed, so day-to-day the pipeline touches a tiny fraction of numbers. Partition the batch by number-hash for embarrassingly parallel processing.

4. Sharding key. Everything hinges on number as the shard key. Fix: shard by number (hashed) across raw, reverse-index, and spam stores. It spreads load evenly (numbers are naturally high-cardinality), co-locates all data for a number on one partition (single-partition consensus and single-key lookup), and matches the Kafka partition key so ingestion and aggregation align. There is no natural hot number on the write side; on the read side hot numbers are absorbed by cache, not by the shard.

5. Sybil / manipulation of names and spam. Coordinated fake accounts try to install a phishing display name or brigade a rival number. Fix: trust-weighted votes and reports (account age, verification, source diversity), confidence thresholds (say nothing rather than commit a manipulated low-diversity name), and verified-business allowlisting. Manipulation needs many diverse, aged accounts, which is expensive and detectable.

6. Slow reaction to live scam campaigns. A burner number must be flagged in minutes, not after a lifetime count. Fix: the streaming spam scorer over a short sliding window writes spam fields to the reverse index directly, bypassing the slower name rebuild, so a spiking number is flagged within minutes and propagated to caches by invalidation.

7. Privacy propagation lag. An unlisted person must disappear promptly. Fix: a tombstone that overrides any computed name, written directly to the reverse index and pushed as a cache invalidation to edge and device, so unlisting takes effect ahead of the next aggregation cycle. Deletion of contributed data runs as a targeted background purge keyed by saver.

8. Number recycling / stale identities. Telecoms reassign numbers; an old name must not haunt a new owner. Fix: recency decay in consensus so old saves fade as new savers overwrite the identity, plus honouring unlist requests from new owners.

9. Single points of failure and multi-region. Fix: Kafka, the KV stores, and the reverse index are replicated; the Lookup Service is stateless and regionally replicated so the latency-critical path stays local. The aggregation pipeline is offline, so its outage never affects lookups - the served index just goes stale, which is invisible for hours. On-device caching means even a full backend outage degrades only to “no data for uncached numbers,” never a hard failure.

Wrap-Up

The trade-offs that define this design:

  • Precomputed derived index over read-time consensus. We refuse to compute a name at ring time; we run clustering, weighting, and scoring offline and serve one flat record per number, trading freshness (a name can be hours stale) for a read path that is a single sub-200ms point lookup.
  • Two stores, two shapes. An 11TB write-optimized raw store we aggregate from and delete from, and a 350GB read-optimized reverse index we serve, connected by a continuously-rebuilt pipeline. Neither could do the other’s job.
  • Weighted consensus with a confidence floor over naive top-count. We cluster away spelling noise, discard labels that are not names, discount Sybil votes, and only commit to a name when the crowd is confident - otherwise we honestly show nothing, because a wrong or manipulated caller ID is worse than none.
  • Streaming, rate-based spam over a lifetime counter. Spam is scored on velocity, reporter diversity, and reports-to-calls ratio in a short window, so live campaigns flag in minutes and popularity is never mistaken for spam.
  • Caching down to the device over always-hitting the backend. On-device contacts, edge LRUs, and a bloom-filter negative cache mean the hot and empty cases never touch the store, and lookups work offline - which is why the caller-ID card can beat the ring animation.
  • Privacy as a first-class, fail-toward-safe constraint. Because we aggregate data on non-users, tombstones override computed names, deletion is a targeted purge, and we only ever return an anonymized aggregate.

One-line summary: a read-heavy caller-ID service where 350M phones upload deduped, hashed contact deltas into an 11TB number-sharded raw store, an offline-plus-streaming pipeline resolves one confident consensus name and a rate-based spam score per number into a 350GB reverse index, and an unknown number rings are answered in under 200ms by a single-key lookup served from on-device, edge, and bloom-filtered caches - eventually consistent everywhere except a fail-closed privacy tombstone and a fast-reacting spam signal.