A password manager looks like a trivial CRUD app. Store some login records, sync them across the user’s phone and laptop, auto-fill them into websites. One table, a REST API, done. The interviewer lets you believe that for about thirty seconds, then asks the one question that reorganizes the entire system: can your servers read the passwords they store? The correct answer - the only answer a serious password manager can give - is no. Not “we encrypt at rest with a key we hold.” No. The server must be structurally incapable of reading a single stored credential, even if the entire database is stolen, even if a rogue employee has root, even if you are legally compelled to hand over everything.

That single constraint - zero-knowledge, end-to-end encryption where the server is a dumb encrypted-blob store - drives every other decision. It means encryption and decryption happen only on the client. It means the key that unlocks the vault is derived from the master password and never leaves the device. It means sync is syncing ciphertext, and the server cannot merge, search, or validate the plaintext. It means when you need breach notifications you cannot just scan everyone’s passwords server-side, because you do not have them. Let me build it properly, starting from the crypto and letting the distributed-systems parts fall out of it.

Functional Requirements (FR)

In scope:

  • Vault storage. A user stores login items: site URL, username, password, notes, TOTP seeds, and other secret fields. Items are grouped into one or more vaults.
  • End-to-end encryption. The server stores only ciphertext. It can never derive the plaintext of any secret field. This is the product, not a feature.
  • Multi-device sync. The same vault is available on the user’s phone, laptop, tablet, and browser extension, kept in sync. An edit on one device shows up on the others within seconds.
  • Auto-fill. The browser extension / mobile OS integration detects a login form, matches it to a stored item by domain, and fills the username and password on user action.
  • Item generation. Generate strong random passwords client-side.
  • Breach notifications. Tell a user if one of their stored passwords has appeared in a known data breach, without the server ever learning the password.
  • Account recovery. A way back in if the user forgets the master password, that does not undermine zero-knowledge (this is genuinely hard and mostly a trade-off, covered below).
  • Sharing. Share a specific item or vault with another user (family plan, team) while keeping it end-to-end encrypted.

Explicitly out of scope (say it so you own the scope):

  • The cryptographic primitives themselves. I will pick and justify algorithms (Argon2, AES-256-GCM, X25519, RSA/ECDH for sharing) but I am not designing a new cipher. Use vetted, standard primitives; rolling your own crypto is how you end up on a breach blog.
  • The browser-extension DOM heuristics. Field detection is a real, messy problem, but it is a client-side parsing concern, not a distributed-systems one. I will note the interface it needs.
  • Billing, SSO/SAML for enterprise, admin consoles. Each is its own system.
  • Passkeys/WebAuthn as a replacement for passwords. Worth a sentence at the end; the manager can store passkeys, but designing the FIDO2 ceremony is separate.

The one decision that drives everything: the server is an untrusted, zero-knowledge sync backend for opaque encrypted blobs. Every hard problem - sync, conflict resolution, sharing, breach checks, recovery - has to be solved without the server being able to read the data. That is the whole game.

Non-Functional Requirements (NFR)

  • Scale: ~100M registered users, ~40M daily active. Average vault ~150 items; power users into the thousands. Assume ~15B total stored items.
  • Latency: unlock (local decrypt) must feel instant, under ~1s including the deliberately slow key derivation. Sync pull p99 under 300ms. Auto-fill match is local, sub-50ms.
  • Availability: 99.99% on the sync/read path. But note the killer property: the app works fully offline. The vault is cached and decrypted locally, so a backend outage degrades sync, not access. This relaxes availability pressure enormously compared to a system where the server is on the critical read path.
  • Consistency: per-item, last-writer-wins with vector clocks / version vectors is acceptable; the vault is a personal dataset with rare concurrent edits. No cross-item transactions needed. Sync is eventually consistent, converging within seconds.
  • Durability: this is paramount and asymmetric. Losing a user’s vault is catastrophic and unrecoverable (the server cannot regenerate secrets it cannot read). Target 11 nines of durability on the encrypted blobs, multi-region replicated, with point-in-time backups - of ciphertext.
  • Security posture: the threat model assumes the server is compromised. Confidentiality must hold against a full database breach and a malicious insider. Integrity (authenticated encryption) must detect any server-side tampering of blobs.

Back-of-the-Envelope Estimation (BoE)

Real numbers with the arithmetic, because they set the shape (and the shape is: this is a small, read-light system, and that is itself the insight).

Users and items:

100M registered, 40M DAU.
Avg 150 items/vault, avg item plaintext ~1KB (URL, user, pass, notes, TOTP).
Encrypted item blob ~1.2KB (ciphertext + IV + auth tag + metadata).
Total items ≈ 100M * 150 = 15B items.

Storage (the whole vault corpus):

15B items * ~1.2KB ≈ 18 TB of ciphertext.
Add version history (keep last ~10 versions of changed items for undo/recovery):
call it ~3x -> ~55 TB.
Plus per-user key material, sharing keys, metadata: negligible by comparison.

Eighteen to fifty-five terabytes. That is tiny. This entire product’s secret data fits on a handful of disks. Storage is not the problem; integrity, confidentiality, and durability of that small dataset are the problem.

Write QPS (edits/adds):

An active user edits/adds a few items per day. Say 5 write-ops/DAU/day.
40M * 5 = 200M writes/day = 200,000,000 / 86,400 ≈ 2,300 writes/sec average.
Peak (3x) ≈ 7,000 writes/sec.

Read/sync QPS:

Reads are dominated by sync polls / delta pulls, not full reads. A device does not re-download the vault each time; it pulls “what changed since my last cursor.”

Devices per user ~3. 40M DAU * 3 devices = 120M active devices.
Each device syncs on wake/foreground, say ~every few minutes when active:
assume ~20 sync checks/device/active-hour, ~4 active hours -> ~80 checks/device/day.
120M * 80 = 9.6B sync checks/day ≈ 111,000 checks/sec average, ~330K peak.

But the vast majority of those sync checks return “nothing changed.” A conditional GET with an If-None-Match/cursor returns a tiny 304-style empty delta. Real payload-bearing pulls are far fewer. So even the “high” number is cheap bytes.

Bandwidth:

Actual changed-data pulls: writes propagate to ~2 other devices.
2,300 writes/sec * 2 fan-out * ~1.2KB ≈ 5.5 MB/sec of ciphertext. Trivial.
Full initial vault download (new device): 150 items * 1.2KB ≈ 180KB, one-time.

Breach-check load:

Breach checks are batched, not per-keystroke. Say each active vault is
re-checked daily against the breach corpus: 40M vaults * 150 items = 6B
password-hash-prefix lookups/day ≈ 70K lookups/sec, but these hit a
separate, cacheable k-anonymity range API, not the vault store.

The takeaways: the dataset is tiny (tens of TB), write and read QPS are modest (thousands to low hundreds of thousands), and the system works offline so the server is off the critical read path. This is emphatically not a throughput monster like a chat or feed system. The entire difficulty is cryptographic architecture: making a system that is useful while its central server is blind. Every engineering dollar goes into confidentiality, key management, sync correctness, and durability - not into QPS.

High-Level Design (HLD)

The architecture is a fat client, thin untrusted server. All cryptography lives on the client. The server is a replicated, versioned key-value store of encrypted blobs plus an auth/sync coordinator that never sees plaintext or the keys that would decrypt it.

The heart of it is the key hierarchy. The master password never leaves the device and is never sent to the server. From it the client derives two independent things: an authentication proof (to log in) and an encryption key (to unlock the vault). These must be cryptographically separated so that the thing you send to the server (auth) can never be used to derive the thing that decrypts data (the key).

        ┌──────────────────────── CLIENT (trusted) ────────────────────────┐
        │                                                                    │
        │   Master Password ──▶ Argon2id(salt) ──▶ Master Key (never sent)   │
        │                              │                                     │
        │                 ┌────────────┴─────────────┐                       │
        │                 ▼                          ▼                        │
        │        Auth Hash (HKDF #1)        Vault-Unlock Key (HKDF #2)        │
        │        sent to server             stays on device                  │
        │             │                          │                           │
        │             │              decrypts ──▶ Vault Encryption Key (VEK)  │
        │             │                          │  (random, wrapped by unlock key) │
        │             │                          ▼                           │
        │             │              AES-256-GCM encrypt/decrypt each item    │
        │        Encrypt/Decrypt, form parsing, auto-fill, password gen      │
        └─────────────┼──────────────────────────────────────────────────────┘
                      │ HTTPS/TLS (auth hash + ciphertext blobs only)
        ┌─────────────▼──────────────────────── SERVER (untrusted) ──────────┐
        │  ┌───────────────┐   ┌──────────────────┐   ┌───────────────────┐   │
        │  │ Auth Service   │   │  Sync Service     │   │ Sharing Service    │  │
        │  │ (verify auth   │   │ (versioned delta  │   │ (relay wrapped     │  │
        │  │  hash, 2FA,    │   │  push/pull,       │   │  item keys between │  │
        │  │  sessions)     │   │  vector clocks)   │   │  users' pubkeys)   │  │
        │  └───────┬───────┘   └────────┬─────────┘   └─────────┬─────────┘   │
        │          │                    │                       │             │
        │  ┌───────▼────────────────────▼───────────────────────▼─────────┐   │
        │  │   Encrypted Vault Store  (blobs, versioned, sharded by user)  │   │
        │  │   + user public keys + wrapped-key envelopes  (all opaque)    │   │
        │  └───────────────────────────────────────────────────────────────┘   │
        │  ┌───────────────────────────────────────────────────────────────┐   │
        │  │   Breach Service  (k-anonymity range API over HIBP-style      │   │
        │  │   hashed-prefix index; never sees full hashes)                 │   │
        │  └───────────────────────────────────────────────────────────────┘   │
        └───────────────────────────────────────────────────────────────────┘

Unlock flow (local, no server needed once cached):

  1. User types the master password. The client fetches the user’s KDF salt and parameters (cached locally, originally from the server at signup).
  2. Client runs Argon2id(master_password, salt) -> Master Key. This is deliberately slow (hundreds of ms) to resist brute force.
  3. From the Master Key, HKDF derives two separate outputs: an Auth Hash and a Vault-Unlock Key. The Master Key itself is then discarded from memory.
  4. The Vault-Unlock Key decrypts the locally-cached wrapped Vault Encryption Key (VEK). Now the client can decrypt every item. The vault is open, entirely offline.

Login / new-device flow (server involved):

  1. Steps 1-3 as above, producing the Auth Hash.
  2. Client sends {email, auth_hash} over TLS. The server has stored Argon2(auth_hash + server_salt) (it slow-hashes the auth hash again server-side, so even the auth hash is not stored in a directly-usable form). It compares; on match, plus 2FA, it issues a session token.
  3. Client pulls the encrypted vault blobs and the wrapped VEK. It decrypts the VEK locally with the Vault-Unlock Key and opens the vault. The server sent ciphertext and never had the key.

Sync flow (edit propagation):

  1. User edits item X on the laptop. Client re-encrypts X into a new ciphertext blob, bumps X’s version vector, and PUTs the blob + version to the Sync Service.
  2. Sync Service stores the new version, appends to the user’s change log, and bumps the vault’s sync cursor.
  3. The phone, on its next sync poll, sends its cursor; the Sync Service returns the delta (X’s new blob). The phone applies it and decrypts locally.

The key insight, repeated because it is everything: the server moves and versions opaque bytes. All meaning lives behind a key it never possesses. Sync, sharing, and recovery are all engineered around that blindness.

Component Deep Dive

The hard parts, naive-first then evolved: (1) the zero-knowledge key hierarchy, (2) multi-device sync of ciphertext with conflict resolution, (3) breach notifications without seeing passwords, (4) end-to-end-encrypted sharing.

1. The zero-knowledge key hierarchy

Naive approach: hash the master password, store it, encrypt the vault with it. Server stores SHA-256(master_password) for login; the vault is encrypted with the master password directly.

Where it breaks:

  • The login secret and the encryption secret are the same thing. If the server verifies login by comparing a hash of the master password, then the server received something derived from the master password. Worse, if you encrypt the vault with the master password directly, then any employee who can see what the client sends at login is one step from the vault key. Auth material and encryption material must be cryptographically independent outputs, so that possessing one yields nothing about the other.
  • Fast hashes are brute-forceable. SHA-256 computes billions/sec on a GPU. Master passwords are human-chosen and low-entropy. A stolen table of SHA-256(master_password) is cracked wholesale. The KDF must be deliberately slow and memory-hard.
  • Re-encrypting the whole vault on a password change is O(vault). If the vault is encrypted directly under a key derived from the master password, changing the master password means decrypting and re-encrypting every item. For a 5,000-item vault that is a lot of crypto, and it happens on every rotation.

First evolution: a slow, memory-hard KDF (Argon2id) with a per-user salt.

master_key = Argon2id(
    password   = master_password,
    salt       = per_user_random_salt,      # stored server-side, not secret
    time_cost  = t, memory_cost = 64MB, parallelism = p
)

Argon2id is memory-hard: a GPU/ASIC attacker cannot parallelize cheaply because each guess needs 64MB of RAM. This turns “billions of guesses/sec” into “thousands,” making low-entropy master passwords survivable. (PBKDF2 with high iteration counts is the older, weaker-but-acceptable fallback; scrypt is another memory-hard option.) But we still have the “auth == encryption” coupling and the re-encryption cost.

The answer: split derivation via HKDF, and a random Vault Encryption Key wrapped by the derived key.

Two moves. First, from the single Master Key, derive two independent keys with distinct HKDF “info” labels:

master_key       = Argon2id(master_password, salt, params)
auth_hash        = HKDF(master_key, info="auth")        # goes to server
vault_unlock_key = HKDF(master_key, info="unlock")      # never leaves device

Because HKDF outputs are independent, the auth hash reveals nothing about the unlock key. The server can hold the auth hash (and slow-hash it again server-side before storing, so a server breach does not even hand over the auth hash in usable form) and still be structurally unable to derive the unlock key.

Second, do not encrypt items directly with the unlock key. Instead generate a random 256-bit Vault Encryption Key (VEK) once, encrypt all items with the VEK, and store the VEK wrapped (encrypted) by the unlock key:

VEK              = random_256_bit()                     # the real data key
wrapped_VEK      = AES-256-GCM_encrypt(VEK, key=vault_unlock_key)
item_ciphertext  = AES-256-GCM_encrypt(item_plaintext, key=VEK)  # per item, unique IV

Now a master password change is O(1), not O(vault): re-derive a new unlock key, re-wrap the same VEK, upload the new wrapped_VEK. Not a single item is touched. The VEK is the stable root of data encryption; the password only ever guards the wrapper.

AES-256-GCM is authenticated encryption: it produces an auth tag, so if the untrusted server flips a single bit of a blob, decryption fails loudly rather than yielding garbage. That is the integrity half of the threat model - the server cannot tamper undetected.

PropertyNaive (hash pw, encrypt with pw)KDF + HKDF split + wrapped VEK
Server can derive vault keyEffectively yesNo - independent HKDF outputs
Brute-force resistanceWeak (fast hash)Strong (Argon2id, memory-hard)
Master password changeRe-encrypt whole vaultRe-wrap one key, O(1)
Server-side tamperingUndetectedDetected (GCM auth tag)
Breach of DB revealsCracked passwords + dataOnly opaque ciphertext

Where does the salt/params come from on a fresh device? The KDF salt and parameters are not secret; the server serves them by email at login time so a new device can derive the same Master Key. The secret is only ever the master password in the user’s head.

2. Multi-device sync of ciphertext with conflict resolution

Naive approach: full vault upload/download, last-write-wins on the whole blob. Each device uploads its entire encrypted vault; newest upload wins.

Where it breaks:

  • Whole-vault last-write-wins loses edits. Laptop edits item A, phone edits item B, both offline. Phone syncs last -> its whole-vault upload overwrites the laptop’s version, and the laptop’s edit to A is gone. Silent data loss in a product whose entire value is not losing your data.
  • Bandwidth and churn. Re-uploading a 5,000-item vault because one item changed is absurd. Sync must be per-item deltas.
  • The server cannot merge. In a normal system you would merge on the server. Here the server sees ciphertext - it cannot look inside two conflicting versions of an item to reconcile fields. Conflict handling has to be structured so the server can order versions without reading them, and the client resolves any true content conflict.

The fix: per-item versioning with version vectors, a server-side monotonic change log, and cursor-based delta sync.

Model the vault as a set of independently-versioned items. Each item carries a version vector (a map of device_id -> counter) so devices can detect concurrent vs. causal edits without the server reading content.

Item on the wire:
  item_id        UUID
  vault_id       UUID
  ciphertext     BLOB            # AES-256-GCM(item, VEK); server cannot read
  version_vector { deviceA: 4, deviceB: 2 }
  updated_at     client timestamp (tiebreak only)
  deleted        tombstone flag

Server side: a monotonic per-vault change log + cursor. Every accepted PUT appends to the vault’s change log and increments a server sequence (the sync cursor). The server orders versions, not content.

sync pull:   client sends its last cursor C
             server returns all item blobs with seq > C, plus new cursor C'
sync push:   client PUTs item blob + its version vector
             server appends to log, assigns seq, returns new cursor

This makes sync incremental: a device pulls only what changed since its cursor, exactly like a per-device tail of a log. A week-offline phone resumes from its cursor and pulls the delta - bounded, not a full re-download.

Conflict resolution, done on the client because only it can read plaintext:

On pull, for each incoming item version vs local version:
  - incoming VV strictly dominates local  -> fast-forward (accept incoming)
  - local VV strictly dominates incoming  -> keep local, re-push
  - VVs are concurrent (neither dominates) -> TRUE CONFLICT
        default: last-write-wins by updated_at (deterministic, converges)
        safer:   keep both as "item (conflicted copy)" and surface to user

Because passwords are sensitive, a good manager leans toward keeping both copies on a true concurrent conflict rather than silently dropping one - the user picks, and nothing is lost. Version vectors are what let the client distinguish a real concurrent edit from a stale device catching up, which naive timestamps cannot do (clock skew would misfire).

Field-level vs item-level granularity. You could go finer - per-field version vectors so “phone changed the note, laptop changed the password” auto-merges. That reduces false conflicts but multiplies metadata and complicates the ciphertext layout (each field its own blob). Item-level is the pragmatic default; items are small and true concurrent edits to the same item are rare. State the trade-off and pick item-level unless the interviewer pushes.

Server never sees plaintext, yet sync works because ordering and causality live in metadata (version vectors, cursors, tombstones) that is safe to expose. The content stays sealed.

3. Breach notifications without seeing passwords

Naive approach: server scans the vault against the breach corpus. The server decrypts each stored password and checks it against a list of leaked passwords (like Have I Been Pwned’s dataset).

Where it breaks immediately:

  • The server cannot decrypt anything. The entire architecture forbids it. This approach is dead on arrival - there is no plaintext server-side to scan.
  • Even sending the password’s hash to a server leaks it. If the client hashes each password with SHA-1/SHA-256 and asks “is this hash in your breach list?”, the server (or a network eavesdropper) learns exactly which breached password you use the moment there is a hit, and can offline-crack the hash otherwise. You cannot send the full hash.

The fix: client-side hashing plus a k-anonymity range query. This is the Have-I-Been-Pwned model and it is elegant.

Client:
  h = SHA1(password)                    # 40 hex chars, done ON DEVICE
  prefix = h[0:5]                       # first 5 hex chars
  suffix = h[5:]                        # remaining 35

  GET /range/{prefix}                   # send ONLY the 5-char prefix
Server (Breach Service):
  returns ALL breached suffixes sharing that prefix, with counts
  e.g. ~400-800 candidate suffixes for any given prefix

Client:
  scan the returned list locally for `suffix`
  found  -> this password appears in N breaches, warn the user
  absent -> safe

The server sees only a 5-character hash prefix, which is shared by hundreds of thousands of possible passwords (~1M+ hashes per prefix across the corpus). It cannot tell which password you actually checked, and it never learns whether you had a hit - the match is computed entirely on the client from the returned list. This is k-anonymity: your query hides in a crowd of size k.

Making the corpus queryable at scale. The breach dataset is billions of leaked-credential hashes (hundreds of GB). Index it by 5-char prefix (a fixed 16^5 ≈ 1M buckets), each bucket holding its suffix list. This is:

Table: breach_hashes
  prefix   CHAR(5)   PARTITION KEY     # ~1M partitions, evenly sized
  suffix   CHAR(35)  CLUSTERING KEY
  count    INT                          # times seen across breaches
  Query:   "all suffixes where prefix = ABCDE"  = single-partition scan

Because a prefix maps to a bounded, static-ish bucket, every range response is cacheable at the CDN edge. The corpus updates only when new breaches are ingested (a slow offline batch job), so cache TTLs of hours are fine. That turns 70K lookups/sec into mostly CDN hits, offloading the store entirely.

Reused-password and weak-password checks are even easier and fully local: the client already has all decrypted passwords in memory when the vault is open, so “you used this password on 6 sites” and “this password is weak” are computed on-device with zero server involvement. Only the breach corpus needs the k-anonymity dance, because that data lives server-side.

4. End-to-end-encrypted sharing

Naive approach: server re-encrypts the item for the recipient. Alice shares an item; the server decrypts it and re-encrypts under Bob’s key.

Where it breaks: the server would need to decrypt - forbidden. And you cannot encrypt an item under “Bob’s master-password-derived key” because Alice does not have it (and must not). Sharing has to happen without the server ever holding plaintext and without either party knowing the other’s master password.

The fix: asymmetric key wrapping (public-key crypto) layered on the symmetric item keys. Give every user a keypair; share by wrapping the item’s key under the recipient’s public key.

Setup: at signup, the client generates an X25519 (or RSA) keypair. The public key is uploaded in the clear (it is public). The private key is encrypted under the user’s VEK and stored as a blob - so it too is zero-knowledge; only the user can unwrap their own private key after unlocking their vault.

Alice shares item I with Bob:
  1. Alice's client already has item_key(I) (or generates one per shared item).
  2. Alice fetches Bob's PUBLIC key from the server (public, safe to serve).
  3. Alice computes wrapped = seal(item_key(I), Bob_public_key)   # X25519 + AEAD
  4. Alice uploads: { item_id, recipient: Bob, wrapped_item_key }
     and the item ciphertext (encrypted under item_key(I)).
  5. Server stores the wrapped key envelope. It CANNOT open it - it lacks
     Bob's private key.

Bob receives:
  6. Bob's client unwraps: item_key(I) = open(wrapped, Bob_private_key)
     (Bob_private_key itself was just unwrapped by Bob's VEK on unlock)
  7. Bob decrypts the item ciphertext with item_key(I). Done - E2E preserved.

The server is a relay for sealed envelopes. It routes wrapped_item_key blobs it cannot open, exactly as it routes item ciphertext it cannot read. For a shared vault (team/family), use a per-vault symmetric key and wrap that once per member, so adding a member is one envelope, not one-per-item.

Revocation is the honest hard part. Once Bob has decrypted an item, you cannot un-ring that bell - he could have copied the plaintext. Revoking removes his envelope and, for real security, rotates the item/vault key and re-wraps for the remaining members so future edits are inaccessible to Bob. Past exposure is unrecoverable; state this trade-off plainly rather than pretending revocation is perfect.

API Design & Data Schema

The API is deliberately dumb: it moves opaque blobs and version metadata. There is no “search items” or “get password” endpoint, because the server could not fulfill them.

REST API

# Auth / key bootstrap
POST /api/v1/auth/prelogin        { email }
  -> { kdf: "argon2id", salt, mem, iter, parallelism }   # non-secret KDF params
POST /api/v1/auth/login           { email, auth_hash, twofa_code }
  -> { session_token, wrapped_VEK, user_private_key_blob }
POST /api/v1/auth/password-change { new_auth_hash, new_wrapped_VEK }
  -> 200   # note: only the wrapper changes; items untouched

# Sync (the core path)
GET  /api/v1/sync?cursor=<seq>
  -> { changes: [ { item_id, ciphertext, version_vector, deleted, seq } ... ],
       cursor: <new_seq> }                                 # delta since cursor
PUT  /api/v1/items/{item_id}
     body: { ciphertext, version_vector, updated_at }
  -> { seq, cursor }                                       # append to change log
POST /api/v1/items/bulk           { items: [...] }         # batched writes
DELETE /api/v1/items/{item_id}    -> writes a tombstone, not a hard delete

# Sharing
GET  /api/v1/users/{email}/pubkey -> { public_key }        # public, safe
POST /api/v1/shares               { item_id, recipient, wrapped_item_key, ciphertext }
DELETE /api/v1/shares/{share_id}  -> revoke (client then rotates key)

# Breach check (k-anonymity)
GET  /api/v1/breach/range/{hash_prefix}
  -> { suffixes: [ { suffix, count } ... ] }               # CDN-cached

Note there is no GET /items/{id}/plaintext, no search?q=. The absence of those endpoints is the design.

Data store choices

1. Encrypted Vault Store - NoSQL wide-column / document (DynamoDB, Cassandra, or partitioned Postgres with blobs). The access pattern is “get all changed items for a user since cursor” and “put an item blob,” partitioned by user, no cross-user joins, no server-side content queries. That is a key-value/wide-column workload. The dataset (~55TB) is small enough that even partitioned Postgres works, but a wide-column store gives clean horizontal sharding by user_id.

Table: items
  user_id        UUID     PARTITION KEY      # a user's vault is one partition
  item_id        UUID     CLUSTERING KEY
  ciphertext     BLOB                         # AES-256-GCM(item, VEK) - opaque
  version_vector MAP<STR,INT>
  seq            BIGINT                        # per-vault monotonic sync counter
  deleted        BOOLEAN                       # tombstone
  updated_at     TIMESTAMP                     # client clock, tiebreak only
  Index: (user_id, seq)  -> delta pull "changes where seq > cursor"
  Shard by: user_id  -> a vault lives together; sync is single-partition

Table: users
  user_id        UUID     PARTITION KEY
  email          STRING   (unique, secondary index)
  kdf_salt       BLOB     kdf_params   JSON     # non-secret, for re-derivation
  server_auth_hash BLOB   # = slow_hash(auth_hash + server_salt); NOT the vault key
  wrapped_VEK    BLOB     # VEK encrypted by vault_unlock_key - server can't open
  public_key     BLOB     # user's X25519 public key (public)
  private_key_blob BLOB   # private key encrypted under VEK - zero-knowledge
  twofa_config   BLOB

Table: change_log            # drives cursor-based delta sync
  user_id   UUID   PARTITION KEY
  seq       BIGINT CLUSTERING KEY ASC
  item_id   UUID
  op        ENUM(put, delete)

Table: shares               # sealed envelopes the server relays but cannot open
  share_id      UUID  PARTITION KEY
  item_id       UUID
  owner_id      UUID
  recipient_id  UUID
  wrapped_item_key BLOB      # item key sealed to recipient's public key

2. Breach corpus - read-optimized, prefix-partitioned store behind a CDN. Static-ish, huge, read-only, perfectly cacheable. Sharded by 5-char hash prefix (see deep dive), fronted by a CDN so most reads never touch origin.

3. Sessions / rate-limit counters - Redis. Short-lived session tokens, per-account login-attempt counters (to throttle master-password guessing at the API), and sync cursors cache. In-memory, ephemeral, rebuildable.

Why NoSQL/blob-store over a rich SQL model: there are no joins to run (the server cannot correlate encrypted content), no multi-item transactions (a vault is a personal dataset with per-item LWW), and the query surface is intentionally just “get/put blob by key” and “delta since cursor.” A relational engine’s ACID transactions and joins are features we deliberately cannot use on ciphertext. The one place SQL-grade guarantees matter - never losing a write - is handled by replication and the append-only change log, not by transactional joins. Durability, not query power, is the requirement, and a replicated KV store delivers it.

Bottlenecks & Scaling

Where it breaks first, in order, and the fix for each. Note that because QPS is modest and the client works offline, the “bottlenecks” here are mostly about security and correctness at scale, not throughput.

1. Master-password brute force (the real first-order threat). A stolen users table invites offline cracking of every master password. Fix: Argon2id, memory-hard, tuned to ~64MB and hundreds of ms per guess, with a per-user salt so rainbow tables are useless and each account must be attacked independently. Server-side, slow-hash the auth hash again before storing, so a DB breach does not even yield a directly-replayable auth hash. Rate-limit and lock login attempts per account in Redis to throttle online guessing. This is where the security budget goes.

2. Durability of an unrecoverable dataset. If a blob is lost, it is gone forever - the server cannot regenerate secrets it cannot read. Fix: replication factor 3+ across availability zones, multi-region async replication, point-in-time backups of ciphertext, and soft-delete via tombstones plus version history so an accidental client delete/overwrite is recoverable (undo). Keep the last N versions of each item. Durability here is a first-class NFR, not an afterthought.

3. Sync fan-out and thundering herds. Millions of devices polling. Fix: conditional/delta sync - devices send a cursor and usually get an empty 304-style response; only real changes carry payload. Add push wake-ups (a lightweight WebSocket or platform push notification “your vault changed, pull now”) so idle devices are not polling on a timer. Jittered backoff on reconnect avoids herds. Because payloads are tiny and mostly empty, this scales cheaply.

4. Breach-corpus read load. Billions of hashes, many lookups. Fix: partition by 5-char prefix into ~1M even buckets and cache aggressively at the CDN edge. The corpus is near-static between breach ingests, so hour-long TTLs make almost every query a cache hit. The origin store handles only cache-miss and ingest traffic.

5. Sharding the vault store. As users grow to 100M, one partition space must spread evenly. Fix: shard by user_id. A vault’s items and change log co-locate, so sync (the hot path) is a single-partition delta scan. user_id distributes evenly (no natural hot user), so there is no hot-shard problem the way a time-based key would create. Sharding by updated_at would be the classic mistake - it piles all current writes onto the “now” shard.

6. Hot vault / power user. A user with 10,000 items and many devices. Fix: even a huge vault is only ~12MB of ciphertext and lives in one partition read as a delta; it is not hot in QPS terms, just larger on initial sync. Paginate initial download by cursor. This is not a real bottleneck given the numbers, but say why: the dataset is small, so “hot key” pressure that dominates chat/feed systems barely exists here.

7. Account recovery without breaking zero-knowledge (the hardest trade-off). If the master password is truly the only key and the user forgets it, the vault is mathematically unrecoverable - which is secure but a support nightmare. Fix (pick per product): a recovery/secret key (1Password’s model) - a high-entropy key generated at signup, combined with the master password to derive the unlock key, that the user must save offline; the server never sees it, and it makes brute force infeasible even with a weak master password. Or enterprise key escrow / account recovery, where an admin holds a recovery key that can re-wrap the VEK - this weakens pure zero-knowledge (the org can recover the vault) and must be an explicit, opt-in policy. Or emergency-access / trusted-contact schemes using the same public-key wrapping as sharing, with a time delay. There is no free lunch: perfect zero-knowledge and easy recovery are in direct tension, and you choose where to sit.

8. Malicious server tampering. A compromised server could serve a modified blob or a modified client. Fix: AES-256-GCM auth tags make blob tampering fail closed on the client. The scarier vector is a tampered client (malicious JS pushed to the browser extension) - mitigate with code signing, subresource integrity, reproducible builds, and preferring native apps over web for the crypto boundary. This is the residual trust: the user must trust the client code, even though they need not trust the server.

9. Single points of failure. Fix: Auth, Sync, and Sharing services are stateless behind a load balancer; the store is replicated; the breach service is CDN-fronted and its origin is read-only. No single box whose loss stops the product - and even a total backend outage leaves every client fully functional offline, syncing again when the backend returns. That offline-first property is the ultimate availability backstop.

10. Multi-region latency. Fix: since the server is off the critical read path (local unlock), latency matters only for sync and login. Deploy sync/auth regionally with GeoDNS, replicate blobs async cross-region (per-user LWW tolerates async replication fine - a user rarely edits from two regions simultaneously). A user’s home region owns their vault and serves their sync.

Wrap-Up

The trade-offs that define this design:

  • Zero-knowledge over convenience. We make the server structurally blind - it stores only ciphertext and can never derive the vault key - which buys the one property that matters (a full server breach leaks nothing usable) at the cost of every clever server-side feature (search, merge, server-side breach scan, easy recovery) being off the table. Every other decision bends around that blindness.
  • KDF + HKDF split + wrapped random VEK. We derive independent auth and encryption keys from a memory-hard Argon2id so the login secret can never yield the data secret, and we encrypt items under a random VEK that the password only wraps - so brute force is slow, a password change is O(1), and GCM auth tags catch any tampering.
  • Sync ciphertext with metadata-only coordination. The server orders versions with cursors and version vectors it can read, while true content conflicts are resolved on the client that alone can decrypt - and we keep both copies on a real conflict rather than silently dropping a password.
  • k-anonymity for breach checks. We check passwords against a breach corpus by sending only a 5-char hash prefix and matching locally, so the server learns neither the password nor the result - the only way to do breach notifications when you refuse to hold the passwords.
  • Public-key wrapping for sharing, honest about revocation. Sharing seals item keys to recipients’ public keys and the server relays envelopes it cannot open; revocation rotates keys for the future but cannot recall what was already decrypted, and we say so.
  • Offline-first relaxes everything. Because the vault is cached and decrypted locally, the untrusted server is off the critical read path - a backend outage degrades sync, not access - so this is a small, read-light, correctness-and-crypto-dominated system, not a throughput monster.

One-line summary: a fat-client, zero-knowledge password manager where a memory-hard KDF derives independent auth and unlock keys, items are encrypted client-side under a random vault key that the master password only wraps, the untrusted server versions and syncs opaque ciphertext via cursors and version vectors with client-side conflict resolution, sharing seals per-item keys to recipients’ public keys, breach checks use k-anonymity hash-prefix range queries so the server never sees a password, and the whole thing works offline because the server is a blind blob store, not the source of truth.