Pastebin looks like a URL shortener with a bigger payload, and people design it that way: mint a key, stuff the text somewhere, return a link. Then the interviewer pushes. Where does the actual paste body live - in the database? What happens to your row size and your backups when someone pastes a 5MB stack trace? How do you serve a billion reads a day when each response is not a 500-byte redirect but a 10KB blob? How do 100 million pastes expire without a cron job melting the database? Now it is a real system.

The single decision that shapes everything: the payload is large and the system is brutally read-heavy. That combination means you do not store the content where you store the metadata, and you do not serve reads the way you serve a redirect. Get the storage split right and the rest is caching and cleanup you already know.

Let me do it properly.

Functional Requirements (FR)

In scope:

  • Create a paste. User submits text or code, optionally with a title, syntax/language hint, expiry, and privacy setting. Returns a short URL like https://pst.bin/aB3xK9.
  • Read a paste. Anyone with the URL (subject to privacy) fetches the content by short key.
  • Expiry. A paste can expire after a chosen TTL (10 minutes, 1 day, 1 month, never) or after being read once (“burn after reading”).
  • Privacy settings. public (listed, indexable), unlisted (only reachable via the exact key), private (only the owner, requires auth).
  • Raw view. Fetch the plain content without the HTML wrapper (/raw/aB3xK9), which is what scripts and curl use.

Explicitly out of scope (say this out loud so you control scope):

  • Editing an existing paste. Pastes are immutable once created. This kills a whole class of cache-invalidation problems; an edit is just a new paste.
  • Full user accounts, social features, comments, forking. We assume optional lightweight auth for private pastes only.
  • Server-side syntax highlighting as a hard requirement. Highlighting is a client-side concern; we store raw text.
  • Full-text search across public pastes. Mention it exists (offline index), do not design it on the hot path.
  • Abuse/malware/secret scanning. Real systems need it; note it runs as an async pipeline off the write path, do not design it here.

The one decision that drives the design: large payload, read-heavy. Reads dominate writes ~100:1, and each read ships kilobytes, not bytes. Every choice optimizes the read path and keeps the big blob out of the hot metadata store.

Non-Functional Requirements (NFR)

  • Scale: 10M pastes created per day, 1B reads per day. That is a 100:1 read:write ratio.
  • Latency: read p99 under 100ms end to end (blob fetch dominates, not a lookup). Metadata lookup p99 under 10ms. Create p99 under 300ms including the blob write.
  • Availability: 99.99% on the read path. A dead paste link is the most visible failure - it is embedded in bug reports, chat messages, and forum posts. Create can tolerate slightly lower availability.
  • Consistency: a paste is immutable, so we need read-your-writes on create (the author must see it immediately) and can serve everyone else from eventually-consistent replicas, caches, and CDN. A key never points to two different bodies over time.
  • Durability: a paste must not silently vanish before its expiry. If someone pasted their production config and linked it in a ticket, losing it early is a real incident. Durable, replicated storage for both metadata and blob.
  • No collisions: two different pastes must never receive the same key.

Back-of-the-Envelope Estimation (BoE)

Real numbers first so the architecture is grounded.

Writes (create):

10M pastes / day
= 10,000,000 / 86,400 seconds
≈ 116 writes/sec  (call it ~120 WPS average)
Peak (3x average) ≈ 350 WPS

Reads:

1B reads / day
= 1,000,000,000 / 86,400 seconds
≈ 11,574 reads/sec  (call it ~12K RPS average)
Peak (3x) ≈ 35K RPS

35K RPS is very manageable for the metadata lookup. The pressure is not request rate - it is payload size, which shows up in bandwidth and storage.

Payload size assumption:

Most pastes are small (a config snippet, an error, a code block).
A few are large (a whole log file). Cap a single paste at 10MB.
Assume average paste body ≈ 10KB.

Storage (content):

10M/day * 10KB = 100 GB/day of content
Per year: 100 GB * 365 ≈ 36.5 TB/year
Over 5 years ≈ 180 TB (before expiry reclaim)

Expiry claws a lot of this back - most pastes are ephemeral - but design for the worst case where many choose “never.”

Storage (metadata):

Per metadata row ≈ 250 bytes (key, blob pointer, flags, timestamps, owner)
10M/day * 250 bytes = 2.5 GB/day
Per year ≈ 900 GB/year of metadata

Metadata is tiny and lives in a fast store. Content is bulky and lives in object storage. That split is the whole game.

Bandwidth:

Read bandwidth at peak: 35K RPS * 10KB ≈ 350 MB/sec
Write bandwidth at peak: 350 WPS * 10KB ≈ 3.5 MB/sec

Read bandwidth is real (350 MB/sec is ~2.8 Gbps). Unlike a URL shortener, this is a bandwidth problem, which is exactly why a CDN in front of an object store is not optional - you do not want your application servers moving 2.8 Gbps of blob.

Cache size:

The classic 80/20: a small fraction of pastes drive most reads (the viral snippet, the popular gist). Cache the hot content bodies.

Suppose ~5M distinct pastes are "hot" on a given day.
5M * 10KB (body) ≈ 50 GB of hot content
Plus metadata for the hot set: 5M * 250 bytes ≈ 1.25 GB

Round to ~50-75GB of cache. Most hot content is served by the CDN, so origin cache is a second line. Metadata cache hit-rate target: >95%.

Key space sanity check (base62, length 7):

62^7 = 3.5 trillion keys
At 10M/day we consume 3.65B/year.
3.5T / 3.65B ≈ 950 years of runway at length 7.

Length 7 is comfortable. Length 6 (62^6 ≈ 56B) gives ~15 years, tighter but fine; length 7 is the safe call.

High-Level Design (HLD)

Two paths with different SLAs and, critically, two different data planes: metadata (small, fast, sharded KV) and content blob (large, cheap, object storage plus CDN). Separating them is the core architectural move.

                          ┌──────────────────────────┐
                          │         Clients          │
                          │  (browsers, curl, API)   │
                          └────────────┬─────────────┘
                                       │
                          ┌────────────▼─────────────┐
                          │        CDN / Edge        │  caches raw blobs
                          │  (public/unlisted reads) │  (immutable content)
                          └────────────┬─────────────┘
                                       │ miss / private
                          ┌────────────▼─────────────┐
                          │       Load Balancer      │
                          └──────┬────────────┬──────┘
                                 │            │
                  WRITE path     │            │   READ path
             ┌───────────────────▼──┐   ┌─────▼──────────────────┐
             │    Create Service    │   │      Read Service      │
             │     (stateless)      │   │      (stateless)       │
             └──┬─────────┬─────────┘   └───┬──────────────┬─────┘
                │         │                 │              │
        ┌───────▼──┐  ┌───▼──────────┐  ┌───▼────────┐  ┌──▼──────────────┐
        │ Key Gen  │  │ Blob Write   │  │  Redis     │  │ Blob Read       │
        │ Service  │  │ to Object    │  │ (metadata  │  │ from Object     │
        │(counter) │  │ Store (S3)   │  │  + hot body│  │ Store (S3)      │
        └──────────┘  └───┬──────────┘  │  cache)    │  └──┬──────────────┘
                          │             └───┬────────┘     │
              ┌───────────▼──────────┐      │ miss         │
              │  Object Store (S3)   │◄─────┼──────────────┘
              │  content blobs       │      │
              │  key = paste_id      │      │
              └──────────────────────┘      │
                                            │
              ┌──────────────────────┐      │
              │  Metadata Store (KV) │◄─────┘  paste_id -> {blob_ptr, flags}
              │  sharded by paste_id │
              │  native TTL on expiry│
              └──────────┬───────────┘
                         │ async events (create, read, expire)
                  ┌──────▼────────┐
                  │ Message Queue │ (Kafka)
                  └──────┬────────┘
                         │
             ┌───────────┼───────────────┐
       ┌─────▼─────┐ ┌───▼──────┐ ┌──────▼──────┐
       │ Analytics │ │ Abuse/   │ │ Search      │
       │ (offline) │ │ malware  │ │ index (pub) │
       └───────────┘ └──────────┘ └─────────────┘

Write flow (create):

  1. Client POST /api/v1/pastes with the body, expiry, privacy.
  2. Create Service validates (size cap, non-empty, allowed language hint).
  3. Create Service requests a unique ID from the Key Generation Service, base62-encodes it into a 7-char paste_id.
  4. Content blob is written to the object store at a deterministic path derived from paste_id.
  5. Metadata row {paste_id, blob_ptr, size, language, privacy, expires_at, owner_id} is written to the sharded metadata store, replicated and durable.
  6. The short URL is returned. Optionally warm the metadata cache. Emit a paste_created event to Kafka for abuse scanning and the public search index.

Read flow (the hot path):

  1. Client requests GET /aB3xK9 (or /raw/aB3xK9).
  2. For public/unlisted pastes, the CDN may already hold the immutable raw blob. Hit -> serve from edge, origin never touched.
  3. On miss (or for the HTML-wrapped view, or a private paste), Load Balancer routes to a Read Service instance.
  4. Read Service looks up metadata in Redis (GET paste:aB3xK9). Hit (>95%) gives it {blob_ptr, privacy, expires_at, ...} without touching the metadata store.
  5. Check expiry (if expires_at < now, return 404/410 and enqueue cleanup). Check privacy (if private, require and verify auth).
  6. Fetch the content: for hot bodies, from the Redis body cache; otherwise from the object store via blob_ptr. Populate caches read-through.
  7. Return the paste. Emit a paste_read event to Kafka asynchronously (never block the read on analytics). For burn-after-reading, mark deleted after the first successful read.

The key insight: the blob never travels through the metadata store, and analytics never sits on the read path.

Component Deep Dive

The genuinely hard parts are (1) where the content blob lives, (2) key generation, (3) serving reads and how privacy interacts with caching, and (4) expiry at scale. I walk each from naive to scalable.

1. Where the content lives: metadata/blob split

This is the part people get wrong, so it comes first.

Approach A: store the body in the database (the naive instinct)

The obvious design: one row per paste, with the content in a TEXT/BLOB column.

CREATE TABLE pastes (
  paste_id   VARCHAR(7) PRIMARY KEY,
  content    TEXT,          -- the whole paste body, up to 10MB
  language   VARCHAR(20),
  privacy    VARCHAR(10),
  expires_at TIMESTAMP,
  created_at TIMESTAMP
);

Where it breaks at scale:

  • Row bloat destroys everything the DB is good at. A store tuned for fast point lookups now holds multi-KB-to-multi-MB rows. A metadata lookup that should touch 250 bytes drags a 10KB (or 10MB) body off disk with it. Cache locality collapses; you cannot keep the working set of metadata in memory because the bodies crowd it out.
  • Backups and replication balloon. You are now replicating and snapshotting 36.5 TB/year of blob through the database’s write path. Replica catch-up, restore time, and storage cost all explode. The DB becomes a very expensive, very slow file server.
  • Bandwidth on the wrong tier. Every read pulls the body through the database connection and the application server. At 350 MB/sec peak you are pushing all of it through your most expensive, hardest-to-scale layer.
  • You cannot put a database row behind a CDN. The single most effective optimization - serving immutable content from the edge - is unavailable, because the content is trapped in the DB.

This design works for a hobby Pastebin. At 10M pastes/day it is the mistake.

Approach B: metadata in a KV store, blob in object storage (the fix)

Split the two data planes by their nature:

  • Metadata (small, hot, needs fast point lookup and TTL): a distributed key-value store keyed by paste_id. This is what the Read Service hits first.
  • Content blob (large, cold-ish, immutable, cheap): an object store like S3/GCS, at a deterministic path blobs/{shard}/{paste_id}. The metadata row stores only a pointer.
metadata (KV, ~250 bytes/row):
  paste_id -> { blob_ptr, size, language, privacy, expires_at, owner_id, created_at }

object store (blob, up to 10MB):
  blobs/aB/aB3xK9   ->  <raw content bytes>

Why this is right:

  • The metadata store stays tiny (900 GB/year, not 36.5 TB), so its entire hot working set fits in RAM/cache and point lookups stay sub-10ms.
  • Object storage is built for exactly this: cheap, durable (11 nines on S3), horizontally infinite, and it fronts natively onto a CDN. The 350 MB/sec of read bandwidth is served by the object store + CDN, not by your app tier.
  • Immutability makes the blob perfectly cacheable. Since a paste never changes, the blob gets a long Cache-Control max-age and lives at the edge. The origin object store sees only cache misses.
  • Blobs and metadata scale independently. You can grow storage without touching the lookup tier.

Deterministic path vs stored pointer: because the blob path is derived from paste_id (e.g. blobs/{first 2 chars}/{paste_id}), you technically do not even need to store blob_ptr - you can compute it. Storing it explicitly is still worth it so you can migrate blobs (e.g. move cold pastes to a cheaper storage class) without rewriting the derivation rule. The 2-char prefix spreads objects across many object-store partitions to avoid a hot prefix.

Small-paste optimization (worth mentioning): a huge fraction of pastes are tiny (a few hundred bytes). Round-tripping to the object store for those adds latency for no bandwidth benefit. An inline threshold - store bodies under, say, 4KB directly in the metadata row and only push larger bodies to object storage - cuts object-store reads dramatically for the common case while keeping the big-blob path off the DB. This is a real production trade-off: it complicates the read path (branch on inline flag) in exchange for fewer object-store round-trips.

DesignMetadata lookupBackupsCDN-ableVerdict
Body in DBSlow (row bloat)Huge, slowNoNaive, breaks
Blob in object storeFast (250B rows)Small metadata + cheap object storeYesThe answer
+ inline small bodiesFast, fewer round-tripsSameYesBest for real traffic

2. Key generation

Same core problem as any short-key system, so I will be brisk. Three options: hash the content, single counter, distributed counter.

Approach A: hash the content (the trap)

paste_id = base62(sha256(content))[:7].

Where it breaks: truncating a hash to ~42 bits collides by the birthday bound at billions of pastes, forcing a read-before-write to check “does this key already hold a different paste?” on every create. Worse for Pastebin than for a URL shortener: two users pasting the identical content (a common config, a popular license) would hash to the same key and clobber each other’s privacy/expiry settings. Content-hashing fights the requirement that each paste is an independent object.

Approach B: single auto-increment counter (clean but does not scale)

Global counter, each paste gets the next integer, base62-encode it. Collision-free by construction, no read-before-write. Correct core idea. But a single UPDATE counter SET val = val + 1 row is a single point of failure and a write bottleneck; if it dies, creation stops globally.

Approach C: distributed range counter (the scalable fix)

Do not hand out IDs one at a time. Hand out blocks.

Coordinator holds: next_block_start = 4,000,000,000  (in etcd/ZooKeeper)

Create Service A asks for a block -> gets [4,000,000,000 .. 4,000,099,999]
Create Service B asks for a block -> gets [4,000,100,000 .. 4,000,199,999]

Each instance serves IDs from its block via a local atomic counter.
At ~90% used it pre-fetches the next block (no stall).
base62-encode each ID to get the 7-char paste_id.
  • The coordinator is hit once per 100K creates. At 350 WPS that is roughly one coordinator call every ~5 minutes - trivial to make HA.
  • IDs are minted from RAM; no DB on the create hot path for the key.
  • Globally unique because blocks never overlap. If an instance crashes mid-block, the remaining IDs are simply burned - with 3.5 trillion keys, wasting a few thousand is irrelevant.
  • Predictability: sequential counters make sequential, guessable keys. For a Pastebin that matters more than for a shortener because unlisted pastes rely on the key being unguessable. Scramble the integer with a bijective permutation (multiply by a large odd constant mod 62^7, or a small Feistel network) before base62-encoding. You keep uniqueness and collision-freedom while making keys non-enumerable. This is what makes unlisted privacy actually private.

Custom keys (if offered): the user supplies the key, so this path does need a uniqueness check - a single-shard INSERT IF NOT EXISTS on the primary key, which is cheap and atomic. Reserve generated keys to length 7 and let custom keys be a different length/charset to avoid collisions between the two spaces.

3. The read path, and how privacy shapes caching

Reads split by privacy, and privacy decides where a response is allowed to be cached. This is the subtle part.

Naive read: every request hits the metadata store, then the object store, then returns. Where it breaks: at 350 MB/sec of blob bandwidth, routing every read through the app tier and object store is expensive and slow, and it wastes the fact that content is immutable and highly skewed toward a hot set.

The fix is layered caching, gated by privacy:

  • public and unlisted: the raw blob is immutable and not secret-by-ACL (unlisted relies on key opacity, not auth), so it is CDN-cacheable. Serve /raw/{key} from the edge with a long max-age. This absorbs the overwhelming majority of read bandwidth. unlisted is safe at the edge because the CDN key includes the unguessable paste_id; nobody without the key can request it.
  • private: must not be cached at a shared CDN, because access depends on the requester’s identity. Mark these responses Cache-Control: private, no-store. Private reads always go to origin, authenticate the caller against owner_id, and only then fetch the blob. Private pastes are a small fraction of traffic, so keeping them off the CDN costs little.

Origin-side, behind the CDN:

  1. Metadata cache (Redis). GET paste:{key} returns {blob_ptr, privacy, expires_at, size, inline_body?}. >95% hit rate, sub-millisecond. This is where expiry and privacy checks happen.
  2. Hot-body cache (Redis). The hottest blob bodies (under some size cap, e.g. 64KB) are cached in Redis so an origin miss on the CDN still avoids the object store. Large bodies are never cached in Redis - they go straight to the object store to avoid blowing the cache.
  3. Object store (+ its own front cache). True long-tail bodies. Read-through: on miss, fetch, populate CDN and (if small) Redis.

Hot key problem: a single viral paste hammering one Redis shard or one object is solved primarily by the CDN (immutable content at the edge scales horizontally by design). As a backstop, each Read Service instance keeps a tiny in-process LRU of the top few thousand hot keys with a short TTL - 50K RPS on one paste spread across 50 instances is 1K RPS each, served from local memory.

HTML view vs raw view: the /raw/{key} response is pure content and CDN-cached aggressively. The HTML-wrapped /{key} view (with syntax highlighting shell, metadata, buttons) is assembled by the Read Service or rendered client-side; keep highlighting on the client so the cached artifact stays language-agnostic and the CDN stores one representation.

4. Expiry and cleanup at scale

Pastes expire. At 10M creates/day, most with a TTL, you are expiring on the order of millions of pastes daily. Doing this wrong quietly wrecks the system.

Naive: a cron job that scans and deletes

DELETE FROM pastes WHERE expires_at < NOW();  -- runs every minute

Where it breaks: a full-table (or full-keyspace) scan for expired rows at this volume is a heavy, lock-contending, IO-thrashing job that competes with live read/write traffic. It deletes millions of rows in bursts, causing replication lag spikes and compaction storms. And it only handles the metadata - the blobs are still sitting in the object store costing money.

The fix: native TTL + lazy deletion + object-store lifecycle

Three cooperating mechanisms, no giant scan:

  1. Native TTL on the metadata store. Stores like DynamoDB and Cassandra support a per-row TTL; set it to expires_at. The store reaps expired rows during its own background compaction - no application-driven scan, no scheduled DELETE storm. The row disappears on the store’s schedule.
  2. Lazy deletion on read. Native TTL reaping can lag (minutes to hours). So the Read Service also checks expires_at < now on every read and treats an expired paste as gone (404/410) even if the row is still physically present. This guarantees an expired paste is never served, regardless of reaper lag. On such a read, enqueue an explicit delete to be safe.
  3. Object-store lifecycle policy for blobs. Metadata TTL removes the pointer, but the blob must be reclaimed too. Two options: (a) write each blob with an object-store lifecycle/expiry matching the paste TTL when the TTL is fixed and known at write time, or (b) for variable expiry, run a low-priority async reaper driven by paste_expired events on Kafka that deletes orphaned blobs. Object deletes are cheap and out of the hot path.

Burn-after-reading is a special case: expires_at is effectively “first successful read.” Serve the content, then atomically mark the metadata row deleted (a conditional update on a read_count guard so two concurrent reads cannot both succeed - only the first CAS wins, the loser gets 404). Enqueue blob deletion. This is the one place we need a small atomic guarantee, and a single-key conditional write on the metadata store gives it cheaply.

This trio - store-native TTL for the bulk, lazy check for correctness, lifecycle/event-driven reaper for blobs - handles millions of expirations a day with zero scheduled scans.

API Design & Data Schema

API

POST /api/v1/pastes
Body:
{
  "content":   "def hello():\n    print('hi')",
  "language":  "python",            // optional hint, client-side highlight
  "privacy":   "unlisted",          // public | unlisted | private
  "expiry":    "1d",                // 10m | 1h | 1d | 1w | 1m | never | burn
  "title":     "quick snippet"      // optional
}
Response 201:
{
  "paste_id":  "aB3xK9",
  "url":       "https://pst.bin/aB3xK9",
  "raw_url":   "https://pst.bin/raw/aB3xK9",
  "expires_at":"2026-07-14T00:00:00Z",
  "privacy":   "unlisted"
}
Errors:
  400  empty content or invalid language
  413  paste exceeds 10MB size cap
  401  privacy=private without auth
  429  rate limit exceeded

GET /{paste_id}          -> HTML view (wrapped, highlight shell)
GET /raw/{paste_id}      -> plain text/plain body
Response 200:
  Content-Type: text/plain; charset=utf-8
  Cache-Control: public, max-age=31536000, immutable   (public/unlisted)
  Cache-Control: private, no-store                      (private)
  <raw paste content>
Errors:
  404  unknown, expired, or burned key
  410  expired (optional, instead of 404)
  401/403  private paste, not the owner

DELETE /api/v1/pastes/{paste_id}   -> owner-only, requires auth
Response 204

Data store choice: NoSQL KV for metadata, object store for blobs

The metadata access pattern is a pure primary-key lookup (paste_id -> metadata) with no joins and no hot-path range scans, at a scale (900 GB/year, 35K RPS) that wants automatic sharding, native TTL, and predictable single-key latency. That is the sweet spot for a distributed KV / wide-column store - DynamoDB or Cassandra - not a relational DB. Native TTL alone is a strong reason to pick these.

Why not SQL for metadata: a single relational instance cannot hold the row count or serve this rate, and once you shard it manually you lose joins/transactions (which we do not need - each paste is independent and immutable) while keeping SQL’s operational weight. The only strongly-consistent, tiny piece is the key coordinator, which lives in etcd/ZooKeeper.

The content blob goes in object storage (S3/GCS): cheap per GB, 11 nines durability, effectively infinite, and CDN-native. Never in the database.

Metadata table: pastes (KV)

Partition key: paste_id    (string, 7 chars)

paste_id      STRING   PK     -- "aB3xK9"
blob_ptr      STRING          -- "blobs/aB/aB3xK9" (object store path)
inline_body   BYTES  NULL     -- present iff size < 4KB (small-paste optimization)
size          INT             -- byte length of content
language      STRING NULL     -- highlight hint
privacy       STRING          -- public | unlisted | private
owner_id      STRING NULL     -- set for private pastes
title         STRING NULL
read_count    INT             -- for burn-after-reading CAS guard
created_at    TIMESTAMP
expires_at    TIMESTAMP NULL  -- native TTL attribute; store auto-reaps

Index: primary key only (paste_id). It is the sole hot-path access.
TTL:   enable native TTL on expires_at.

Owner index (optional, for “my pastes”):

pastes_by_owner
  partition key: owner_id
  sort key:      created_at (desc)
  value:         paste_id
-- only populated for authenticated creators; off the anonymous hot path.

Key coordinator (tiny, strongly consistent, in etcd/ZooKeeper):

counter_block
  next_block_start  BIGINT   -- monotonic, updated once per ~100K creates

Analytics rollup (separate store, e.g. ClickHouse):

paste_reads
  paste_id    STRING
  ts_bucket   TIMESTAMP   -- hourly bucket
  read_count  BIGINT
  geo         STRING
  referrer    STRING
order by: (paste_id, ts_bucket)

Analytics in its own store means heavy aggregation never touches the read path.

Bottlenecks & Scaling

Where it breaks first, in order, and the fix for each:

1. Read bandwidth (breaks first). 350 MB/sec of blob content cannot flow through the app tier and object store on every request. Fix: put immutable public/unlisted blobs behind a CDN with a year-long immutable max-age. The edge serves the vast majority of bytes; the origin object store sees only misses. This is why the metadata/blob split matters - you cannot CDN a database row.

2. Metadata store hot working set. Millions of point lookups a second must not hit disk. Fix: keep metadata rows tiny (blob lives elsewhere) so the working set fits in a Redis layer with >95% hit rate; read-through on miss. Shard the metadata store by paste_id (hash partitioning) so uniformly-distributed keys spread load evenly - no natural hot shard, since base62-of-counter keys are uniform.

Shard key choice: paste_id. Every read is by key, so this is the natural partition key and every lookup is single-shard, single-key. Sharding by created_at or owner_id would scatter by-key reads across shards - wrong.

3. Key generator as SPOF / bottleneck. Fix: distributed range allocation. The coordinator is hit once per block (~every 5 minutes), trivially HA in etcd/ZooKeeper. Instances pre-fetch the next block; crash-burned blocks are irrelevant against a 3.5T key space.

4. Object store hot object / hot prefix. A single viral paste, or a skewed key prefix, overloads one object or partition. Fix: the CDN absorbs a single hot object by design; a per-instance in-process LRU backstops it. Spread objects across partitions with a 2-char path prefix so no single object-store prefix goes hot on writes.

5. Expiry storms. Deleting millions of expired pastes must not thrash the store. Fix: native store TTL for the bulk reap, lazy expiry check on read for correctness during reaper lag, and object-store lifecycle / event-driven blob reaper for the content. No scheduled full-scan DELETE.

6. Analytics and abuse scanning on the write/read path. Fix: emit paste_created / paste_read events to Kafka asynchronously. Analytics aggregation, malware/secret scanning, and the public search index all consume off Kafka. A read or create never blocks on any of them; scanning that flags a paste can quarantine it after the fact.

7. Single-region latency for global users. Fix: CDN edge nodes serve blobs globally already. Deploy Read Service + metadata cache in multiple regions with GeoDNS/anycast; replicate the metadata store and object store cross-region. Because pastes are immutable, multi-region replication has no write-conflict problem. The create path can stay primary-region with async replication out; a brief lag only means a just-created paste might 404 in a far region for a second, mitigated by read-your-writes on the primary and CDN warm-on-create for the author.

8. Single points of failure. Stateless Create and Read services behind load balancers scale horizontally; the coordinator runs as an HA quorum; cache, metadata store, and object store are all replicated. No single box whose loss stops reads.

Wrap-Up

The trade-offs that define this design:

  • Split metadata from blob. Small metadata in a sharded KV store with native TTL, big content in object storage fronted by a CDN. This is the single most important call - it keeps lookups fast, backups cheap, and 350 MB/sec of reads off the app tier. Putting the body in the database is the classic mistake.
  • Distributed counter + base62 with bijective scrambling over hashing. Zero collisions, zero read-before-write, and non-enumerable keys so unlisted privacy actually holds.
  • Immutability makes caching safe and total. Pastes never change, so blobs get a year-long immutable CDN cache with no invalidation, at the edge, in Redis, and in-process. This is what makes a billion reads a day feasible.
  • Privacy gates the cache tier. public/unlisted at the edge; private never shared-cached and always authenticated at origin.
  • Expiry without scans. Store-native TTL for the bulk, lazy check on read for correctness, lifecycle/event-driven reaper for blobs. Burn-after-reading uses a single-key CAS so exactly one read wins.
  • Everything non-essential off the hot path. Analytics, abuse scanning, and search index all consume async from Kafka; a read or create never blocks on them.

One-line summary: a read-heavy, immutable system that mints collision-free scrambled base62 keys, stores tiny metadata in a TTL-native sharded KV store while the large content blob lives in CDN-fronted object storage, gating the cache tier by privacy and reaping expired pastes with native TTL plus lazy checks instead of scans.