“Send an email” is one API call. Then you ask the real questions. Who signs the message so Gmail does not dump it in spam? Which IP address does it go out on, and what happens to every other customer on that IP when one of them sends garbage and gets it blocklisted? What do you do when Yahoo’s MX server answers “451 try again later” - retry now, retry in an hour, give up? When the recipient clicks “this is spam,” how do you ever find out, and how do you stop mailing them? How do you know the message was even opened? A transactional email service is not a mail-send function. It is a distributed SMTP sending plant whose actual product is deliverability - getting mail into the inbox, not the spam folder, not bounced - at a scale where you are sending billions of messages a day through thousands of receiving mail servers you do not control, on a pool of IP addresses whose reputation is a shared, fragile, slowly-earned asset.
The core tension: the easy part (accept an API call, store a message) is trivial, and the hard part (convincing someone else’s mail server to accept and inbox your message, repeatedly, forever) is a reputation game played against adversarial spam filters with no clean API. Let me build the whole thing.
Functional Requirements (FR)
In scope:
- Submit an email. A customer sends a message via REST API (
POST /mail/send) or via authenticated SMTP relay. It has from/to, subject, HTML/text body, attachments, headers, and a template option. - Reliable delivery to the recipient’s mailbox provider. We resolve the recipient domain’s MX servers and deliver over SMTP, with retries, backoff, and eventual bounce classification.
- Authentication and signing. Every message is signed with DKIM, aligned with SPF and DMARC for the sending domain, so mailbox providers trust it.
- Sender reputation management. Outbound mail is spread across pools of IP addresses. We manage warm-up, per-IP and per-domain reputation, and isolate bad senders so one customer cannot poison the pool for everyone.
- Bounce handling. Hard bounces (mailbox does not exist) and soft bounces (full mailbox, greylisting) are parsed, classified, and fed back. We suppress addresses that hard-bounce.
- Feedback loops (FBL) and complaints. When a recipient marks mail as spam, mailbox providers send us a complaint via FBL; we parse it, attribute it to the customer, and suppress.
- Engagement tracking. Open tracking (pixel) and click tracking (link rewriting), reported back to the customer as events.
- Suppression lists. Per-customer and global lists of addresses we must not mail (hard bounces, unsubscribes, complaints, invalid).
- Event webhooks. Delivered, bounced, deferred, opened, clicked, spam-reported, dropped - streamed back to the customer.
Explicitly out of scope (own the boundary):
- Inbound mail / full mailbox hosting. We send transactional mail; we are not building Gmail. We do receive bounces and FBL mail, but we do not host user inboxes.
- Marketing campaign UI - audience segmentation, drag-and-drop editors, scheduling. We are the delivery substrate those sit on.
- The template authoring UI. We store and render templates; the editor is a separate product surface.
- Deep anti-spam content scoring on our side. We enforce policy and reputation, but we are not building SpamAssassin; we describe the hooks.
The decision that shapes everything: our product is not “send,” it is “land in the inbox,” and inboxing is governed by a reputation we build slowly and can destroy in an afternoon. Throughput is almost easy. The hard engineering is reputation isolation across shared infrastructure, correct SMTP conversation handling against thousands of heterogeneous receivers, and closing the feedback loops (bounce, complaint, engagement) so the system learns who to stop mailing. That is where the design lives.
Non-Functional Requirements (NFR)
- Scale: a large provider. Assume 5 billion messages/day. That is the headline number everything derives from. Thousands of customers, long tail plus a few whales. Peak is roughly 3x average (business-hours concentration plus scheduled sends), so ~170K messages/sec at peak.
- Latency: this is not a low-latency system in the inbox-arrival sense - email is asynchronous by nature and a few seconds to a few minutes is fine, and deferrals can legitimately take hours. What must be fast is API/SMTP acceptance: p99 under ~100ms to accept and durably queue a message. Delivery latency target: p50 under a few seconds, p99 under a minute for messages that are not being deferred by the receiver.
- Availability: 99.95%+ on ingestion. Accepting mail must almost never fail; a durable queue absorbs any downstream sending problem. Sending itself is best-effort against receivers that go down or throttle us - that is expected and handled by retries, not by failing ingestion.
- Durability: once we return
202 Accepted, the message is on durable storage and will be attempted until delivered, bounced, or expired (typically a 72-hour retry window matching standard MTA behavior). Losing an accepted password-reset email is a real user-facing failure. - Consistency: eventual. Event reporting (delivered/opened) can lag seconds. The one thing that must be strong is suppression: once an address is on the hard-bounce or complaint suppression list, we must not send to it again. A suppression miss mails a known-bad address and damages reputation, so suppression checks are authoritative, not best-effort.
- Ordering: none required globally. Each message is independent. We do not promise message A arrives before message B.
- Deliverability as an SLO: the real quality metric. Inbox placement rate, bounce rate under provider thresholds (keep hard-bounce under ~2-5% or providers start blocking), complaint rate under ~0.1% (Gmail/Yahoo suspend senders above ~0.3%). These thresholds literally drive the reputation isolation design.
Back-of-the-Envelope Estimation (BoE)
Real numbers with arithmetic - they dictate the architecture.
Ingestion throughput:
5,000,000,000 messages/day
= 5e9 / 86,400 sec
≈ 57,870 messages/sec average
Peak (~3x, business hours + scheduled blasts) ≈ 170,000 messages/sec
SMTP delivery attempts (more than sends, because of retries/deferrals):
Assume ~1.4 SMTP attempts per message on average
(first attempt + some deferrals/greylisting retries).
5e9 * 1.4 = 7e9 attempts/day
≈ 81,000 attempts/sec average, ~240K/sec peak
Each attempt is a (possibly reused) SMTP/TLS conversation to a receiving MX.
Storage - message bodies:
Average message size (HTML + text + small inline images, excluding big attachments)
≈ 75 KB. Attachments are the long tail; many messages are ~20 KB, some are MBs.
5e9 * 75 KB = 375 TB/day of raw body.
We do NOT keep bodies long. Bodies live in blob storage only until delivered + a short
grace window (say 3-7 days for retries + debugging), then purged.
Online body store ≈ 375 TB/day * 5 days ≈ ~1.9 PB, on object storage, cheap and tiered.
Storage - event/metadata records (the part we keep):
Per message we keep a compact metadata + events record, not the body:
message_id 16 bytes
customer_id 8
from/to (hashed) ~64
status 1
ip_sent_on 4
timestamps ~24
event pointers ~40
------------------------
≈ 160 bytes, round to ~250 with indexes/overhead
5e9 * 250 bytes = 1.25 TB/day of metadata
* 90 days retention ≈ ~110 TB online, sharded. Older rolls to cold analytics.
Plus engagement events (opens/clicks): assume 30% open, 5% click:
opens: 1.5e9/day, clicks: 2.5e8/day -> ~1.75e9 engagement events/day
* ~120 bytes ≈ 210 GB/day of events.
Bandwidth:
Outbound body egress at delivery: 5e9 * 75 KB ≈ 375 TB/day egress to receivers
≈ 375e12 * 8 / 86,400 ≈ ~35 Gbps average, ~100 Gbps peak.
This is the real cost center alongside IPs. Egress dominates the bill.
IP address math (the heart of reputation):
A warmed, reputable IP can sustainably send to a strict provider (Gmail) at a bounded
rate - call it ~1-5 messages/sec sustained per IP per receiving domain before throttling,
higher for lenient receivers. Say an IP pushes ~100K-500K msgs/day sustainably when warm.
5e9 / ~300K per IP/day ≈ ~16,000 IPs needed in steady state.
Round to the order of ~10,000-20,000 IPs across pools and regions.
These are a scarce, expensive, slowly-warmed asset. The whole reputation design exists
because you cannot just "add more IPs" instantly - a cold IP sends almost nothing.
Suppression list:
Hard bounces + complaints + unsubscribes accumulate. Global suppression could reach
billions of entries over time. Store as a scalable KV / bloom-filtered set:
assume ~2e9 suppressed addresses * ~40 bytes (hashed) ≈ 80 GB authoritative,
fronted by a per-shard bloom filter for fast "definitely not suppressed" checks.
The headline: raw throughput (170K/sec) is large but tractable with horizontal scaling. The genuinely hard, non-obvious constraints are the ~16,000 IP reputation budget (you cannot buy your way past a cold IP), the egress bandwidth bill, and the feedback loops that must close correctly or your reputation decays. Those are the design.
High-Level Design (HLD)
The spine: a thin ingestion layer (REST + SMTP relay) that validates, checks suppression, and durably enqueues; a sending pipeline that signs, picks an IP from the right pool, resolves MX, and runs the SMTP conversation; and a set of feedback-loop ingestion paths (bounces, FBL complaints, open/click tracking) that flow back into suppression and reputation. Durable queues between accept and send are what buy availability and the 72-hour retry window.
┌──────────────────────────────────────────────────────────────┐
│ Customers │
│ REST API (POST /mail/send) Authenticated SMTP relay │
└──────────────┬──────────────────────────────┬─────────────────┘
│ │
┌────────▼────────┐ ┌────────▼─────────┐
│ REST Ingest API │ │ SMTP Ingest MTA │
│ validate+auth │ │ AUTH, parse │
└────────┬────────┘ └────────┬─────────┘
│ suppression check │
│ ┌────────────────────────┐ │
└──▶│ Suppression Service │◀──┘
│ (bloom + KV, per-cust │
│ + global) │
└────────────────────────┘
│ store body │ enqueue envelope
┌───────▼──────────┐ ┌────────▼──────────────┐
│ Blob Body Store │ │ Submission Queue │
│ (S3/object) │ │ (Kafka, partitioned)│
└──────────────────┘ └───────────┬───────────┘
│ consumed
┌───────────▼───────────┐
│ Sending Scheduler │
│ 1 pick IP pool/IP │
│ 2 DKIM sign │
│ 3 MX resolve (cached) │
│ 4 enqueue to MTA │
└───┬──────────┬─────────┘
reputation read ┌─────┘ └──────┐ per-destination queues
┌──────────────────────┐ │ ┌───────────▼───────────────┐
│ Reputation / IP Pool │◀─┘ │ Destination Queues │
│ Manager (per-IP, │ │ sharded by receiving │
│ per-domain state) │ │ domain (gmail, yahoo...) │
└──────────────────────┘ └───────────┬───────────────┘
│
┌─────────────────────▼─────────────────┐
│ Outbound MTA Fleet │
│ SMTP conversation to recipient MX, │
│ connection pooling per (IP, domain), │
│ rate shaping, parse SMTP reply codes │
└───┬───────────────┬───────────────┬────┘
delivered │ deferred │ bounced │
┌────────────────▼───┐ ┌───────▼──────┐ ┌─────▼────────────┐
│ Event Store + │ │ Retry/Delay │ │ Bounce Classifier│
│ Webhook Dispatcher │ │ Queue (TTL) │ │ -> Suppression │
└────────────────────┘ └──────────────┘ └──────────────────┘
Feedback-loop ingress (asynchronous, flowing back into suppression/reputation):
┌─────────────────────┐ ┌──────────────────────┐ ┌───────────────────────┐
│ Return-path bounces │ │ FBL complaint mail │ │ Open pixel / click │
│ (async DSN to our │ │ (ARF from Gmail, │ │ redirect endpoints │
│ bounce MX) │ │ Yahoo, etc.) │ │ (tracking service) │
└─────────┬───────────┘ └──────────┬───────────┘ └───────────┬───────────┘
└──────────────────────────┴───────────────────────────┘
into Suppression + Reputation + Event Store
Ingestion flow (synchronous, must be fast):
- Customer calls
POST /mail/send(or opens an authenticated SMTP session to our relay). We authenticate the API key / SMTP credentials and validate the message (required headers, size limits, customer sending policy). - For each recipient, we check the suppression list (customer-scoped and global). Suppressed recipients are dropped immediately with a
droppedevent - we never attempt them. This is the single most important pre-send check for reputation. - We write the message body to blob storage (returning a reference) and the envelope (from, to, customer_id, body_ref, headers, assigned message_id) to the durable Submission Queue.
- Return
202 Acceptedwith themessage_id. The customer is done; everything expensive is asynchronous.
Sending flow (asynchronous):
- The Sending Scheduler consumes envelopes. It asks the IP Pool Manager which IP to send from, based on the customer’s assigned pool (dedicated vs shared), the receiving domain, and current per-IP reputation/throttle state.
- It DKIM-signs the message for the sending domain and ensures SPF/DMARC alignment, then resolves the recipient domain’s MX records (heavily cached).
- It routes the message onto a per-destination queue sharded by receiving domain (one lane for Gmail, one for Yahoo, etc.) so we can shape rate per receiver independently.
- The Outbound MTA fleet consumes a destination queue, opens or reuses a pooled SMTP/TLS connection from the chosen IP to the recipient MX, and runs the SMTP conversation (
EHLO,MAIL FROM,RCPT TO,DATA). - It parses the SMTP reply:
- 2xx -> delivered. Emit
deliveredevent; record which IP/domain, feeding reputation. - 4xx -> transient (greylisting, rate limit, “try later”). Re-enqueue on the retry/delay queue with backoff, within the 72-hour window.
- 5xx -> permanent. Hand to the Bounce Classifier; hard bounce -> suppress the address.
- 2xx -> delivered. Emit
- Asynchronous bounces (DSNs): some receivers accept then later bounce via a message to our return-path. Our bounce MX ingests these, classifies, and suppresses.
- Open/click tracking: we inject a tracking pixel and rewrite links. When the recipient opens or clicks, our tracking endpoints log the event and dispatch a webhook.
- FBL complaints: mailbox providers forward “marked as spam” reports (ARF format) to our complaint address; we parse, attribute to the customer, suppress, and feed reputation.
The structural insight: there are two decoupling layers. The Submission Queue decouples the customer from all sending work (availability + the long retry window). The per-destination queues decouple receivers from each other so a throttling Gmail cannot back up Yahoo delivery, and so each receiver gets independently shaped traffic. Wrapped around the whole thing is a feedback loop - bounces, complaints, engagement - that continuously updates suppression and reputation, which is what makes the next send better.
Component Deep Dive
The hard parts, naive-first then evolved: (1) the outbound MTA and SMTP conversation at scale, (2) sender reputation across shared IP pools, (3) bounce and complaint feedback loops, (4) open/click tracking.
1. The outbound MTA and SMTP delivery pipeline
The job: run millions of SMTP conversations a minute to thousands of heterogeneous receiving mail servers, honoring each one’s quirks and rate limits, with retries.
Naive approach: a worker loops recipients and opens a fresh SMTP connection per message
The obvious first cut: for each queued message, resolve MX, open a TCP+TLS connection to the recipient MX, do the full SMTP handshake, send one message, close, repeat.
def send(msg):
mx = resolve_mx(msg.to_domain) # DNS every time
conn = smtp_connect(mx) # new TCP + TLS handshake
conn.ehlo(); conn.starttls()
conn.mail_from(msg.from); conn.rcpt_to(msg.to)
conn.data(msg.body) # one message
conn.quit() # tear it all down
Where it breaks:
- Connection churn. A fresh TCP + TLS handshake per message is brutal: handshakes dominate latency and CPU, and receivers interpret a storm of one-message connections from your IP as abusive, which hurts reputation. Real MTAs send many messages per connection (SMTP pipelining, connection reuse).
- DNS hammering. Resolving MX per message floods your resolver; MX records are stable and must be cached.
- No per-receiver rate awareness. Gmail will throttle or defer you if you open too many concurrent connections or push too fast from one IP. A naive worker opening unbounded connections gets
421 too many connectionsand tanks its IP reputation. - No backoff on deferral. A
451means “try later”; retrying immediately in a tight loop is exactly the wrong thing and looks like an attack. - Head-of-line blocking. One slow receiver (Yahoo deferring) stalls a worker that also has Gmail messages queued behind it.
A connection-per-message loop fails on reputation, efficiency, and isolation all at once - it is the behavior of a spammer, not an MTA.
Evolved approach: per-(IP, destination) connection pools, destination-sharded queues, and reply-code-driven rate shaping
Model the MTA as a fleet of senders where the unit of work is a (source IP, destination domain) channel, not a single message.
- Destination-sharded queues. Messages are routed into per-receiving-domain queues (Gmail lane, Yahoo lane, Outlook lane, long-tail lane). Each lane is drained by workers that understand that receiver’s throttling behavior. A throttling Gmail backs up only the Gmail lane.
- Connection pooling with pipelining. For each
(IP, destination MX)we maintain a pool of persistent SMTP/TLS connections and push many messages per connection using SMTP pipelining. This amortizes the handshake and is what receivers expect from a legitimate MTA. - MX caching. MX lookups are cached with TTL; a DNS resolution serves thousands of messages.
- Reply-code-driven adaptive rate shaping. The MTA watches the stream of SMTP reply codes per
(IP, destination)and treats them as a live control signal:
| Reply | Meaning | MTA action |
|---|---|---|
| 250 | delivered | success; nudge the allowed rate up slowly |
| 421 | too many connections / not now | reduce concurrency to this destination, back off |
| 451 / 4xx | greylisting, temp-fail, rate limit | re-queue to delay queue with exponential backoff |
| 550 / 5xx | mailbox unknown, blocked | hard bounce -> classify + suppress |
| 554 “spam” | content/reputation reject | reputation hit; flag customer, throttle |
- Backoff and the 72-hour window. Transient failures go to a delay queue keyed by next-attempt time, with exponential backoff (e.g. 15m, 1h, 4h, 12h…) up to a 72-hour expiry, after which the message is a timed-out bounce. The worker never sleeps on a deferral; it re-enqueues and moves on.
- Concurrency governors per (IP, destination). Each destination has a learned ceiling on concurrent connections and send rate per IP. Exceeding it earns
421s, so the governor keeps us just under the line. This is per receiver because Gmail, Yahoo, and a small corporate Exchange all tolerate wildly different rates.
Gmail lane ─▶ [Gmail queue] ─▶ Gmail workers
│ pool of persistent conns per sending IP
┌─────────┼─────────┐
conn(IP_a) conn(IP_b) conn(IP_c) (pipelined, many msgs each)
│ │ │
watch reply codes -> adjust concurrency + rate per (IP, gmail)
│
2xx->delivered 4xx->delay queue(backoff) 5xx->bounce classifier
Why this is right: the MTA now behaves like a well-run mail server - persistent pooled connections, per-receiver rate discipline driven by live feedback, proper backoff - which is exactly what protects IP reputation while maximizing throughput. Isolation by destination means one slow receiver cannot stall the rest, and the adaptive governor lets us push each receiver as fast as it will tolerate and no faster.
2. Sender reputation across shared IP pools
This is the defining problem of the whole system. Deliverability is a function of the reputation of the IP address and sending domain the mail goes out on. Reputation is earned slowly (consistent volume, low complaints, low bounces) and lost fast. Because IPs are a scarce warmed asset shared across customers, one customer’s bad behavior can poison an IP for everyone on it.
Naive approach: one big shared pool, round-robin every message across all IPs
The simplest design: put all ~16,000 IPs in one pool and round-robin every outgoing message across all of them regardless of customer.
Where it breaks:
- Reputation blast radius is the whole fleet. If one customer sends to a dirty list and racks up bounces and spam complaints, that behavior is smeared across every IP, so every customer’s deliverability drops. One bad actor degrades the entire platform.
- No warm-up discipline. A brand-new IP round-robined into full production traffic sends cold, gets throttled and flagged, and never builds reputation - you cannot add capacity.
- Receivers track consistency. Gmail builds a reputation model per IP and per domain partly on consistent volume. Random round-robin means each IP has erratic volume, which reads as suspicious; dedicated high-reputation senders look different from a jittery shared blob.
- No way to isolate a whale or a sensitive sender. A bank wants its password-reset mail on IPs no one else touches; one shared pool cannot offer that.
- Compromised-account blast. If one customer’s API key leaks and a spammer blasts through it, it poisons every IP before you notice.
A single undifferentiated pool makes reputation a shared fate with no isolation, no warm-up path, and no way to protect good senders from bad ones.
Evolved approach: tiered IP pools, reputation-based routing, automated warm-up, and per-customer isolation
Treat IPs as a managed, segmented, reputation-scored resource.
- Pool tiers.
- Dedicated IPs for whales and sensitive senders (banks, large transactional customers). Their reputation is their own; they cannot be poisoned by others and cannot poison others.
- Shared pools, segmented by sender quality. Multiple shared pools, and customers are routed to a pool matching their behavior: a high-reputation shared pool for customers with clean history (low bounce/complaint), and a probation/low-reputation pool for new or risky customers. New customers and anyone whose complaint rate spikes land in the lower pool, so their behavior cannot contaminate the clean pool.
- Reputation scoring loop. The IP Pool Manager maintains per-IP and per-sending-domain state: recent delivery rate, bounce rate, complaint rate, and receiver-reported signals (e.g. Google Postmaster Tools, SNDS). A score per IP drives routing - a dipping IP gets less traffic until it recovers; a healthy one gets more.
- Automated IP warm-up. New IPs cannot take full volume. The warm-up controller ramps an IP’s allowed daily volume on a schedule (e.g. day 1: a few thousand, doubling every few days over ~4-6 weeks), and biases warm-up traffic toward engaged recipients (addresses that recently opened), because positive engagement is what builds reputation fastest. Warm-up is per receiving domain too, since Gmail and Outlook must each be warmed.
- Per-customer reputation isolation and quarantine. Each customer has a reputation score (bounce rate, complaint rate, spam-trap hits). Crossing thresholds triggers automatic action: throttle, move to the probation pool, or hard-pause sending and alert. This is the circuit breaker that stops a compromised or abusive account from torching shared IPs. It is the single most valuable control in the system.
- Domain reputation and authentication alignment. Modern receivers weight the sending domain’s reputation (via DKIM
d=domain and DMARC alignment) at least as much as the IP. So we enforce per-customer authenticated sending domains (their ownd=with a CNAME-delegated DKIM key), which both improves alignment and scopes domain reputation to the customer - another isolation axis.
Message ─▶ IP Pool Manager
│ customer reputation score? sending domain? receiver?
┌─────────┼───────────────────────┬─────────────────────┐
dedicated pool high-rep shared pool probation pool warm-up controller
(this customer (clean customers) (new/risky) (ramps new IPs on
only) engaged traffic)
│ per-IP reputation + per-domain reputation + receiver throttle state
└─▶ choose specific IP, below its warm-up/rate ceiling, healthiest first
Why this is right: reputation damage is contained - a bad customer hits only their own pool (or dedicated IP), never the clean shared pool, and a per-customer quarantine stops the bleed automatically. Warm-up gives a real path to add capacity instead of cold IPs that never work. Reputation-based routing continuously steers traffic toward healthy IPs and away from dipping ones, and domain-level authentication scopes reputation to the actual sender. This segmented, scored, self-healing pool is the actual product.
3. Bounce and complaint feedback loops
The job: learn, reliably, which addresses are dead or hostile, and stop mailing them - because continuing to mail bad addresses is the fastest way to destroy reputation.
Naive approach: treat only synchronous 5xx at send time as a bounce, log it, do nothing else
The first cut: if the SMTP DATA gets a 5xx, mark the message bounced and log it. Ignore the rest.
Where it breaks:
- Asynchronous bounces are missed. Many receivers accept the message (
250) and then bounce it later by sending a DSN (Delivery Status Notification) to your return-path address. If you only look at the synchronous reply, you never learn the mailbox does not exist, and you keep mailing it. - No suppression. Logging a bounce without suppressing means you re-mail the same dead address on the next send, racking up bounces that receivers count against you.
- Spam complaints are entirely invisible. When a recipient clicks “report spam,” there is no SMTP error - the receiver silently files it and, if enrolled, sends you an FBL complaint via a separate channel. Ignoring FBL means you never learn you are annoying people, and your complaint rate quietly rises until Gmail blocks you.
- No classification. A full mailbox (
452) is transient and should be retried; a nonexistent mailbox (550) is permanent and should suppress. Treating them the same either suppresses recoverable addresses or keeps retrying dead ones. - Spam traps. Some bad addresses are spam traps planted by blocklist operators; mailing them is catastrophic for reputation, and you only avoid them via good list hygiene fed by bounce history.
Synchronous-5xx-only handling misses async bounces entirely, never closes the loop into suppression, and is blind to complaints - which are the signals that actually govern reputation.
Evolved approach: a return-path bounce MX, an FBL ingestion pipeline, typed classification, and authoritative suppression
Build real feedback-loop ingestion as first-class inbound pipelines.
- Return-path (VERP) bounce handling. We set the SMTP envelope return-path to a unique per-message address using VERP (Variable Envelope Return Path), e.g.
bounce-<message_id>@bounces.oursvc.com. When an async DSN arrives at our bounce MX, the address itself tells us exactly which message bounced - no fuzzy matching. Our bounce MX fleet ingests these DSNs, parses the status code and diagnostic, and classifies. - Typed bounce classification.
| Class | Example codes | Action |
|---|---|---|
| Hard bounce | 550 mailbox unknown, 553 invalid | Suppress address permanently (global + customer). Never retry. |
| Soft bounce | 452 mailbox full, 421 temp | Retry within the 72h window with backoff; suppress only after repeated soft bounces. |
| Block | 554 spam / reputation reject | Reputation hit for the IP/customer; throttle, do not suppress the address. |
| Auto-reply / vacation | 2xx-ish DSN | Ignore; not a bounce. |
- FBL (Feedback Loop) ingestion. We enroll our sending IPs/domains in mailbox providers’ FBL programs. When a recipient marks our mail as spam, the provider sends an ARF (Abuse Reporting Format) complaint to our complaint address. An FBL pipeline parses ARF, extracts the original message (via headers we embed, e.g. a
Feedback-IDand the VERP return-path), attributes it to the customer, suppresses that recipient for that customer, and records a complaint event against the customer’s and IP’s reputation. - Authoritative suppression as a hard gate. Suppression is checked at ingestion (synchronously, before accepting a recipient) and the list is treated as authoritative. Structure: a global suppression set plus per-customer sets, stored in a scalable KV, fronted by a bloom filter per shard so the common “not suppressed” case is a fast in-memory check. A bloom hit falls through to the authoritative KV to avoid false positives. This is the one place we accept a small synchronous cost on the ingest path because sending to a known-bad address is exactly the mistake that destroys reputation.
- Feedback into reputation. Every bounce and complaint updates the per-IP, per-domain, and per-customer reputation scores from deep dive 2, closing the loop: bad lists -> bounces/complaints -> suppression + reputation drop -> routing to probation pool / quarantine -> protecting everyone else.
Async DSN ─▶ Bounce MX ─▶ parse VERP return-path (exact message) ─▶ classify
│
ARF complaint ─▶ FBL pipeline ─▶ parse Feedback-ID ─▶ attribute to customer
│
┌─────────────────────────────────────────┤
Suppression Service (global + per-customer) Reputation scores
(bloom + authoritative KV) (IP, domain, customer)
│ │
checked synchronously at ingest drives pool routing + quarantine
Why this is right: we capture bounces on both the synchronous and asynchronous paths (VERP makes async attribution exact), we actually ingest complaints instead of being blind to them, we classify so transient issues retry and permanent ones suppress, and suppression is an authoritative pre-send gate. The whole loop feeds reputation, which is the mechanism by which the system stops mailing bad addresses and thereby protects deliverability - the entire point of the product.
4. Open and click tracking
The job: report when a recipient opens a message and clicks a link, at billions of events a day, without hurting deliverability.
Naive approach: embed a tracking pixel and rewritten links that write straight to the primary database
Inject a 1x1 pixel <img src="track.oursvc.com/open?id=MSG"> and rewrite every link to track.oursvc.com/click?id=MSG&url=.... On each hit, synchronously write a row to the main event database.
Where it breaks:
- Write amplification into the hot store. 1.75 billion engagement events/day hitting the primary event DB synchronously will crush it; these are high-volume, bursty (a big send opens all at once), append-only writes.
- Tracking endpoint is on the critical path of someone else’s email client. If the tracking service is slow, the recipient’s mail client hangs loading the pixel - a bad look and irrelevant to our DB write. The pixel must return instantly (a tiny image) regardless of whether the event is persisted yet.
- Hot keys on viral sends. A single large campaign means millions of events for a few
message_ids / templates in a short window, hammering one shard. - Open tracking is noisy, and over-tracking hurts deliverability. Image proxies (Apple Mail Privacy Protection, Gmail image caching) pre-fetch pixels, inflating opens and sometimes marking opens that never happened. Rewriting every link also makes mail look more like spam (mismatched display vs actual URL) if done carelessly, and broken tracking links are a deliverability and trust problem.
Synchronous writes from a latency-sensitive public endpoint into the primary store do not survive the volume or the burstiness, and naive rewriting degrades deliverability.
Evolved approach: edge tracking endpoints, async event pipeline, and careful link signing
Make tracking an asynchronous, edge-served pipeline that never blocks the pixel and never touches the hot DB synchronously.
- Edge-served, instant-return endpoints. The open pixel and click redirect are served from lightweight, geographically distributed endpoints. The open endpoint returns the 1x1 GIF immediately; the click endpoint issues a
302redirect to the real URL immediately. Event recording is fire-and-forget - the handler drops an event onto a queue and returns, never waiting on a DB write. - Async event pipeline. Tracking events flow into a durable stream (Kafka), consumed by writers that batch-insert into a write-optimized event store (wide-column / time-series), and by the webhook dispatcher that streams
opened/clickedevents to the customer. Batching absorbs the burstiness. - Signed, opaque tracking tokens. Instead of a raw
message_idin the URL, the pixel/link carries a signed opaque token encoding(message_id, link_id, customer). Signing prevents tampering and lets the endpoint validate without a DB lookup. Click links preserve the display domain via the customer’s own tracking subdomain (CNAME) so the links are not an obvious third-party redirect, which protects deliverability and trust. - Open-tracking realism. We account for image proxies: opens via known proxy prefetch (Apple MPP, Gmail’s image cache) are flagged as machine-opens vs human-opens, so reported engagement is not garbage. Click tracking is the more reliable engagement signal and is what the reputation/warm-up system leans on for “engaged recipients.”
- Hot-key spread. Events are partitioned by
message_id(high cardinality) rather than bycustomerortemplate, so a single big campaign’s events spread across partitions instead of hammering one. The webhook side aggregates per customer downstream.
Recipient opens mail / clicks link
│ (served from edge, returns pixel/302 instantly)
┌────▼─────────────┐
│ Tracking endpoint│──(fire-and-forget)──▶ [Event stream (Kafka)]
│ verify signed tok│ │
└──────────────────┘ ┌─────────────────┼───────────────┐
batch writers webhook dispatcher engagement signal
-> event store -> customer webhook -> reputation/warm-up
Why this is right: the public endpoints stay instant and never block on persistence, so a recipient’s mail client is never delayed and a traffic burst cannot take down the primary store. The async batched pipeline absorbs 1.75 billion events/day, signed tokens keep tracking tamper-proof and lookup-free, CNAMEd tracking domains protect deliverability, and partitioning by message_id spreads viral-send hot keys. Engagement flows back as the “engaged recipient” signal that warm-up and reputation depend on.
API Design & Data Schema
API
The core send endpoint customers call:
POST /api/v3/mail/send
Headers: Authorization: Bearer <api_key>
Body:
{
"from": { "email": "[email protected]", "name": "Acme" },
"personalizations": [
{ "to": [{ "email": "[email protected]" }],
"subject": "Reset your password",
"dynamic_template_data": { "name": "Ravi", "reset_url": "..." } }
],
"template_id": "d-abc123", // optional; or inline content below
"content": [
{ "type": "text/plain", "value": "..." },
{ "type": "text/html", "value": "..." }
],
"categories": ["password-reset"], // for analytics/segmentation
"tracking_settings": { "open_tracking": true, "click_tracking": true },
"ip_pool_name": "transactional", // optional; route to a specific pool
"custom_args": { "order_id": "A123" } // echoed back in webhook events
}
Response 202:
{ "message_id": "b1e2...", "status": "accepted" }
Errors:
400 invalid payload / unverified sending domain / missing content
401 bad API key
403 sending domain not authenticated (SPF/DKIM not set up)
413 message too large
429 customer send-rate exceeded
Alternatively, customers relay via authenticated SMTP (smtp.oursvc.com:587, STARTTLS, AUTH LOGIN with API key), which maps onto the same submission path.
Sending-domain and suppression management:
POST /api/v3/sender_domains -> start domain auth, returns DNS records (SPF, DKIM CNAMEs, DMARC)
GET /api/v3/sender_domains/{id} -> verification status
GET /api/v3/suppressions/bounces -> list suppressed (bounced) addresses
DELETE /api/v3/suppressions/bounces/{email} -> remove from suppression (explicit override)
GET /api/v3/suppressions/spam_reports -> complaint suppressions
Events and status:
GET /api/v3/messages/{message_id} -> per-recipient delivery + engagement status
POST /api/v3/webhooks -> register customer event webhook URL
# We POST event batches to the customer's URL:
# [{ message_id, email, event: delivered|bounce|deferred|open|click|spamreport|dropped,
# timestamp, ip, reason?, url?, custom_args }]
We return 202, not 200 - accepted and durably queued, not delivered. custom_args are echoed back in every webhook event so the customer can correlate without storing our ids.
Data stores
Different access patterns, different stores. Be explicit.
1. Message body store - object/blob storage (S3-class). Bodies are large (~75 KB avg, attachments bigger), write-once, read-a-few-times (send attempts), then purged after delivery + a short grace window. Object storage with lifecycle TTL is exactly right; the metadata record holds a body_ref.
Blob: messages/{yyyy-mm-dd}/{message_id} -> MIME body (headers + parts + attachments)
Lifecycle: delete after delivered + 5 days (or 72h expiry + grace).
2. Message metadata + delivery status - NoSQL wide-column (Cassandra/Scylla/DynamoDB). One row per (message, recipient). High write volume (5B/day), key-scoped lookups by message_id, status updates on delivery/bounce, TTL expiry, no joins. Wide-column sweet spot.
Table: messages
message_id UUID PARTITION KEY
customer_id BIGINT
to_email_hash BYTES -- hashed recipient (PII minimization)
from_domain STRING
status STRING -- accepted|queued|sent|delivered|deferred|bounced|dropped|spam
bounce_class STRING NULL -- hard|soft|block
ip_sent_on INET NULL
sending_pool STRING
attempts INT
created_at TIMESTAMP
updated_at TIMESTAMP
TTL: 90 days
Shard by: message_id (every status update + lookup is single-key)
Table: customer_messages -- "show this customer's recent sends"
customer_id BIGINT PARTITION KEY
bucket STRING CLUSTERING (e.g. yyyy-mm-dd-hh)
message_id UUID CLUSTERING
Shard by: customer_id (range scan a customer's recent messages)
3. Engagement events - write-optimized time-series / wide-column. Opens and clicks, append-only, 1.75B/day, bursty.
Table: events
message_id UUID PARTITION KEY
event_type STRING CLUSTERING -- open|click
ts TIMESTAMP CLUSTERING
ip INET
user_agent STRING
url STRING NULL -- for clicks
is_machine BOOL -- proxy/prefetch flag
Shard by: message_id (spreads viral-send hot keys)
TTL: 90 days online, roll to cold analytics
4. Suppression store - scalable KV + bloom filter. Global and per-customer suppressed addresses, checked synchronously at ingest. Billions of entries, point membership lookups.
KV: suppress:global:{email_hash} -> { reason, since }
KV: suppress:{customer_id}:{email_hash} -> { reason, since }
Per-shard bloom filter in memory -> fast "definitely not suppressed" negative check,
falls through to KV on a bloom hit to avoid false positives.
5. Reputation / IP pool state - low-latency store (Redis + a durable backing table). Per-IP, per-domain, per-customer scores and throttle/warm-up state, read on every send routing decision, updated by the feedback loop.
Redis: ip:{addr} -> { reputation, warmup_day, daily_cap, sent_today, throttle_until }
domain_rep:{sending_domain} -> { score }
customer_rep:{customer_id} -> { bounce_rate, complaint_rate, status }
Updated atomically (Lua) as deliveries/bounces/complaints stream in.
6. Customer / sending-domain config - SQL (Postgres). Accounts, API keys, authenticated sending domains, DKIM keys, pool assignments, webhook URLs. Small, relational, strongly consistent, low write volume - classic relational workload with real constraints (unique domains, foreign keys).
Table: sender_domains
id BIGINT PK
customer_id BIGINT FK
domain STRING UNIQUE
dkim_selector STRING
dkim_privkey BYTES (encrypted)
verified BOOL
pool_name STRING
Why these choices: the hot paths (metadata, events, suppression) are high-volume, key-scoped, TTL’d, join-free - NoSQL wide-column / KV for horizontal scale. Bodies are big blobs -> object storage. Reputation is read on every send and updated constantly -> in-memory store with a durable backing. Account/domain config is small, relational, and needs real constraints and consistency -> SQL. There is no workload here that needs global ACID transactions across the hot path, which is why NoSQL fits sending and SQL stays scoped to configuration.
Bottlenecks & Scaling
Where it breaks first, in order, and the fix for each.
1. Sender reputation collapse (breaks first, and worst). One customer mails a dirty list, bounces and complaints spike, and if they are on a shared IP it drags down everyone’s inbox placement. This is the defining failure. Fix: segmented IP pools (dedicated + quality-tiered shared + probation), per-customer reputation scoring with automatic throttle/quarantine, and reputation-based routing that steers away from dipping IPs. A compromised or abusive account is paused by the circuit breaker before it poisons the clean pool. Domain-level authentication scopes reputation to the sender.
2. Receiver throttling and 4xx deferrals under a big send. Gmail/Yahoo defer you if you push too hard from one IP; a naive blast earns 421/451 storms.
Fix: per-(IP, destination) concurrency governors driven by live reply codes, destination-sharded queues so one throttling receiver backs up only its own lane, connection pooling with pipelining, and exponential backoff on the delay queue within the 72-hour window. We shape outbound to exactly what each receiver tolerates.
3. IP capacity is not elastic. You cannot add throughput by adding cold IPs - they send almost nothing until warmed. Fix: the automated warm-up controller ramps new IPs over weeks, biased toward engaged recipients, per receiving domain. Capacity planning treats warmed IPs as the real, slow-to-grow constraint; dedicated-IP customers are pushed to warm their own.
4. Suppression miss -> mailing a known-bad address. A single dead/complaining address re-mailed is a direct reputation hit; spam traps are catastrophic. Fix: authoritative suppression as a synchronous ingest gate, bloom-filter-fronted for speed, fed by both synchronous 5xx and async VERP DSNs and FBL/ARF complaints. Suppression is strong consistency, not eventual, on the send path.
5. Blind to complaints. “Report spam” has no SMTP error; without FBL you never learn you are annoying people until you are blocked.
Fix: FBL enrollment + ARF ingestion pipeline, attributing each complaint to the customer via embedded Feedback-ID/VERP, suppressing and dinging reputation. Complaint rate is a first-class monitored SLO (keep under ~0.1%).
6. Engagement-event write amplification (1.75B/day, bursty). Synchronous writes from latency-sensitive tracking endpoints would crush the store and delay recipients’ mail clients.
Fix: edge endpoints that return the pixel/302 instantly and record fire-and-forget into a Kafka stream; batched writers into a write-optimized store; partition events by message_id to spread viral-send hot keys.
7. Egress bandwidth and body storage (375 TB/day egress, ~1.9 PB bodies). Real cost center. Fix: purge bodies after delivery + short grace (TTL lifecycle on object storage), keep only compact metadata for 90 days then roll to cold. Place MTA egress in regions near major receivers; reuse connections to cut TLS overhead.
8. Hot keys on reputation and suppression stores under a viral send. One customer/template/IP hammered at once.
Fix: shard reputation by ip/domain/customer_id, suppression by email_hash; atomic Lua for score updates to avoid lock contention; replicate hot reputation keys. High-cardinality keys (message_id, email_hash) spread naturally.
9. Priority inversion - a bulk send delaying a password reset. If transactional and bulk share a lane, a 100M-message campaign sits in front of an OTP email.
Fix: priority lanes / separate pools. Transactional traffic uses dedicated pools and short high-priority queues; bulk runs on its own lane that can lag without hurting critical mail. Customers can assign an ip_pool_name to keep transactional separate.
10. Single points of failure. Every stateful piece (Kafka, Redis, the stores, object storage) is replicated across AZs/regions; every stateless piece (ingest API, scheduler, MTA workers, tracking endpoints, bounce/FBL MX) is horizontally scaled behind LBs. Messages that exhaust retries or fail classification land in a dead-letter queue for inspection and replay - nothing vanishes silently. No single box whose loss stops mail from flowing.
Shard keys, stated plainly: Submission and destination Kafka topics -> partition by receiving domain for destination lanes (independent rate shaping) and by message_id within for spread. Messages table -> message_id (single-key status updates). customer_messages -> customer_id (recent-sends scan). events -> message_id (spreads viral hot keys). suppression -> email_hash. reputation -> ip / sending_domain / customer_id. Never shard the message/event stores by created_at - it concentrates all current writes on the “now” shard, a permanent hot spot.
Wrap-Up
The trade-offs that define this design:
- Accept fast, deliver slowly and asynchronously. The customer gets a 202 in under 100ms; all SMTP work happens behind a durable queue with a 72-hour retry window. We trade immediate delivery for availability, durability, and the ability to retry a deferring receiver for hours.
- Reputation isolation is the product. We segment IPs into dedicated, quality-tiered shared, and probation pools, score reputation per IP/domain/customer, and quarantine bad actors automatically - so one customer’s bad list can never poison another’s inbox placement. This is the single most important design decision and everything else serves it.
- Close the feedback loops or die. VERP return-paths make async bounce attribution exact, FBL/ARF ingestion captures otherwise-invisible complaints, and both feed an authoritative suppression gate and reputation scoring. A system that cannot learn who to stop mailing destroys its own reputation.
- The MTA behaves like a good mail server, not a spammer. Persistent pooled pipelined connections, per-receiver adaptive rate shaping driven by live SMTP reply codes, and proper backoff - this is what keeps IPs in good standing while maximizing throughput.
- Tracking is asynchronous and edge-served. Pixels and redirects return instantly and record fire-and-forget, so billions of engagement events never block a recipient’s mail client or the primary store, and engagement flows back as the signal warm-up and reputation depend on.
One-line summary: a thin REST/SMTP ingestion layer that durably queues messages after an authoritative suppression check, a DKIM-signing scheduler that routes each message onto a reputation-scored IP from a segmented pool, a destination-sharded MTA fleet that runs pooled pipelined SMTP conversations with adaptive per-receiver rate shaping and backoff, and a set of closed feedback loops (VERP bounces, FBL complaints, edge-tracked opens/clicks) that continuously update suppression and reputation so billions of transactional emails a day land in the inbox without one bad sender poisoning the pool.
Comments