Design a Vector Search Engine - System Design
Everyone builds vector search the same way the first time: load the embeddings into a list, and for each query compute the cosine similarity against every vector, then sort and take the top 10. It works on ten thousand vectors. Then the interviewer makes it the real problem: a billion 768-dimensional embeddings, a query vector has to come back with its approximate nearest neighbors in under 50ms, new vectors are upserted continuously and must be searchable within a second, the index is far too big to keep fully in RAM on one machine, and recall (how many of the true nearest neighbors you actually return) must stay above 95% even as you shard across memory-constrained nodes. ...