11b — System design

Sixty minutes in the second round. It only matters after you pass the first, so it is not urgent — but it is the interview where seniority is most visible, and the one where "I know the components" is worth the least.

Prereqs: 01, 05, 07 · Reading: parts 0–2 are 20 min and are the course; parts 3–11 are reference, consulted per case · Cards: 16 · Reference: latency, availability, estimation

The map

This interview is not a test of whether you can name technologies. It is a test of whether you can take a deliberately vague problem, close the scope yourself, put numbers on it, and defend the trade-offs those numbers force. The vagueness is the exam question — a candidate who starts drawing has already failed the first part of it.

The shape of the hour is fixed and you control it: scope, then scale, then API and data model, then a diagram, then one component in real depth, then what breaks. The depth section is where the score is decided, and depth means talking about failure, concurrency and numbers, not about adding more boxes.

Underneath the whole thing is one method, which Google publishes as NALSD: turn every requirement into a concrete resource number, then let the arithmetic kill your own design and force the next iteration. Doing that visibly — "here's the simplest thing, and here's the calculation that breaks it" — is the behaviour the rubric calls depth.

How to use this document. Parts 0, 1 and 2 are the course: the rubric, the framework and the arithmetic. Learn those. Parts 3 through 7 are reference — read once, then return by section when a case study demands it. Part 8 is where the learning happens, and the instruction there matters: close the document and draw before you read the answer.

What decides your score here

Closing the scope yourself, including the non-goals. Saying "in scope A, B and C; out of scope auth, billing and the admin UI — sound right?" is the cheapest seniority signal available, and most candidates never do it.

Numbers that change a decision. An estimate you compute and then ignore is wasted interview time. Every number should be followed by the sentence "and here's what that means for the design".

Depth over breadth in the deep dive. One component explained through its failure mode beats five components named. The three-part shape is: here is the mechanism, here is what goes wrong, here is the fix and what it costs.

Talking about production at all. SLOs, rollout, monitoring, cost. Almost nobody raises them unprompted, and doing so is the clearest available marker of a senior candidate.

Part 0 — How this interview is judged

0.1 What is actually being measured

Google's system design interview does not test whether you memorised architectures. It tests how you think when handed an ambiguous, open-ended problem at massive scale, which is why the question arrives deliberately vague.

The interviewer scores four things. Problem framing is whether you close the scope — users, use cases, constraints and non-goals — rather than assuming it. Architecture coherence is whether the API, data model, components and request path actually fit together, rather than being a set of boxes that individually make sense. Depth and trade-offs is whether you know where the design gets hard, and can say why you chose A over B. And production readiness is whether capacity, reliability, observability, security and cost appear at all.

The low-score version of each is easy to recognise: drawing at minute two, boxes that do not connect, "I'll use Kafka" with no justification, and a system with no way to fail.

0.2 What changes from mid-level to senior

This is the part to internalise, because you are being assessed against the second column of every pair.

A mid-level candidate answers the interviewer's questions; a senior one drives the elicitation and states the non-goals unprompted. A mid-level candidate estimates when asked; a senior one estimates unprompted, and the estimate visibly changes the design. A mid-level candidate produces a design that works; a senior one produces a design and names the alternative they rejected, with the reason.

On failure the gap widens. Mid-level mentions replication. Senior enumerates the failure modes per component and states the degraded behaviour for each — what the user sees when this dependency is gone. Under pressure, mid-level adjusts when corrected; senior anticipates: "this breaks at ten times the scale, here, and the fix would be…". And where mid-level stops at the diagram, senior talks about rollout, monitoring, cost and migration.

The last one is the least intuitive: mid-level candidates add components, senior candidates remove them and defend a minimal v1. Complexity is easy; judgement is what is scarce.

Expect a perturbation. Halfway through, the interviewer will inject something — a load spike, a dependency failure, a tighter SLO, a consistency conflict — and watch whether you adapt the architecture or freeze. Prepare for that actively: on every case in Part 8, ask yourself what happens if the scale doubles right now, or if the database dies.

0.3 The anti-patterns that fail you

Drawing before asking is the number one mistake and the cheapest to avoid. Buzzword-driven design — naming four technologies with no sentence of justification per piece — reads as memorisation and destroys the depth score. Disproportionate complexity, microservices for a hundred requests a second, is the inverse of the judgement being assessed.

Reciting CAP is worthless because everyone does it; apply it to the system at hand with the effect the user observes, or leave it out. Invented numbers are worse than rough ones: better to be wrong by a factor of two and show the arithmetic than to be right without being able to defend it.

Silence means the interviewer can neither assess you nor help you. Not finishing — forty minutes on requirements and no deep dive — is a fail on time management, and you own the clock. And over-defending wastes free points: when the interviewer points at a hole, "that's a good point, you're right, this breaks when X" is a strong answer, not a weak one.

0.4 NALSD, the method Google publishes

Google formalised its internal design method as Non-Abstract Large System Design, in chapter 12 of the SRE Workbook. The "non-abstract" is the whole point: every requirement becomes a concrete resource number — disk, IOPS, RAM, bandwidth, cores.

It runs on four questions, in order. Is it possible — is there a design that works, ignoring cost and limits? Can we do better — is there an algorithmically better approach, pre-computing instead of computing, or log-time instead of linear? Is it feasible — does it fit real hardware and a real budget, and this is where the arithmetic lands. And is it resilient — does it survive the loss of a machine, a rack, a datacenter, a region?

You iterate: answer the first two to get a basic design, then all four in the scaling phase, and return to the start whenever a calculation or a failure mode kills what you have. The book's own worked example goes through five iterations — one machine, then MapReduce, then a distributed joiner, then sharding, then multi-datacenter with Paxos — and each transition is forced by a number rather than by taste.

That is why this matters in an interview. Announcing "let me start with the simplest thing that could work, and then break it deliberately with the numbers" and then killing your own design with a calculation is exactly the behaviour being scored. You are demonstrating the reasoning, not presenting a result.

Part 1 — The 60-minute framework

1.1 The time budget

Roughly: three minutes framing, seven on functional requirements, six on scale, six on API and data model, sixteen on the high-level design, fourteen on the deep dive, six on bottlenecks and production, and two for your questions.

Two of those are non-negotiable. The deep dive never gets sacrificed, because it is where the score is; if you are running late, lock the high-level design and move on. And the last six minutes matter more than they look, because production readiness is what almost nobody reaches.

Announce the plan at minute three. "Here's how I'd like to use the time: about seven minutes on requirements and scope, five on scale estimates, then I'll draw the high-level design and we can deep dive wherever you find it most interesting. I'll keep an eye on the clock." That alone signals seniority, because you are driving rather than being led. And check the clock out loud once, at the halfway point.

1.2 Requirements — closing the scope in seven minutes

The goal is not to gather everything; it is to close a defensible scope quickly.

Five questions, in order of payoff. Who are the users and what is the single most important action they take anchors you to one use case and prevents a generic design. Is this read-heavy or write-heavy, and by what ratio is the highest-yield question in the whole interview: a hundred-to-one read ratio justifies aggressive caching, read replicas and denormalisation, and a one-to-one ratio kills all three. How fresh does the data need to be decides whether you can be eventual or must be strong, which changes the database. Global or single region decides whether you need geographic replication and conflict resolution. And what is out of scope — can I assume auth, billing and abuse are solved — is the cheapest seniority signal there is.

Then close it explicitly, out loud, and write it in the corner of the drawing:

"Let me lock the scope. In scope: a user requests an answer in their locale and gets it in under 300 ms p99; new source content reaches all locales within an hour; we serve 40 languages. Out of scope: authentication, the authoring UI, and billing. I'm also assuming machine translation with humans only sampling for quality, not in the request path. Is that the right shape of the problem?"

1.3 Scale — numbers that change decisions

Part 2 has the arithmetic. What matters here is what you produce and how you say it: average and peak QPS, storage per year including replication, outbound bandwidth, whether the working set fits in memory, the latency target, and the availability target with its error budget in minutes.

And then the sentence that makes the number worth computing. Always follow an estimate with the design conclusion it forces:

"That 150 ms transcontinental round trip is the whole story: if we want p99 under 300 ms for users in Brazil, we cannot serve from a US-only region. That forces either regional replicas or edge caching, and I'll pick between them when we get to the data model."

A number that changes no decision was wasted time, and the interviewer notices which kind you produced.

1.4 API and data model

Three to five concrete endpoints, with the details that show you think in contracts: a version prefix, cursor-based pagination rather than offsets — offsets duplicate and skip items under concurrent writes — and an idempotency key on every exposed write, because the network will retry.

GET  /v1/answers/{answer_id}?locale=pt-BR
     -> 200 {text, locale_served, fallback_used, source_version, model_version}

POST /v1/answers/{answer_id}/translations
     Idempotency-Key: <uuid>
     {target_locales: [...], priority: "interactive"|"batch"}   -> 202 {job_id}

Then the data model, and — this is the part that gets skipped — state the partition key and why:

"I'd partition by answer_id. Every read is 'give me this answer in this locale', so hashing on answer_id keeps a request on a single shard with no scatter-gather. The risk is a hot shard when one answer goes viral, and I'd handle that with a read-through cache in front, which absorbs the hot key without resharding."

Choosing the store follows from the access pattern rather than from preference. Relational when you need relations, transactions, or queries you have not thought of yet. Key-value or document when access is always by a known key and you need horizontal write scaling — accepting that you lose joins and ad-hoc queries. Wide-column when writes are massive and you scan by row-key prefix, accepting that the query dictates the data model. Search when you need full-text and ranking, remembering it is an index and not the source of truth. And a cache when the data is hot and reconstructible.

Never say "NoSQL because it scales". Say: "reads are always by a known key and I need horizontal write scaling, so a key-value store fits. I'm giving up ad-hoc queries, and I'm accepting that because the product only has these three access patterns."

1.5 The high-level design, and the test for it

Start with five to seven boxes, not twenty. Label the arrows with the protocol and the payload. Leave space on the right, because the deep dive will expand one box. Use colour for exactly two things — the read path and the write path — and thicker arrows for the hot path.

Then apply the test: narrate one request end to end. If you cannot do that without stumbling, the diagram has a hole in it.

"A user in São Paulo requests answer 42 in pt-BR. DNS with geo-routing sends them to the São Paulo edge. The CDN checks its cache keyed on answer, locale and version — a hit, which I'd expect for about ninety percent of traffic given the popularity skew, returns in fifteen milliseconds. On a miss it goes to the regional API server, which checks Redis, and on a Redis miss reads the source from the regional replica. Now there's a decision: if a translation exists for an older model version, serve it and refresh asynchronously; if none exists, return English immediately with a fallback flag. The user never waits on the model."

That narration contains the happy path, an estimated hit rate, both miss paths, and a defended product decision. That is what a 4 sounds like.

1.6 The deep dive

The interviewer usually picks the component. If they say "you choose", pick the one with the most design tension rather than the one you know best.

Depth means failure, concurrency and numbers — not more components. The shape is three parts:

The mechanism. "The cache is read-through, keyed on the source hash, the locale and the model version." What goes wrong. "When a popular entry expires, thousands of requests miss at the same instant and all hit the model. At 400 QPS on one key with 200 ms inference, that's 80 concurrent inferences for a single item." The fix and its price. "A per-key regeneration lock: the first miss computes, the others wait up to 50 ms and then serve the stale value. That trades a little staleness for bounded backend load. The alternative is early refresh with jitter, which avoids the wait but costs background work."

The classic targets and the tension in each: caching is about invalidation, stampede and hot keys; sharding is about the key choice and resharding without downtime; queues are about ordering, at-least-once and backpressure; consistency is about what the user sees during the window; storage is about B-trees versus LSM and write amplification; and model serving is about batching and the throughput-latency trade.

1.7 Bottlenecks and production

Four questions, answered before they are asked.

Where does this break first at ten times the scale? Name a component and the fix. What would I monitor? p50 and p99 per endpoint, error rate, queue depth and age, cache hit rate, saturation, and one business-level indicator. What would I cut from v1? This is judgement made visible — dropping the pre-computation pipeline until traffic justifies it says more than adding it did.

And what does it cost? Almost nobody raises this, and in any system with a model in it, it is where the real design pressure lives: "Inference dominates. At ten million translations a day at a tenth of a cent each, that's ten thousand dollars a day with no cache; a ninety percent hit rate takes it to one thousand. The cache isn't an optimisation here, it's the business model."

Part 2 — Non-abstract arithmetic

"The 'Non-Abstract' in NALSD refers to concrete resource estimation: disk I/O, network bandwidth, RAM, cores." — SRE Workbook, chapter 12

The numbers themselves live in the reference card, which is what you scan the night before. This part is the reasoning that makes them worth having.

2.1 The three estimates, and what each one is for

QPS comes from daily users times actions per day, divided by a day. Use 10⁵ seconds for a day and the arithmetic becomes a one-line division you can do out loud: a million daily users at ten actions each is ten million requests, so about a hundred per second on average, and two or three times that at peak.

What matters is the calibration that follows. A hundred requests a second fits on one machine, and saying so is a seniority signal rather than a naive one. Ten thousand is a serious system. A million is a fleet. Most candidates skip straight to distributed architecture without checking which of those three they are in.

Storage is events per day times bytes per event, times 365, times the replication factor, plus about thirty percent for indexes. The number usually turns out to be either trivially small or obviously enormous, and either conclusion changes the design immediately.

Bandwidth and cores are the two people forget. Bandwidth is QPS times response size, and a ten gigabit NIC gives you about 1.2 gigabytes a second. Cores is the calculation almost nobody does and it lands well: cores = QPS × CPU-seconds per request. Ten thousand requests a second at five milliseconds of CPU is fifty cores — then divide by a target utilisation of 0.6, which gives eighty-four, which is three machines. That is a complete capacity answer in two lines.

2.2 Letting the arithmetic kill the design

This is the part that distinguishes NALSD from estimation for its own sake. You compute all four resource dimensions — bandwidth, IOPS, memory and CPU — and the largest requirement is your fleet.

Google's own worked example does exactly this. Five hundred thousand queries a second at two kilobytes each is 86 terabytes a day, about a hundred with indexing. Divide by 64 gigabytes of RAM per machine and you need over fifteen hundred machines. Divide the query rate by 200 IOPS per disk and you need two and a half thousand disks. Those two lines are what killed the single-machine design and forced distribution.

Reproduce that movement in the interview. Not "I'll distribute because it's big" — which is an assumption — but "the IOPS arithmetic forces me to distribute", which is a derivation.

2.3 Availability, and why it composes badly

The numbers to know are that three nines is about 43 minutes of downtime a month and four nines is about four. What matters more is that availability in series multiplies: three components at 99.9% give 99.7%, which is worse than any of them individually.

The design consequence is direct. Every synchronous dependency on the critical path costs you availability, so the way to keep an SLO is to make dependencies optional, asynchronous, or degradable rather than to make each one more reliable.

"With four services in the synchronous path, each at three nines, the composed availability is 99.6% — worse than any individual component. That's why I'd make the quality scorer an asynchronous consumer instead of an inline call: it stops being able to take the serving path down."

Redundancy works the other way — two independent components at 99% in parallel give 99.99% — but independent is doing the work in that sentence. If both sit in the same rack, they are not.

2.4 Queues, utilisation, and why p99 explodes

Little's law — concurrency equals arrival rate times response time — is the most useful formula here. Two thousand requests a second at fifty milliseconds means a hundred requests in flight, which is the minimum size of your thread or connection pool. Size it at fifty and you have created a queue, and the latency doubles.

Then the relationship that explains capacity planning: waiting time grows as ρ/(1−ρ). At half utilisation the multiplier is one; at eighty percent it is four; at ninety it is nine; at ninety-five it is nineteen. That is why nobody runs a service at ninety percent CPU, and saying it turns a capacity number into a defended one:

"I'd size for 60% peak utilization, because beyond about 80% queueing delay grows non-linearly and p99 falls off a cliff — the headroom isn't waste, it's what buys the tail latency."

And the reason tail latency dominates: if a page makes ten parallel backend calls and each has a p99 of one second, the chance that at least one is slow is about ten percent. One component's tail becomes the composed system's typical case. That is the tail at scale argument, and the standard fix is hedged requests — fire a second copy once the first passes p95 and take whichever returns first, which costs about five percent extra traffic and cuts the tail sharply.

Part 3 — Distributed fundamentals

3.1 CAP, stated correctly

The popular version ("pick two of three") is misleading. The useful version is:

Network partitions happen — they are not a choice. During a partition, you choose: answer with possibly stale data (AP) or refuse to answer (CP).

Outside a partition you have both C and A. Which is why the refinement exists:

PACELC: if Partition → A or C; Else → Latency or Consistency. The second half matters most day to day: even with no failure at all, strong consistency costs latency — you wait for the quorum, or you wait out Spanner's commit-wait.

Never recite CAP. Apply it: "During a regional partition I'd keep serving reads from the stale regional replica and mark them stale: true in the response, but I'd reject writes in that region — reads being a minute old is invisible to the user, whereas a divergent write would create two conflicting translations for the same source version."

3.2 Consistency models, strongest to weakest

The three session guarantees in the middle are what save real products. The classic bug — "I posted a comment and it vanished" — is missing read-your-writes: the write went to the primary and the next read landed on a lagging replica. Cheap fix: after writing, read from the primary for N seconds, or carry a version token in the session and require a replica at or beyond that version.

A phrase that signals maturity: "I don't need global linearizability here — I need read-your-writes and monotonic reads for the session. Those are much cheaper: I can get them with a version token in the client's request instead of paying consensus on every read."

3.3 Quorums

With N replicas, W for writes and R for reads:

R + W > N   ⇒   every read sees the last acknowledged write
W > N/2     ⇒   there cannot be two conflicting acknowledged writes

3.4 Consensus: Paxos and Raft

The problem: getting N machines to agree on a value despite failures and an unreliable network.

Raft (the one you should be able to explain — it narrates far more easily than Paxos):

• Roles: leader, follower, candidate. Only the leader accepts writes.
• Term: a number that only increases; each term has at most one leader. This is how split-brain is prevented — an old leader that comes back discovers a higher term and steps down.
• Election: a follower with no heartbeat for an election timeout (randomised, ~150–300 ms, and the randomisation exists to avoid tied elections) becomes a candidate and requests votes. A majority elects.
• Replication: the leader appends the entry to its log, replicates it, and commits once a majority has acknowledged.
• Practical consequence: a write costs one round trip to the majority. Across datacenters, ~25 ms.

Numbers worth citing: in the SRE Workbook example, each cross-datacenter Paxos operation costs about 25 ms, which limits each process to roughly 40 sequential operations per second — and that calculation is what forced the design to 25,500 parallel tasks. Consensus does not scale with sequential throughput; it scales with sharding.

When NOT to use consensus: for data that tolerates conflict. Consensus is expensive. Use it for metadata, leadership, locks and configuration — not for the hot data path.

3.5 Time, clocks and TrueTime

Physical clocks drift. You cannot order events across machines by wall-clock timestamp. That breaks "last write wins" in quiet ways.

TrueTime, the mechanism — worth knowing, it is a point of pride in the house: instead of returning an instant, TT.now() returns an interval [earliest, latest] with uncertainty ε. On Google's infrastructure ε is typically under 4 ms, swinging from about 1 ms right after a sync to about 7 ms before the next. To commit, Spanner waits until TT.now().earliest > commit_timestamp — the famous ~7 ms commit-wait.

The idea in one sentence: "Spanner buys external consistency by literally waiting out the clock uncertainty — about 7 ms per write transaction. It's not free; it's a latency tax paid in exchange for being able to reason about a global order."

3.6 Replication

Replication lag is visible to the user — always name the effect: "With async replicas the user can write a comment and not see it on reload, because their read landed on a replica that's 200 ms behind. I'd fix that with read-your-writes routing rather than by making replication synchronous, which would cost every write."

CRDTs in one sentence, for when multi-primary comes up: structures whose merge operation is commutative, associative and idempotent, so replicas converge without coordination (G-Counter and PN-Counter counters, OR-Sets, collaborative text). The price: metadata grows, and not every business rule is expressible that way.

3.7 Partitioning (sharding)

Consistent hashing

The problem it solves: with hash(k) mod N, adding one machine remaps almost every key. With consistent hashing it remaps about 1/N.

How it works: map both nodes and keys onto a ring from 0 to 2³²−1; each key belongs to the first node clockwise. A node joining or leaving affects only the neighbouring arc.

Virtual nodes (the detail that separates having read about it from understanding it): with few nodes the ring is unbalanced. Place each physical node at ~100–200 points on the ring. That evens out the load and spreads a failed node's load across all the others instead of dumping it on the neighbour.

Hot shards and hot keys are a different problem that sharding does not solve: a viral key concentrates traffic on one partition regardless of the algorithm. Fixes: a cache in front, replicating the hot key into N copies (key#1..key#N, picked at random on read), or request coalescing.

Resharding without downtime, if they ask for the deep dive: dual writes to both schemes → background copy of the history → consistency verification by sampling → switch reads → remove the old scheme. Each step is reversible, and that reversibility is what the interviewer wants to hear.

3.8 Distributed transactions

2PC (two-phase commit): the coordinator asks "can you commit?", everyone answers, the coordinator decides. The problem: it blocks — if the coordinator dies after the prepare, the participants hold their locks indefinitely. That is why pure 2PC is rare at scale.

Saga: break the transaction into local steps, each with a compensation. If step 3 fails, run the compensations for 2 and 1. The price: no isolation — intermediate states are visible. It is the real pattern in microservices.

Transactional outbox — the trick that solves 80% of cases and is worth a lot in the interview. The problem is "write to the database and publish to the queue atomically". The solution: in a single local transaction, write both the business row and a row in an outbox table. A separate process reads the outbox and publishes. You trade distributed atomicity for at-least-once plus idempotency, which is always cheaper.

3.9 Failure detection

• Heartbeat plus timeout is the base. The dilemma: a short timeout gives false positives (you kill a healthy node and make the overload worse); a long one detects slowly.
• Phi accrual failure detector: instead of a boolean, it produces a suspicion level based on the historical distribution of heartbeat intervals, letting each service pick its threshold. Used in Cassandra and Akka.
• Fencing tokens — the detail that impresses: a node presumed dead may simply have been paused (a long GC) and can come back believing it still holds the lock. The fix: the lock service issues a monotonically increasing number, and the storage layer rejects writes carrying a token lower than the last one it saw. Without this, every distributed lock has a latent race.
• Split brain: two halves each believe they lead. Prevention: majority quorum and monotonic terms (Raft). A minority cannot elect a leader.

Part 4 — Building blocks

4.1 The edge: DNS, anycast, GSLB

• DNS geo or latency routing sends the user to the nearest POP. A low TTL gives failover agility; a high one reduces load. Careful: resolvers ignore short TTLs, so DNS is not a fast, reliable failover mechanism.
• Anycast: the same IP announced from many places, with BGP delivering to the nearest. Failover in seconds, with no dependence on DNS caching. It is how public DNS and CDNs work.
• GSLB (global load balancing): picks the region by proximity and by capacity and health. The design point: nearest is not always right — if the region is saturated, spilling to the neighbour costs 40 ms and saves the SLO.

4.2 Load balancing

Policies: round-robin (simple, poor with heterogeneous requests), least-connections (a good real default), least-loaded with backend feedback (better, needs a signal), consistent hashing (keeps a client or key on the same server, which makes local caching effective), and power of two choices (sample two servers, send to the less loaded — nearly as good as the global optimum, at O(1) cost and with no central state; an elegant detail to cite).

Maglev, Google's L4 balancer, is the in-house reference: software on commodity machines, using consistent hashing so that any balancer machine can receive any packet and still send it to the same backend — which is what lets the balancer itself scale horizontally and survive machine loss without breaking connections.

Production details that score: active health checks (and the care not to take down the whole fleet when the health check is expensive), connection draining on deploy (stop sending new traffic, let existing traffic finish), and sticky sessions are debt — they break autoscaling and concentrate load; prefer session state in Redis.

4.3 Caching — the highest-yield component in the interview

Where to cache (cheapest to most expensive)

browser → CDN/edge → LB → in-process cache → distributed cache (Redis) → database

Each tier cuts both latency and the cost of the next tier. Say which you use and why. An in-process cache has nanosecond latency and no network, but every instance holds a copy (invalidation is hard, memory multiplies); a distributed cache costs a 0.5 ms round trip but is coherent and shared.

Patterns

Eviction

LRU (the default), LFU (better with a stable long tail), W-TinyLFU (the practical state of the art: a frequency sketch decides whether a candidate deserves admission, which stops a sequential scan from wiping the cache — it is Caffeine's algorithm). Citing it is a cheap differentiator.

The four cache problems the interviewer wants to hear

1. Cache stampede / thundering herd. A hot key expires and 10,000 requests hit the database at once. Fixes: a per-key regeneration lock (only one recomputes; the rest wait or serve the stale value), TTL with jitter (ttl × (1 + rand(±10%)), which prevents synchronised expiry), probabilistic early refresh.
2. Cache penetration. Requests for keys that do not exist pass straight through and hammer the database (it is an attack vector). Fix: cache the negative with a short TTL, or put a bloom filter of existing keys in front.
3. Hot key. One key concentrates more traffic than a node can take. Fix: replicate the key across N suffixes, an in-process cache in front, or request coalescing.
4. Invalidation. "There are only two hard things…" — but the mature answer is: prefer versioned keys to explicit invalidation. If the key is answer:42:v7:pt-BR, bumping the source version makes the old key unreachable and it dies of TTL on its own. You have traded a hard distributed problem for an integer.

The metric that matters is hit rate. Below about 80% the cache costs more than it returns (you pay miss latency plus memory). State the hit rate you expect and where you got it — usually from the popularity distribution (Zipf: the top 20% typically serve 80% of the traffic).

4.4 CDN and the edge

Solves the 150 ms. Serves static assets and cacheable responses — which is the case for translations. Design points: the cache key (include the locale), Cache-Control versus stale-while-revalidate (serve the old one while refreshing — excellent for perceived latency), invalidation by purge (slow and expensive, avoid) versus by version in the URL (instant and cheap), and an origin shield (an intermediate tier that stops 200 POPs from missing to the origin at once).

4.5 Storage: how databases actually store things

A WAL (write-ahead log) exists in both: write the intent to a sequential log and fsync before touching the data. It is how durability is achieved without paying for synchronous random writes.

Indexes: each index speeds up reads and slows down writes and occupies space. A covering index answers a query without touching the table. Always say which index supports which query.

Isolation (SQL levels): read uncommitted → read committed (Postgres's default) → repeatable read → serializable. The phenomena: dirty read, non-repeatable read, phantom read, and write skew (the one snapshot isolation does not prevent — two doctors both go off call because each transaction saw the other present). MVCC: every read sees a snapshot; writers do not block readers. It is why Postgres needs VACUUM.

Erasure coding vs replication (for blobs and files): 3× replication costs 200% overhead and recovers by reading one copy; Reed-Solomon (6 data plus 3 parity) costs 50% and recovers by reading 6 blocks — far cheaper in space, more expensive in network during reconstruction. Rule of thumb: replicas for hot data, erasure coding for cold. Colossus does exactly this, migrating data between flash and disk as it cools.

4.6 Queues and streams

Why they exist: to decouple producer from consumer, absorb peaks (the queue is a buffer), enable retries, and smooth out expensive work (model inference, for instance).

Delivery semantics — the answer that separates a senior:

• At-most-once: acknowledge before processing. Loses messages. Almost never what you want.
• At-least-once: acknowledge after processing. The real default. It duplicates.
• Exactly-once: impossible end to end with participants that can fail (it is the two generals problem). What is done in practice is at-least-once plus an idempotent consumer, which is observationally equivalent. Kafka offers "exactly once" inside its own transactional boundary, not all the way into your external database.

A strong phrase: "I'd design for at-least-once and make the handler idempotent, keyed on job_id, with the dedup table having a TTL. Chasing true exactly-once across the queue and the database means distributed transactions, and I'd rather pay for an idempotency key than for 2PC."

Production details: a DLQ (dead-letter queue) after N attempts, with an alert — a DLQ without an alert is a graveyard; a visibility timeout larger than the worst-case processing time; backpressure (if the consumer cannot keep up, stop accepting production rather than exhausting memory); and the most predictive metric there is, the age of the oldest message in the queue — better than depth, because it does not depend on throughput.

4.7 Processing: batch and stream

• MapReduce and batch: high throughput, latency measured in minutes to hours. Good for recomputing everything.
• Streaming: latency in seconds. It brings the problem of time.
• Event time vs processing time: the event happened at 10:00 and arrived at 10:07. If you aggregate by processing time, your 10:00 window is wrong.
• Watermark: the assertion "I expect no more events earlier than T". It closes the window. The trade-off is explicit: an aggressive watermark gives a fast, incomplete result; a conservative one gives a correct, slow one. Dataflow and Beam, which came out of Google, are exactly the formalisation of this (the what / where / when / how model).
• Lambda vs Kappa: lambda keeps a batch path and a streaming path (correct and expensive, two codebases to maintain); kappa keeps only the streaming path and reprocesses by re-reading the log when it needs to correct. Kappa is the modern answer, made viable by retentive logs.

4.8 Search and the inverted index

Inverted index: term → posting list (sorted doc_ids). The query "A AND B" is an intersection of sorted lists. Delta compression plus varints shrink the index to a fraction of the corpus.

How to shard the index — a classic deep-dive question:

Two-phase ranking: cheap retrieval of thousands of candidates, then expensive re-ranking (a model) of the top ~100. It is the universal pattern, and it applies to search, recommendation and RAG.

Autocomplete: a trie with the top-k pre-computed at every node (otherwise you scan the subtree on every keystroke), served from RAM, updated in batch. The latency target is ~50 ms because it fires on every keystroke.

4.9 Probabilistic structures (trading exactness for memory)

The bloom filter calculation: with about 10 bits per element and 7 hash functions, the false positive rate is around 1%. For one billion URLs: ~1.2 GB. Saying that — rather than "I'd use a bloom filter" — is the difference between knowing the name and knowing the tool.

4.10 Rate limiting

Distributed: a counter in Redis with INCR plus EXPIRE, or a Lua script to make read-modify-write atomic. The real trade-off: exact coordination on every request costs one round trip; the alternative is local quotas (each instance gets a slice of the limit and syncs once a second), which is approximate but removes Redis from the hot path. Say which you choose and what error you accept.

4.11 ID generation

The point worth making: time-ordered IDs give write locality in the index (inserts at the end of the B-tree) — but that creates a hot shard if you shard by ID. Both sides exist; name the one you choose.

4.12 Geospatial

Geohash (a string from grid subdivision — a common prefix means proximity, but with discontinuities at boundaries), quadtree (adaptive subdivision by density), S2 (Google's library: it projects the sphere onto a cube and uses a Hilbert curve, which preserves locality far better and avoids polar distortion). If the problem contains "nearby", cite S2 — it is literally the in-house technology.

4.13 Observability

The three pillars and what each answers:

SLI / SLO / error budget, Google's vocabulary:

• SLI is the measurement (e.g. the fraction of requests under 300 ms).
• SLO is the target (99.9% over a 28-day window).
• Error budget is 1 − SLO. It is a risk budget: if there is budget left, ship features; if it is spent, freeze launches and spend on reliability. That turns "how reliable?" from an opinion into a decision with a number.

Alert on symptoms, not causes: alert on the user-facing SLI (latency, errors), not on "CPU at 90%". And use burn rate — "we're burning the error budget 14× faster than allowed" — rather than a fixed threshold, which generates noise.

4.14 Security and multi-tenancy (the minimum expected)

• AuthN vs AuthZ: who you are versus what you may do. Short-lived tokens (JWT, OAuth) with refresh; revocation is JWT's real problem — solve it with a short TTL or a revocation list.
• mTLS between services and least privilege. Zero trust: the internal network is not trust.
• Encryption in transit (TLS) and at rest (managed keys, rotation).
• Tenant isolation: tenant_id in every key, per-tenant quotas (which prevent noisy neighbours), and the database-per-tenant versus shared-database decision (isolation versus cost per tenant).
• Privacy: data minimisation, retention with a deadline, and the right to erasure — which is an architectural requirement, since deleting from backups and derived indexes is hard. Design for crypto-shredding: throw away the key rather than hunting the data.

Part 5 — Production reliability

This is the part that most separates senior from mid, and the one nearly every candidate skips. A beautiful design with no answer to "and what happens when this service gets slow?" scores a 2.

5.1 Cascading failure

The most important failure mode in large systems, and the most misunderstood.

The mechanics: one service gets slow → its clients accumulate in-flight requests (Little's law: concurrency = rate × latency) → threads and connections exhaust → the clients get slow → their clients accumulate → the whole system falls over. And worse: restarting does not fix it, because on the way back all the pent-up demand arrives at once and knocks it over again.

The accelerants:

• Retries. Load rises precisely when the system is suffering. A chain of three services, each with three attempts, multiplies the load by 27.
• Thundering herd. Every client reconnects, or every cache entry expires, at the same instant.
• Unbounded queues. The queue grows, latency grows with it, and you process requests whose client already gave up — 100% wasted work at the worst possible moment.

The defences, in order of importance:

1. Deadlines with propagation. Each request carries the time remaining; each hop subtracts what it spent. If the remainder is gone, fail immediately without calling the backend. This alone eliminates the entire class of "work for nobody".
2. Short, bounded queues. Prefer rejecting fast to queueing long. A queue is latency.
3. Load shedding by criticality (5.2).
4. A retry budget (5.3).
5. Circuit breaker: after N failures, open the circuit and fail fast; let one probe through periodically to close it. It stops you hammering a dead service.
6. Bulkhead: separate pools per dependency, so one dependency's slowness cannot consume every thread and kill the others.

5.2 Load shedding and criticality — how Google does it

When demand exceeds capacity, you will reject something. The choice is whether you reject at random and suffer, or reject deliberately and degrade well.

Criticality (Google's four classes), automatically propagated down the RPC chain:

Under overload, the backend drops from the lowest class upward. Propagating criticality matters: a sub-request spawned by a batch job should inherit SHEDDABLE.

The saturation signal: Google does not measure QPS for this (requests have different costs) — it measures resources: CPU rate against reserved capacity, and an executor load average (an exponentially smoothed count of active threads) that trips when active threads exceed available processors. That ignores short spikes and catches sustained overload.

Client-side adaptive throttling — the most elegant idea in this section. The problem: even rejecting costs resources; if the backend spends everything saying "no", nobody is served. The solution: the client counts requests (attempted) and accepts (accepted by the backend) over a window, and drops locally with probability:

P(drop) = max(0, (requests − K × accepts) / (requests + 1))

With K = 2, the client starts self-limiting once the backend is rejecting about 50%. A smaller K (1.1) is more aggressive, which is useful when rejecting costs nearly as much as processing.

A phrase worth a 4: "I'd put adaptive throttling in the client library rather than only rate limiting at the server, because at real overload the rejections themselves become the load. The client tracking its own accept ratio and shedding locally is what keeps the backend from spending its entire capacity saying no."

5.3 Retries — which are almost always done wrong

Rules Google applies, and you should cite:

1. Per-request limit: at most 3 attempts.
2. Per-client budget: retries capped at 10% of total requests. Once it is spent, stop trying — the system is in trouble, and insisting only makes it worse.
3. Never retry at stacked layers. Pick one level of the stack (usually the highest, which knows the intent). Retrying at three layers is a factor of 27.
4. Exponential backoff with jitter, always:

   wait = random(0, min(cap, base × 2^attempt))     ← "full jitter"

   Without jitter, every client comes back synchronised and creates a second spike.
5. Only retry what is safe — idempotent, or carrying an idempotency key.
6. Propagate the signal: an overloaded backend should answer "overloaded; don't retry" and the client should obey. The backend inspects the retry-attempt histogram: if most of its traffic is already retries, it stops inviting more.

5.4 Graceful degradation

The system must have a defined behaviour for "I cannot do the ideal thing right now". Always name the degraded mode of each dependency — it is one of the most likely deep-dive questions.

The principle: a worse answer, delivered, almost always beats an error. But state the limit — there are cases (balance, permission, security) where the correct move is to fail closed. Knowing which is which is senior judgement.

5.5 Rollout, and how not to take everything down at once

Most incidents come from changes. The sequence you should be able to recite:

1. Feature flag — separate deploy from release. Turning it on needs no new binary; turning it off is instant.
2. Canary — 1% of traffic, comparing SLIs against the control, with automatic rollback on a threshold. A canary with no pre-agreed abort criterion is just a slow deploy.
3. Progressive by failure domain — 1% → 10% → one zone → one region → global, with bake time between steps (memory and cache bugs only appear after hours).
4. Shadow traffic (dark launch) — send a copy of real traffic to the new system, discard the response, and compare. It is how you validate a model or a rewrite with no user risk.
5. Rollback is a design requirement: a schema migration must be backward compatible (expand → migrate → contract), otherwise you cannot go back.

5.6 Capacity and multi-region

• N+2: capacity to lose one unit to failure and another to maintenance, simultaneously.
• Draining a region is the real test. Google does this routinely (DiRT). If the system only survives when nobody touches it, it is not reliable — it is lucky.
• Active–active (every region serves): better latency and trivial failover, but you must handle concurrent writes across regions. Active–passive: simple and consistent, but the idle capacity costs money and the failover is a procedure that must be tested.
• RTO (how long until you are back) and RPO (how much data you accept losing). State both with numbers: "RPO of 5 minutes because replication is async with a 5-minute lag bound; RTO of 10 minutes because failover is automated but promotion requires quorum."
• Blast radius: design so that one failure hits one cell, not the fleet. Cell-based architecture is the modern answer: each cell is a complete, independent copy of the system serving a slice of the users.

Part 6 — Google's own stack

You do not need to know this deeply. You need to speak the language of the house: when you say "a wide-column store", being able to add "— Bigtable, in your case". It shows you studied the company, and it gives the interviewer a shared vocabulary.

How to use this without sounding like a flashcard: only cite it when it solves something concrete in your design.

✅ "For the translation store, the access pattern is a scan over answer_id prefix to get all locales, so a wide-column store fits — Bigtable internally — with row key answer_id#locale#model_version. That gives me all locales of one answer in a single row range."

❌ "I'd use Spanner and Bigtable and Borg."

Part 7 — AI and ML system design

Why this part exists: in 2026 Google began including GenAI and LLM system design in the loop, including for non-ML roles. And your role is Localization on AI Answers. The chance the problem has a model in it is high. Reported questions include "Design an LLM-based Q&A system", "Design a RAG system over private data" and "Design a language detection system".

7.1 How an LLM actually serves

Two phases, with opposite resource profiles — knowing this separates those who have used one from those who have read about one:

total latency ≈ TTFT + (output_tokens − 1) × TPOT
e.g. 300 ms + 299 × 20 ms ≈ 6.3 s for 300 tokens.

The design consequence, which is the entire point: you do not put long generation on the synchronous path of a web request. Either you stream (the user sees the first token in 300 ms and reads while it generates — perception is dominated by TTFT), or you go asynchronous with a queue, or you serve from cache.

KV cache: it stores the attention keys and values of previous tokens so they are not recomputed. It grows linearly with sequence length and, with long context, occupies more memory than the model weights themselves. It is the scarce resource in serving, and it is why "how many concurrent users fit on one GPU" is a memory question, not a FLOPs question.

Continuous batching (what vLLM popularised): rather than waiting for the whole batch to finish, as soon as one request in the batch completes, another from the queue takes its slot. It eliminates head-of-line blocking and greatly raises throughput without penalising short requests. PagedAttention treats the KV cache as paged memory, ending fragmentation and allowing larger batches.

Prefix caching / RadixAttention: if every request starts with the same system prompt, the prefix is computed once and reused. In a translation pipeline with a fixed instruction, that is a direct and substantial saving.

The central serving trade-off: a larger batch means higher throughput and worse individual latency. You choose by product. Say it explicitly:

"I'd run two pools with different batching policies: an interactive pool with small batches and a tight deadline for the on-demand path, and a batch pool with large batches and high queue tolerance for pre-computation. Same model, different SLOs — mixing them means the batch work destroys interactive tail latency."

7.2 RAG — retrieval-augmented generation

Ingestion pipeline (offline):

documents → chunking → embeddings → vector index (+ inverted index)

• Chunking is underrated: large chunks dilute similarity, small ones lose context. 200–500 tokens with overlap is the starting point. Keep metadata (source, language, version) — that is what lets you filter and cite.
• Embeddings: 768–1536 dimensions. Memory cost: 10⁸ chunks × 768 dims × 4 B = 300 GB. Quantising to int8 brings it to 75 GB. That calculation changes the architecture, so do it out loud.

Query pipeline (online):

query → embedding → ANN (top ~100) → rerank (top ~10) → assemble prompt → LLM → answer plus citations

• ANN: exact search is O(N); use approximate. HNSW (a navigable graph, excellent recall and latency, high memory), IVF-PQ (quantised, low memory, lower recall). The knob you turn is recall versus latency, and the design question is "how much recall do I actually need?"
• Hybrid search: vector plus BM25 (lexical). Vector search fails on proper nouns, codes and acronyms; lexical search fails on paraphrase. Combining them (reciprocal rank fusion) is the practical standard.
• Two-phase reranking — the same pattern as classic search: retrieve cheaply, rerank expensively.
• Grounding and citations: force the model to cite the chunk. Without that there is no way to measure hallucination or to debug.

RAG-specific failure modes, which are worth a lot in a deep dive: empty retrieval (do not invent — answer "I don't know"), conflicting documents, a stale document (version them and prefer the recent), prompt injection arriving through retrieved content (treat the document as data, never as instructions — isolate the role in the prompt), and cross-tenant leakage (the permission filter must be applied in retrieval, not afterwards).

7.3 Evaluation and model rollout

You cannot canary a model by watching latency and errors alone — the regression is one of quality.

Rolling out a new model: shadow → offline comparison per language → canary by language (never globally at once: regressions are usually language-specific) → progressive → full, with rollback driven by a quality metric, not just latency.

The cache key must contain the model version. Without it you serve an old model's output forever and have no way to roll out or roll back. This is one of the most valuable details in this whole document:

cache_key = hash(source_text) + locale + model_version + prompt_version

Cost: inference dominates. Do the arithmetic and turn it into an architectural decision:

10⁷ translations/day × $0.001 = $10,000/day with no cache
at a 90% hit rate             → $1,000/day

"At this volume the cache isn't a performance optimization — it's the difference between a viable and an unviable unit economics. I'd design the cache key first and the serving path second."

7.4 Language detection (a real Google question)

Worth having ready, because it is literally this role's domain:

• The model: character n-grams plus a lightweight classifier (fastText-style) solves it in about 1 ms of CPU with no GPU. Do not use an LLM for this — saying so is a judgement point.
• The hard problem is not the model, it is short text. "OK", "haha", proper nouns: confidence collapses. Return a probability distribution plus a confidence, not a label, and let the caller choose the threshold.
• Signals beyond the text: Accept-Language, the account locale, geo-IP, the user's history. Combine them — the product decision is "which language do I serve", which is broader than "what language is this text".
• Hard cases worth naming: code-switching (a sentence in two languages), variants (pt-BR vs pt-PT, zh-Hans vs zh-Hant), transliteration (Hindi written in Latin script), and scripts shared across many languages.
• Serving: a small model embedded in the service process (no RPC), p99 under 5 ms, updated with the binary. A cache makes no sense — the model is cheaper than the cache lookup.

Part 8 — Case studies

How to train with this part: read only "the prompt" and the "before you read" note for each case, close the document, and draw for 45 minutes with a timer. Then compare. Reading the solution straight through produces the illusion of competence and none of the competence.

8.1 URL shortener — the case that teaches the method

The prompt: "Design a URL shortener like TinyURL."

Before you read: what are the three scoping questions you would ask?

Requirements

Functional: shorten a long URL; redirect; optionally custom aliases, expiry, analytics. Non-goals declared: authentication, billing, UI.

Scale, and the calculation that decides everything

100M URLs created/day       → 100M / 10⁵ s = ~1,000 QPS of writes
read:write ratio of 100:1   → 100,000 QPS of reads, peak 300,000
storage: 100M/day × 500 B   = 50 GB/day = 18 TB/year

The conclusion: 100,000 QPS of reads against 1,000 of writes. This is a cacheable read problem, not a database problem. The whole design follows from that.

The short key — the real design point

62⁷ ≈ 3.5 trillion — enough for roughly 100 years at 100M/day. Do that arithmetic in front of them.

Design

        ┌──────────┐   miss   ┌───────────┐
GET /xY │ CDN/edge │ ───────► │ API (L7)  │
        └──────────┘          └─────┬─────┘
                                    │ get(xY)
                              ┌─────▼─────┐  miss  ┌──────────────┐
                              │  Redis    │ ─────► │ KV store     │
                              │ (LRU)     │        │ shard by key │
                              └───────────┘        └──────────────┘
POST /shorten ──► API ──► id block (Spanner) ──► KV store ──► invalidate cache

Deep dives they will pull on

• 301 vs 302: 301 (permanent) is cached by the browser — it saves traffic, but you lose the analytics and cannot change the destination. 302 keeps control. It is a product decision, and saying so is the point.
• Hot key: a viral link. The CDN absorbs it; without one, an in-process cache with a short TTL.
• Custom alias: needs uniqueness ⇒ a conditional write (compare-and-set), not "check then write".
• Expiry: a TTL in the KV store is cheaper than a sweeper job.
• Abuse: rate limit per IP or account, malicious-URL checking (asynchronous, with later blocking).

Where it breaks at 10×

At 1M QPS of reads, the CDN is the system. The question becomes hit rate and invalidation, not the database.

8.2 Distributed rate limiter

The prompt: "Design a rate limiter." (a confirmed Google question)

Scope first

• Limit on what? IP, API key, user, endpoint? (different answers)
• Where does it live: a client library, a sidecar, the gateway, or a central service? (different cost)
• Does it have to be exact? ← the question that decides the design. If 5% error is acceptable, the design gets ten times cheaper.

Algorithm

Token bucket as the default: a capacity (the burst allowed) and a refill rate. You store only (tokens, last_refill_ts) and refill lazily on read — no timers.

now = now()
tokens = min(cap, tokens + (now − last) × rate)
if tokens >= 1: tokens -= 1; allow
else: reject, Retry-After = (1 − tokens)/rate

Fixed windows have a classic bug you should mention: with a limit of 100 per minute, a client can send 100 at 00:59 and 100 at 01:00 — 200 requests in two seconds. A sliding window counter interpolates the two windows and fixes it with two counters.

Distributed — the heart of the question

The mature answer: "I'd give each of the N gateway instances limit/N locally and sync counters every second. That's approximate — a client hitting one hot instance can exceed its share briefly — but it removes Redis from the hot path entirely. For the small set of heavy abusers, I'd promote them to centrally-tracked exact limits. You pay exactness only where it matters."

Production

• Fail open or fail closed? If Redis dies, do you block everyone or allow everyone? For abuse protection, fail open (availability first). For a paid quota, fail closed. Name the choice — it is exactly the kind of judgement being assessed.
• X-RateLimit-Remaining and Retry-After headers (with jitter, or everyone comes back together).
• Rejecting must be cheap — expensive rejection is how overload becomes collapse (Part 5).

8.3 Web crawler

The prompt: "Design a web crawler."

Scale

1 billion pages in 30 days → 10⁹ / (30 × 10⁵ s) ≈ 400 pages/s
100 KB/page → 40 MB/s of download = 320 Mbps
raw storage: 10⁹ × 100 KB = 100 TB (compressed ~20 TB)

Components

seeds → [URL frontier] → fetchers → parser → link extractor → dedup → frontier
                                       │
                                       └→ storage (blob) → indexer

The real problems (this is where the score is)

1. Politeness. You cannot hammer a host. The frontier needs per-host queues with a scheduler respecting robots.txt and a minimum interval. This requirement shapes the whole data structure, and most candidates forget it.
2. URL dedup. 10⁹ URLs in memory is not viable. A bloom filter: about 10 bits per element gives 1.2 GB at a 1% false positive rate (you lose 1% of pages — acceptable). Normalise the URL first (lowercase host, strip the fragment, sort query params).
3. Content dedup. Different URLs with identical content: SimHash for near-duplicates.
4. Spider traps. Infinite calendars, dynamically generated URLs. Defences: a depth limit, a per-domain page limit, repetitive-pattern detection.
5. Prioritisation. Approximate PageRank, freshness, historical change rate. A crawler that treats everything equally wastes most of its bandwidth.
6. Recrawl. Adaptive frequency: a page that changes hourly versus a static museum page.

The likely deep dive

"How do you parallelise across 1,000 machines without violating politeness?" → shard the frontier by hash of the host, so every URL for one host always lands on the same machine, which serialises the access. Elegant, and it shows you understood that the constraint is per host, not global.

8.4 Distributed LRU cache

The prompt: "Design a distributed LRU cache." (a confirmed Google question)

This case is good because it has a data-structure half and a systems half — and you are assessed on both.

Single node (the DS&A half)

HashMap<key, node> plus a doubly linked list. Get: find in the map in O(1), move the node to the head in O(1). Put: insert at the head, evict the tail if over capacity. Every operation is O(1). In an interview you should be able to code this in ten minutes.

Concurrency on one node: a global lock serialises everything. Better: internal sharding (16–256 segments each with its own lock, like ConcurrentHashMap), or an approximate LRU (sample K keys and evict the oldest — which is what Redis does, and it is cheaper than maintaining an exact list under concurrency).

Distributed (the systems half)

• Routing: consistent hashing with ~150 virtual nodes per physical node. The client computes the node — no proxy in the path, saving a hop.
• Adding or removing a node remaps 1/N of the keys; the remapped ones become misses, which produces a load spike on the database. Mitigation: warm the new node before giving it traffic, or ramp it in gradually.
• Is the LRU global or per node? Per node, and that is correct: a global LRU would need coordination on every get and would destroy the point. Say that — it is the answer that shows you understood the cost.
• Replication: a cache is reconstructible, so replication is optional. But without it, one node's death produces a miss storm of 1/N of the traffic. A cost decision, with a number.
• Consistency with the database: versioned keys beat broadcast invalidation.

Failures

A dead node is detected by health check; requests go to the next node on the ring (which misses and populates). Protect the database with a concurrency limit on repopulation — otherwise a cache node's death takes down the database, which is the classic cascading failure.

8.5 Metrics and logging service

The prompt: "Design a metrics and logging service." (a confirmed Google question)

The central point: metrics and logs are different systems

Recognising that these are two systems sharing one ingestion path is already half the score.

Scale

10,000 machines × 1,000 series × 1 point/10 s = 10⁶ points/s
a point = (ts 8B + value 8B) → compressed with Gorilla, ~1.4 B/point
10⁶ × 1.4 B × 86,400 = ~120 GB/day after compression

Citing Gorilla (delta-of-delta on the timestamp plus XOR on the float, roughly 12× reduction) is a high-density technical detail.

Design

apps → local agent (aggregates, samples, buffers to disk) → regional collector
     → TSDB (hot retention in RAM, cold in blob)  → query and alerting
     → log pipeline → index plus cold blob

Points to defend:

• Aggregation at the edge (in the agent) cuts volume by orders of magnitude. Push versus pull: pull (Prometheus) discovers targets and detects death naturally; push works better for ephemeral jobs. Monarch is push, regional, with global federation for queries.
• Downsampling by age: 10 s for 7 days, 1 min for 30 days, 1 h for 2 years.
• Cardinality is the mortal enemy. A label carrying user_id creates millions of series and kills the system. Impose a limit and reject (with an alert to the series owner).
• The monitoring system must not depend on what it monitors. Circularity is how you go blind precisely during the incident. Say that.
• Loss is acceptable in metrics, not in audit logs. Two different SLOs ⇒ two paths with different criticality.

8.6 Search autocomplete

The prompt: "Design search autocomplete."

Why it is hard

It fires on every keystroke. The p99 target is ~50 ms including network. That means: no database in the path, no inference in the path, everything in RAM.

Scale

10⁹ searches/day, 20 characters on average → up to 2 × 10¹⁰ suggestion requests/day
                                            → ~200,000 QPS (far less with client-side debouncing)

The first design decision, and it lives in the client: a ~100 ms debounce and cancellation of the previous request. That cuts 60–80% of traffic before it touches the server. Mentioning client-side optimisation is rare among candidates and worth a point.

Structure

A trie with the top-k pre-computed at every node. Without that, every prefix requires scanning the subtree. With it, the answer is a pointer read.

memory: 10⁷ phrases × 20 chars, with top-5 (4B id + 4B score) per node ≈ tens of GB
⇒ shard the trie by prefix (a-f, g-m, …) and replicate for throughput

Updating

The trie is read at 200k QPS and written rarely ⇒ build it offline. Pipeline: search logs → aggregation by window (hour or day) → filtering for spam, PII and prohibited content → trie construction → atomic publication of a new version (swap a pointer; never mutate the structure being served).

That pattern — an immutable structure rebuilt in batch and published by version — is one of the most useful in systems design. It applies to tries, indexes, models and translation tables alike.

Deep dives

• Personalisation versus cacheability: personalising kills the shared cache. Middle ground: a cached global result plus lightweight client-side reordering using local history.
• Freshness: "a term that exploded in the last ten minutes" does not fit the daily batch. A separate trending layer with a short window, merged at read time.
• Multilingual (relevant to this role): the trie is per locale; Unicode normalisation (NFC), case folding, diacritics ("sao" should find "são"), and non-Latin scripts requiring transliteration or IME input. This is literally the job.
• Spelling: edit distance over the trie (fuzzy search) or a bigram index.

8.7 News feed / timeline

The prompt: "Design a news feed."

The case exists to test one trade-off, and you should go straight at it.

The correct answer is hybrid, and the criterion must be stated: fan-out on write for the majority; for accounts above a threshold (say 100k followers), do not fan out — the reader fetches those accounts' posts at read time and merges. Best of both, and the threshold is tunable by measurement.

Additions worth points: fan out only to active users (do not write into the inbox of someone who has not opened the app in 30 days — rebuild on demand); the inbox is a cache, not the source of truth, so it must be reconstructible; two-phase ranking (cheap retrieval, expensive rerank); and pagination by cursor, never by offset — with continuous inserts, offsets duplicate and skip items.

8.8 LLM-based question answering

The prompt: "Design an LLM-based Q&A system." (a confirmed Google question)

Requirements to pin down

Acceptable latency (streaming changes everything), sources (open web? private corpus?), whether citations are required, languages, and the policy for "I don't know". Non-goal: training the model.

Scale

10M questions/day → ~120 QPS average, 350 at peak
a 300-token answer → TTFT 300 ms + 300 × 20 ms ≈ 6 s
120 QPS × 6 s = 720 concurrent generations (Little's law!)
at ~8 concurrent requests per GPU → ~90 GPUs just for steady state

That Little's-law-applied-to-GPUs calculation is pure gold in a 2026 interview. Do it.

Architecture

query → normalisation plus language detection
      → semantic cache (query embedding, similarity > 0.95) ──hit──► answer
      → hybrid retrieval (ANN plus BM25) → rerank → top 10 chunks
      → prompt assembly (with citations and content isolation)
      → LLM (streaming) → guardrails → answer plus sources
      → logging for evals and feedback

Deep dives

• Semantic cache: similar questions ("what's the capital of Brazil" / "Brazilian capital") should reuse the answer. The risk: a loose threshold returns the wrong answer confidently. Combine high similarity with entity verification.
• Streaming: TTFT is the metric the user feels. Optimise the prefill and start showing early; the rest of the generation hides behind their reading.
• Routing by complexity: a small model for easy questions, a large one for hard ones, with a cheap classifier deciding. It halves cost with no perceptible loss.
• Fallback: model unavailable ⇒ return the retrieved passages with an extractive summary rather than an error. Degrade rather than fail.
• Security: retrieved content is data, not instructions (prompt injection); a PII filter on the output; and "I don't have enough information" must be a first-class path, not an accident.

8.9 ⭐ Translation and localization pipeline for AI Answers

The most likely case in your interview. It is literally the name of the team. Prepare this one until you can draw it with your eyes closed.

The prompt you should expect

"Google shows AI-generated answers at the top of search results. Design a system that serves those answers in the user's language, worldwide."

Phase 1 — Requirements (the questions you ask)

• How many languages? (assume ~40 for the arithmetic, but ask)
• Is the source answer generated in English and translated, or generated natively in each language? This is the most important question in the problem and almost nobody asks it.
• Machine translation with humans only sampling? (assume yes)
• What is the tolerance for a bad translation in a medical or financial answer? (⇒ risk classes)
• How often does the source answer change? (this defines invalidation)
• Acceptable latency: this sits on the SERP path ⇒ a brutal budget, ~200 ms.

Close it like this: "In scope: serving an existing answer in 40 locales at SERP latency, keeping translations fresh as source answers change, and rolling out new translation models safely. Out of scope: generating the source answer, the ranking of answers, and human translation workflow tooling. I'll assume machine translation with human raters sampling offline."

Phase 2 — Scale

10⁹ searches with an AI Answer/day globally
  → 10⁹ / 10⁵ s ≈ 10,000 QPS average, 30,000 at peak
  → 60% of traffic is non-English ⇒ ~6,000 QPS needing localization

Corpus: 10⁸ active answers × 2 KB = 200 GB of source
Pre-translating everything × 40 languages: 8 TB (fits easily!) — but the cost is NOT storage:
  10⁸ × 40 = 4 × 10⁹ translations × $0.001 = $4M per full model refresh
  ⇒ pre-computing everything is economically unviable on every model change

Popularity is Zipf: ~1% of answers carry ~80% of the traffic
  → pre-computing the top 1% × 40 languages = 4 × 10⁷ translations = $40k per refresh  ✅

This calculation carries the entire design. It turns "hybrid" from a guess into a conclusion.

"Storage isn't the constraint — 8 TB is nothing. Inference cost is the constraint, and the traffic distribution is Zipf, so pre-computing the head and serving the tail on demand isn't a compromise, it's what the numbers say."

Phase 3 — API and data

GET /v1/answers/{answer_id}?locale=pt-BR&max_wait_ms=150
 -> 200 {text, locale_served, fallback_used, source_version, model_version, quality_tier}

max_wait_ms in the contract is a strong detail: the caller (the SERP) declares its budget, and the service decides on its own between waiting for the translation and returning the fallback.

Bigtable, row key = answer_id#locale#model_version
  columns: text, quality_score, created_at, rater_verdict
  ⇒ a scan over the answer_id# prefix returns every locale of an answer in one row range

The row key is the design: it gives you both the read query and the audit-by-model-version query.

Phase 4 — High level

                       ┌──────── read path (hot) ────────┐
user → SERP → edge cache (CDN, key includes locale and versions)
                      │ miss
                      ▼
              localization service (regional)
                      │
      ┌───────────────┼──────────────────┐
      ▼               ▼                  ▼
  Redis cache    regional Bigtable   language fallback
  (hot              (persisted        (pt-BR→pt→en)
   translations)     translations)
                      │ absent
                      ▼
              translation queue (priority: interactive | batch)
                      │
                      ▼
           MT service (interactive pool | batch pool)
                      │
                      ▼
        quality scorer (async) → write → invalidate by version

┌──────── offline path ────────┐
traffic logs → popularity per (answer, locale)
            → pre-computation scheduler → batch queue
human raters ← stratified sampling ← published translations

Phase 5 — Deep dives (pick one and go deep)

(a) The cache key — start here, it is the heart

cache_key = answer_id + ":" + source_version + ":" + locale + ":" + model_version

Each component earns its place:

• source_version: the source answer changed ⇒ every translation of it is stale. With the version in the key, invalidation is automatic and instant — the old keys simply stop being queried and die of TTL. Without it you would need to sweep 40 keys per answer across N caches. This replaces a hard distributed problem with an integer increment.
• model_version: without it you serve an old model's output forever, and you can neither roll out nor roll back.
• locale explicitly, so translations do not leak between variants (pt-BR ≠ pt-PT).

(b) Pre-compute vs on demand — the hybrid decision

The cut-off rule, stated with a number: "Pre-compute a (answer, locale) pair once its predicted traffic exceeds roughly one request per day — below that, the pre-computation costs more than the on-demand inference it saves. I'd compute that prediction from the last 7 days of traffic and refresh the pre-compute set daily."

(c) The miss path — the product decision

When there is no translation and the budget is 200 ms, inference (300–2,000 ms) does not fit:

1. Is there a translation from an earlier model_version? Serve it and reprocess in the background.
2. Otherwise, fall back by language (pt-BR → pt → en) and mark fallback_used: true.
3. Enqueue the translation with interactive priority — the next user will be served.
4. Never make the user wait on the model on the SERP path.

"The user-facing decision here is: a slightly stale translation, or English, beats a spinner. I'd never block the SERP on inference. And I'd instrument fallback_used as a product SLI — it's the metric that tells us whether the pre-computation policy is actually working."

(d) Quality and model rollout

• Offline evals per language (COMET or chrF against a reference) in CI.
• Shadow: the new model translates the same traffic without serving; side-by-side comparison.
• Canary by language, never globally — translation regressions are language-specific. The order: low-traffic languages first, high-risk languages (medical, financial) last.
• Human raters on a sample stratified by language × risk category.
• Online signals: query reformulation, abandonment rate, explicit feedback.
• Rollback driven by a quality metric, not just latency — and rollback is cheap precisely because model_version is in the key: you switch the served version and the old cache is still there.

(e) Risk classes — the "user first" detail that shows Googleyness inside a technical design: a badly translated medical, legal or financial answer does real harm. Handle it by class: for high risk, require a minimum quality score, and if it is not met, serve English rather than a doubtful translation. That is a product requirement emerging from the engineering — and volunteering it is exactly the behaviour Google calls "do the right thing".

Phase 6 — Failures, 10×, and cost

At 10×: the bottleneck becomes inference cost, not serving capacity. The answer is to raise the hit rate (better keys, longer TTLs, a semantic cache for equivalent questions) and improve the popularity predictor — not to buy more GPUs.

Cost, said out loud: "Serving is cheap; inference is the bill. At a 92% cache hit rate, we run roughly 5 × 10⁷ inferences a day instead of 6 × 10⁸ — that's the difference between a $600k/day and a $50k/day system. Which is why I'd spend the engineering effort on the cache key and the popularity predictor before I'd spend it on the serving path."

8.10 Other cases — a five-minute skeleton each

Do these mentally. If you cannot give the answer in two sentences, that is the next one to study.

Part 9 — The spoken script

Communication is one of the four assessment axes. An excellent design explained badly scores a 2. Learn the blocks below until they come without effort — that frees your attention for the problem.

9.1 Opening

"Before I design anything, let me make sure I understand the problem and lock the scope. I'll take about seven minutes on requirements, five on scale, then draw the high-level design, and we can deep dive wherever you find most interesting. I'll watch the clock."

9.2 Scoping questions

"Who are the users, and what's the single most important action they take?" "Is this read-heavy or write-heavy, and roughly what ratio?" "How fresh does the data need to be — is a few seconds of staleness acceptable?" "Is this global from day one, or single-region?" "What's explicitly out of scope for today? Can I assume auth and billing are solved?"

9.3 Closing the scope

"Let me lock the scope: in scope are A, B and C. Out of scope: authentication, the admin UI, and billing. I'm assuming X and Y — tell me if either assumption is wrong, because Y in particular would change the design. Sound right?"

9.4 Estimating

"Let me put numbers on this, because the design depends on them." "A million daily users at ten actions a day is ten million requests, over roughly ten to the fifth seconds in a day — so about a hundred QPS average, call it three hundred at peak." "I'm going to round aggressively — I care about the order of magnitude, not the digits." "And here's what that number means for the design: …" ← never skip this sentence

9.5 Proposing and justifying

"I'll start with the simplest thing that could work, and then break it deliberately." "There are two reasonable options here. Option A is X, which buys us Y at the cost of Z. Option B is… I'd pick A, because in this system the read path dominates and Z is cheap for us." "I'm deliberately not adding a message queue yet — at three hundred QPS it's complexity without a payoff. If write volume grew tenfold, I'd revisit that."

9.6 Narrating the diagram

"Let me walk one request end to end, so we can check the design has no gaps." "On a cache hit — which I'd expect for about ninety percent of traffic — we return in five milliseconds. On a miss, two things can happen, and the interesting one is…"

9.7 Deep dive

"The failure mode I'd worry about here is…" "If this node dies mid-write, the message stays in the queue because we only ack after committing — so it gets redelivered, which is exactly why the handler has to be idempotent on job_id." "That's the trade-off: we're accepting up to sixty seconds of staleness in exchange for removing the database from the hot path entirely."

9.8 When they point at a hole (learn to enjoy this)

"That's a good point — you're right, that breaks when the source answer changes. Let me fix it: I'd put the source version in the cache key, which makes invalidation automatic." "Good catch. I was assuming X, and if that's not true the design changes here and here."

Never defend a broken design. Acknowledging and fixing quickly is assessed above getting it right first time, because it is what the real job requires.

9.9 When you do not know

"I haven't worked with that specific system, but the property I'd need is X — let me reason about it from first principles and you can correct me."

That is a strong answer. Faking knowledge is the worst possible outcome: the interviewer almost always finds out, and then the Googleyness score drops too.

9.10 Managing the clock and closing

"We're about halfway — I want to make sure we get to a deep dive, so let me lock the high-level design here and move on." "To summarize: the design is X. The main trade-off I made was A over B, because C. The first thing that breaks at ten times the scale is D, and I'd fix it with E. If I were shipping a v1 next month, I'd cut F and G and add them once the traffic justifies it."

That sixty-second closing summary is free and most people skip it. It leaves the interviewer with an organised narrative in their head at exactly the moment they sit down to write your feedback.

Part 10 — Quiz (100 questions)

Answer out loud, in English, in thirty seconds or less each. If you hesitate, mark it and go back to the module.

Scale and estimation

1. How many seconds in a day, for arithmetic purposes? (10⁵)
2. 1M DAU × 10 actions/day gives what average and peak QPS?
3. How long is an intercontinental round trip? And one inside the datacenter?
4. How many times faster is RAM than an SSD? And than a spinning disk?
5. How many cores for 10,000 QPS at 5 ms of CPU per request?
6. What is 99.9% availability in downtime per month?
7. Three services at 99.9% in series give what?
8. What does Little's law say, and how do you use it to size a thread pool?
9. Why does nobody run at 90% utilisation?
10. Why does p99 matter more than the mean when a page makes ten calls?
11. What is a hedged request and what does it cost?
12. How much does a 768-dimension float32 embedding occupy? And in int8?

Distributed fundamentals

13. State CAP in a useful way (not "two of three").
14. What does PACELC add to CAP?
15. The observable difference, to a user, between strong and eventual consistency.
16. What is read-your-writes and how do you implement it cheaply?
17. What is monotonic reads and which bug does it prevent?
18. Why does R + W > N guarantee an up-to-date read?
19. In Raft, what prevents two simultaneous leaders?
20. Why is the election timeout randomised?
21. What does a cross-datacenter consensus operation cost, in orders of magnitude?
22. What is TrueTime and why does Spanner wait ~7 ms before committing?
23. What does a vector clock give you that a timestamp does not?
24. What is a fencing token and which race does it prevent?
25. Why is 2PC considered blocking?
26. What is the transactional outbox pattern and what does it solve?
27. What is a saga and what does it sacrifice?
28. What is split brain and how does quorum prevent it?
29. What is a CRDT and when is it worth it?
30. Synchronous vs asynchronous replication: what does the user perceive?

Sharding and storage

31. Sharding by hash vs by range: what do you lose with each?
32. What problem does consistent hashing actually solve?
33. What are virtual nodes for?
34. Does consistent hashing solve hot keys? (no — explain why)
35. How do you reshard without downtime?
36. B-tree vs LSM-tree: which for heavy writes, and why?
37. What is write amplification and where does it appear?
38. What is a WAL for?
39. What is MVCC and why does Postgres need VACUUM?
40. What is write skew and which isolation level prevents it?
41. 3× replication vs erasure coding: overhead and reconstruction cost?
42. Why is offset pagination bad under concurrent writes?

Caching

43. Cache-aside vs write-through vs write-behind: when each?
44. What is a cache stampede and name three fixes.
45. Why TTL with jitter?
46. What is cache penetration and how do you defend against it?
47. How do you handle a hot key?
48. Why is a versioned key better than explicit invalidation?
49. What hit rate makes a cache worthwhile, and where do you estimate it from?
50. In-process cache vs distributed cache: the trade-off?
51. What is W-TinyLFU and which LRU problem does it solve?
52. What is stale-while-revalidate and why does it help perceived latency?

Queues and streaming

53. At-most-once, at-least-once, exactly-once: which is actually used, and why?
54. Why is end-to-end exactly-once impossible?
55. What makes a handler idempotent in practice?
56. What is a DLQ for, and what is missing if you do not alert on it?
57. Which is the better queue saturation metric: depth or age? Why?
58. Event time vs processing time; what is a watermark?
59. Lambda vs Kappa: why did Kappa win?
60. What is backpressure and what happens without it?

Network, load balancing and search

61. L4 vs L7 load balancer.
62. What is anycast and why is it better than DNS for failover?
63. What is "power of two choices" and why does it work so well?
64. Why are sticky sessions technical debt?
65. What is connection draining?
66. Inverted index: what is a posting list?
67. Index sharding by document vs by term: which is used and why?
68. What is two-phase ranking?
69. How do you serve autocomplete in 50 ms?
70. How does a bloom filter work, and how many bits per element for 1% error?
71. When do you use HyperLogLog instead of a set?
72. What is a Count-Min Sketch for?

Reliability

73. Describe the mechanics of a cascading failure.
74. Why does restarting not fix a cascade?
75. What is deadline propagation and what does it eliminate?
76. Why are unbounded queues dangerous?
77. What are the four criticality classes and what are they for?
78. What is the client-side adaptive throttling formula, and what does K=2 mean?
79. What are the two retry limits Google uses? (3 per request, 10% per client)
80. Why is backoff without jitter nearly useless?
81. Why is retrying at multiple layers a disaster?
82. What is a circuit breaker, and when does it close again?
83. What is a bulkhead?
84. Give three examples of graceful degradation and one case where degrading is wrong.
85. SLI, SLO, error budget: define all three and say how the budget changes decisions.
86. Why alert on symptoms rather than causes? What is burn rate?
87. What is shadow traffic and when do you use it?
88. Why does a canary need an abort criterion defined in advance?
89. What is expand-migrate-contract in a schema migration?
90. Define RTO and RPO and give an example with numbers.

AI/ML and this role's domain

91. Prefill vs decode: which is memory-bandwidth bound?
92. What are TTFT and TPOT, and how do you estimate total latency?
93. Why can the KV cache take more memory than the weights?
94. What is continuous batching and what problem does it solve?
95. What is the central trade-off of batching in serving?
96. What belongs in a translation cache key, and why each piece?
97. Why is a model canary done per language rather than globally?
98. Why hybrid search (vector plus BM25) rather than vector alone?
99. What failure modes are specific to RAG?
100. Why not use an LLM for language detection?

Part 11 — A four-week plan, and the traps

11.1 The plan

You have two gaps (concurrency and system design) and the first round is DSA only. System design only appears in the second round — so this plan assumes it starts after the first round, or runs in parallel at a light pace.

The rule that makes the difference: the week 4 designs must be done speaking out loud and recording. Watching ten minutes of your own recording teaches you more about your Communication score than reading this entire document.

Suggested order for the timed designs: URL shortener (warming up the method) → rate limiter → LLM Q&A → translation pipeline (twice; it is this role's case).

11.2 The traps, in order of frequency

1. Drawing before asking. The number one mistake.
2. Estimating and then not using it. A number that changes no decision was wasted time.
3. Buzzwords with no justification. One "why" sentence per component. No exceptions.
4. Disproportionate complexity. Senior cuts.
5. Staying on the surface. The deep dive is where the score is. Failure, concurrency, numbers.
6. Ignoring failure. Every box in the diagram needs an answer to "and if this dies?".
7. Not managing the clock. You own it.
8. Defending a broken design. "You're right" is a strong answer.
9. Silence. If you are quiet, you are being assessed on missing data.
10. Never mentioning production. SLOs, rollout, monitoring, cost. This is what separates senior.
11. Forgetting the user. The best technical answer that makes the product worse is a wrong answer.
12. Not closing. The sixty-second summary at the end is free.

11.3 The final check, the night before

• Google Drawings: I can create a box, an arrow, text and a duplicate without thinking
• I can recite the latency table and the three estimation formulas
• I can narrate one request end to end in English, without stumbling
• I can draw the translation case from memory
• For every component of my favourite design, I can say how it fails and how it degrades
• I have one good question for the end (about the team, not about benefits)
• Camera, lighting, water, paper, and the Meet link tested

Say it

Cover the answers. Out loud, in English. Sixteen cards from this module are scheduled by drill.py conceitos; these are the six that carry the most weight.

Now do this

Reading this document again will not improve your score. Drawing will.

1. URL shortener — 45 minutes, timed, in Google Drawings, narrated out loud and recorded. Read only the prompt; close the document before you start.
2. Rate limiter — 45 minutes, same rules. This one is chosen because the interesting decision is "does it have to be exact?", and that is a scoping question.
3. Translation pipeline — 60 minutes. Twice, on different days. It is the case this role is named after.

After each one, listen to ten minutes of your own recording. That will teach you more about your communication score than any amount of re-reading, and it is the only way to hear what the interviewer hears.

Stop when you can produce the three estimates and the request narration without hesitating, and the translation case comes out of memory.

Sources

Grounded in the following sources, consulted in September 2026:

• Google SRE Workbook, ch. 12 — Introducing Non-Abstract Large System Design
• Google SRE Book — Handling Overload
• Latency Numbers Every Programmer Should Know (Jeff Dean / jboner gist)
• A peek behind Colossus, Google's file system
• Spanner: TrueTime and external consistency
• Spanner: Google's Globally-Distributed Database (OSDI 2012)
• Google System Design Interview Questions — Exponent
• What to Expect in Google's System Design Interview
• System Design Interview Rubric by Level — PracHub
• Taming Throughput-Latency Tradeoff in LLM Inference (Sarathi-Serve)
• A Survey on Efficient Inference for Large Language Models
• Translation API architecture and caching strategies — Translated
• The recruiter's prep email (Google, September 2026) — the official interview scope