Get Your CertSystem Design & Architecture Interview QuestionsPlayCompete
‹ All System Design & Architecture Interview Questions questions

System Design & Architecture Interview Question 3: Cache Stampede Hot Key

System Design & Architecture Interview Questions · Test 1 · question 3 of 10

Cachinghard

An e-commerce home page shows a "today's deals" block. It is computed by an aggregate database query that takes about 2 seconds and is cached with the cache-aside pattern under a single key with a 60-second TTL. Thousands of requests per second read this key. The cache hit rate is 99.9% and the cache has plenty of free memory.

Monitoring shows that once a minute the database CPU jumps to 100% for a few seconds and home-page p99 latency spikes, then everything returns to normal. What is the most likely cause, and what is the best fix?

Select one answer
Show answer and explanation

Answer:

  • A. On expiry, every concurrent request misses and runs the query at once; coalesce misses so only one request per key queries the database

This is a cache stampede, also called a thundering herd. With plain cache-aside, an expired hot key means every request that arrives during the 2-second recompute also misses, and each one runs the same expensive query against the database. The spike happens exactly once per TTL period, which is the giveaway. Request coalescing makes sure only one request is fetching an uncached key at a time while the rest wait for and reuse its result. That turns thousands of identical queries into one. Refreshing the value in the background before it expires, or serving the stale value while one caller refreshes it, solves the same problem.

Option B does not fit the evidence. The stem says the cache has plenty of free memory, and eviction under memory pressure would not follow a precise 60-second rhythm.

Option C addresses a different concern, which is keeping cached rows in sync with writes. The deals block is a computed aggregate with a TTL, not a row that one write updates, and write-through does nothing to stop concurrent misses when the key expires.

Option D makes the stampede happen six times as often. How much data changed has nothing to do with the cost of the recompute, which is the fixed 2-second query.

Reference: Caching challenges and strategies - Amazon Builders' Library

Practice Test 1 ›

Study guides for System Design & Architecture Interview Questions

System design interview questions: how to prepareSystem design interview questions explained: what the round tests, common questions, an answer framework, and how AI and ML system design differ.