Day 31 of 100
The Cache Stampede
🔥 Problem Statement
A popular cache entry — something read constantly, computed from an expensive database query — expires right on schedule, exactly as its TTL says it should. In the same instant, thousands of requests that were all reading from that cache key hit empty and go straight to the database simultaneously, all trying to recompute the same value at once.
The database, sized comfortably for the trickle of cache-miss traffic it normally sees, gets hit with a spike it wasn't built for, and briefly struggles under load it would never see if requests just queued up politely instead of all landing at once.
🏗️ Environment
- Shared cache in front of a popular, expensive-to-compute endpoint
- High concurrent traffic on this specific cached key
- Database: sized for cached-read load, not for raw concurrent load
- Symptom: sudden database spike, tightly correlated with cache TTL expiry
🤔 Your Challenge
Why does a single cache expiration event turn into a database incident, and how would you prevent that?
- If a thousand concurrent requests all read the same expired cache key, what does each of them individually do next by default?
- Is the database actually seeing more total load than usual, or the same load compressed into a much shorter window?
- What would 'only one request recomputes the value, others wait for it' look like as a design instead?
- How would randomizing TTL slightly across cache entries change this pattern?
Solution Hidden
Think through the problem yourself before looking at the answer.
💡Solution
Step 1 — Understand the symptoms
The database isn't seeing more total requests than it would over, say, the next several minutes without a cache — it's seeing roughly that same demand compressed into a single instant, because every one of those thousands of concurrent readers independently discovers the cache is empty at the same moment and independently decides to recompute the value by querying the database. Without any coordination between those readers, "cache miss" becomes "thousand simultaneous cache misses," and the database absorbs a spike shaped nothing like its normal load pattern.
Step 2 — Identify the likely bottleneck
This is a specific, well-known failure mode called a cache stampede (or dogpile effect): a popular key, a hard expiration, and no coordination among concurrent readers about who's responsible for repopulating the cache once it's empty. It's not a sizing problem with the database in the general sense — the database can handle this query rate over time — it's a concurrency problem: too many identical, redundant recomputations landing in the same instant.
Step 3 — Investigation
- Confirm the timing correlation precisely — does the database spike align exactly with this specific key's TTL expiry, repeatedly, across multiple occurrences? A tight, repeated correlation is close to definitive confirmation of a stampede rather than an unrelated load event.
- Measure how many concurrent requests are actually reading this specific key around the expiry moment — this quantifies exactly how much redundant recomputation is happening.
- Check whether other popular cached keys show the same pattern at their own respective expiry moments — if so, this is a general caching-strategy gap, not a one-key anomaly.
- Check current cache/database interaction code for any existing coordination (a lock, a "recompute in progress" flag) — confirming there is none is what establishes this as the root cause rather than a symptom of something else.
Step 4 — Recommended action
A few complementary fixes, often used together:
- Request coalescing / locking: when a cache miss occurs, the first request to notice takes a lock and recomputes the value; every other concurrent request for the same key waits for that recomputation to finish and reads the resulting cached value, rather than independently querying the database.
- TTL jitter: instead of every instance of a key expiring at exactly the same configured TTL, add a small random variance — this spreads expiration (and therefore potential stampedes) across a window instead of a single instant, especially valuable when many different keys share the same nominal TTL.
- Pre-warming / background refresh: for genuinely hot keys, proactively refresh the cache slightly before expiry (in the background, without waiting for a miss) so it effectively never goes empty during real traffic.
Which combination makes sense depends on how hot the specific key is and how expensive the underlying computation is — for the hottest, most expensive keys, all three are often worth combining.
Step 5 — Engineering lesson
A cache's job isn't just "store the answer so we don't recompute it" — for popular data, it's also "make sure a recompute, when it does eventually happen, happens exactly once, not once per concurrent reader." Missing that second half of the job is invisible under low or moderate traffic and becomes a real incident the moment a hot key's TTL expires under real concurrent load — the cache did exactly what it was configured to do, and that was the problem.
🧠 Today’s Takeaway
A cache expiration event can become a database incident.
🔔 Don’t Miss Tomorrow’s Challenge
A new real-world engineering challenge is released every day.
100 Days → 100 Challenges → 100 New Things Learned.
Get the useful stuff, not the noise.
Occasional notes on engineering, Platform Engineering, AI, cloud and things I’m learning along the way.

