Cache Stampede

In practice

Do the arithmetic once and the risk stops being abstract. A homepage query runs at 2,000 req/s with a 99% hit rate — the database sees 20 req/s and the query takes 400 ms. The entry expires. For the 400 ms it takes to recompute, every arriving request misses: 800 concurrent identical queries against a pool of 50 connections. The pool exhausts, unrelated queries queue behind it, and the outage looks like a database problem.

Three mitigations, roughly in order of preference:

• Coalescing / mutex. The first miss takes a lock and recomputes; the rest wait for the result. One origin query instead of 800.
• Probabilistic early expiry. Recompute before the TTL with a probability that rises as expiry approaches (XFetch). Refresh happens while the old value is still serving.
• Stale-while-revalidate. Serve the expired value and refresh in the background. Correct only where slightly stale data is acceptable — which is most read paths, if you ask honestly.

When it matters

Any high-traffic key with an expensive origin: homepage feeds, pricing tables, permission sets, rendered fragments, embedding lookups.

Common mistake

Setting the same TTL for every key populated by the same warm-up job. They then expire in the same second, and the stampede hits every key at once. Add ±10% jitter to every TTL.