Tail Latency
Also known as: long tail latency, high-percentile latency
Tail latency is the response time of the slowest requests — typically p99 and beyond. It matters more than the average because a page that fans out to many backends is as slow as its slowest call, so a rare per-service delay becomes a common per-page one.
Last reviewed · Part of the Architecture Glossary
In practice
Fan-out is the amplifier. If one backend call is slower than its p99 one time in 100, and a page makes 50 such calls, the probability the page hits at least one slow call is 1 − 0.99^50 = 39%. The service looks fine on its own dashboard; the page is slow for two users in five.
Where the tail comes from, in rough order of frequency:
- Queueing. Utilisation above ~70% makes the queue term dominate; at 90% the wait is roughly 9x the service time.
- Garbage collection and compaction. Stop-the-world pauses, LSM compaction stalls.
- Cold caches and cold starts after a deploy or scale-out.
- Retries and timeouts that add a full timeout period to the affected request.
- Noisy neighbours — shared CPU, shared IOPS, a hot partition.
Mitigations worth knowing: hedged requests (issue a duplicate after p95 elapses, take the first response — Google measured a large p99 reduction for ~5% extra load), request cancellation, and reducing fan-out width.
When it matters
Any user-facing composite page, any service with an SLO expressed at a percentile, any system where timeouts cascade.
Common mistake
Alerting on mean latency. The mean can improve while the tail doubles — you cannot detect a tail regression with an average, and the tail is what users experience as "the site is broken."
See also
- p99 (99th Percentile)p99 is the latency value below which 99% of requests complete — one request in a hundred is slower.
- Coordinated OmissionCoordinated omission is a measurement error in which a load generator waits for a slow response before sending the next request, so the requests that would have been issued during the stall are never recorded.
- Little's LawLittle's Law states that the average number of items in a stable system equals the average arrival rate times the average time each item spends there: L = λW.
- BackpressureBackpressure is the mechanism by which an overloaded component signals upstream to slow down, rather than accepting work it cannot complete.