Backpressure
Also known as: flow control, load shedding signal
Backpressure is the mechanism by which an overloaded component signals upstream to slow down, rather than accepting work it cannot complete. Without it, queues grow without bound, latency climbs until every response is useless, and the system fails by memory exhaustion instead of by an honest rejection.
Last reviewed · Part of the Architecture Glossary
In practice
An unbounded queue is not a buffer, it is a delayed crash. Little's Law makes the failure mode concrete: with arrival rate λ = 1,000/s and service rate 800/s, the queue grows by 200 items every second forever. Latency grows with it, and by the time a request is served the client has long since timed out — so the work is pure waste while it consumes the capacity that would have cleared the backlog.
What backpressure looks like at each layer:
- In-process: bounded queues that block or reject on
put; reactive streamsrequest(n); a semaphore around a dependency. - HTTP:
429withRetry-After, admission control, a concurrency limiter (AIMD or Netflix'sconcurrency-limits). - TCP: the receive window — the original backpressure mechanism, and a decent mental model for the rest.
- Queues: consumer-driven prefetch, so a slow consumer pulls less rather than being pushed more.
When it matters
Every ingestion path, every worker pool, every gateway in front of a dependency with finite capacity.
Common mistake
Bounding the queue but then blocking the producer thread on put. The pressure just moves one hop upstream and exhausts the caller's thread pool instead. Reject explicitly and let the client decide — shedding 5% of load with a fast 429 beats failing 100% slowly.
See also
- Circuit BreakerA circuit breaker wraps calls to a dependency and stops making them once the failure rate crosses a threshold, failing fast for a cool-down period before letting a trial request through.
- Thundering HerdA thundering herd is a surge of simultaneous requests to one resource, triggered by an event that releases many waiting clients at once — a cache expiry, a service restart, a synchronised retry, a cron.
- Graceful DegradationGraceful degradation is the property of continuing to deliver core functionality with reduced features when a dependency fails, instead of returning an error.