What is Backpressure?

A flow-control mechanism that slows or rejects upstream work when a downstream component cannot process it safely at the current rate.

Why does Backpressure matter?

Without backpressure, queued agent runs, tool calls, or streamed events can exhaust memory, exceed rate limits, and amplify retries.

When the evaluator queue reaches its limit, pause new agent jobs or return a retryable response instead of accepting unbounded work.

Backpressure protects capacity before failure. A circuit breaker stops calls after failures show a dependency is unhealthy.

No lesson links to this term yet. Search the course catalog for it.

Rate LimitA policy that caps requests, tokens, concurrent work, or another resource within a defined time or capacity window.
Retry with BackoffRepeating a failed transient operation after progressively longer delays, usually with randomized jitter and a strict retry limit.
Circuit BreakerA reliability control that temporarily stops calls to a dependency after failures cross a threshold, then probes whether the dependency…
Admission ControlA pre-acceptance gate that decides whether a request may enter a bounded queue or service under the system's current capacity, priority,…
AutoscalingA control loop that changes the number or capacity of serving workers from observed demand, resource use, or application metrics within…
Continuous BatchingA serving scheduler that adds and removes generation requests at iteration boundaries instead of waiting for every request in a fixed…
Load SheddingDeliberately rejecting, dropping, or cancelling selected work at one or more overload boundaries when demand exceeds the capacity…
SaturationThe degree to which a constrained resource or service has exhausted its capacity, including queued work that cannot begin promptly.