Reliability & operations · Glossary term
What is Saturation?
The degree to which a constrained resource or service has exhausted its capacity, including queued work that cannot begin promptly.
Why does Saturation matter?
Utilization alone can appear acceptable while memory, accelerator slots, queue depth, or a downstream quota is already limiting useful throughput.
Saturation in practice
Identify each critical resource, measure active and waiting work, relate saturation to tail latency and errors, and alert before the queue enters an unstable growth regime.
What is the common confusion about Saturation?
Saturation is not one universal percentage. The limiting resource and its queueing behavior depend on the workload and architecture.
Learn Saturation in the course
Lessons that name Saturation in a title or section
- The Autonomous Coding Agent Landscape (2026)
SWE-bench Verified went from 4% to 80.9% in under three years. Same Claude Sonnet 4.5 scored 43.2% on SWE-agent v1 and 59.8% on Cline autonomous — the scaffolding around the model now matters as…
Covered in Phase 15: Autonomous Systems.
Related terms
- ObservabilityThe ability to understand an AI system's behavior from recorded inputs, outputs, state transitions, tool calls, timings, costs, errors,…
- AutoscalingA control loop that changes the number or capacity of serving workers from observed demand, resource use, or application metrics within…
- BackpressureA flow-control mechanism that slows or rejects upstream work when a downstream component cannot process it safely at the current rate.
- Tail LatencyThe latency experienced by the slowest portion of requests, commonly summarized with a high percentile under a stated workload and time…
- Admission ControlA pre-acceptance gate that decides whether a request may enter a bounded queue or service under the system's current capacity, priority,…
- Dynamic BatchingA runtime policy that forms inference batches from queued requests according to compatible shapes, maximum size, priority, and allowed…
Sources
More terms in Reliability & operations
This entry comes from glossary/terms.md on GitHub. Browse all 250 glossary terms.