Reliability & operations · Glossary term

What is Shadow Traffic?

A copy of live request traffic sent to a candidate system for observation while the candidate response remains outside the primary user response path. Because the copied request still executes, its side effects must be isolated.

Why does Shadow Traffic matter?

It exposes the candidate to real input shapes and load while limiting user impact, which can reveal failures absent from synthetic tests.

Shadow Traffic in practice

Remove or tokenize sensitive fields, route tools and dependencies to sandboxed or no-op targets, block writes at capability boundaries, preserve request correlation, and prevent shadow load from competing with user traffic.

What is the common confusion about Shadow Traffic?

Keeping a candidate response off the primary path does not make execution side-effect-free. A canary release differs because it serves real users from the candidate for a controlled share of traffic.

Learn Shadow Traffic in the course

Start with

Taught in Phase 17: Infrastructure & Production.

  • Canary ReleaseA deployment strategy that exposes a new version to a limited slice of traffic or infrastructure before expanding the rollout.
  • Evaluation (Eval)A defined process for measuring model or system behavior on representative tasks using explicit success criteria, data, scorers, and…
  • TraceA correlated record of one request or task across model calls, retrieval, tools, state transitions, retries, approvals, and evaluations.
  • Model ServingThe runtime and API layer that loads versioned model artifacts, accepts inference requests, schedules execution, manages resources, and…

Sources

More terms in Reliability & operations

Open the Reliability & operations list in the glossary

This entry comes from glossary/terms.md on GitHub. Browse all 250 glossary terms.