Voice Activity Detection & Turn-Taking — Silero, Cobra, and the Flush Trick
Every voice agent lives or dies on two decisions: is the user speaking now, and are they done? VAD answers the first. Turn-detection (VAD + silence-hangover + semantic endpoint model) answers the second. Get either wrong and your assistant either cuts users off or never shuts up. Three distinct decisions a voice agent makes on every 20 ms chunk: Is this frame speech? — VAD. Binary, per-frame. Has the user started a new utterance? — onset detection. Has the user finished? — end-pointing (turn-end). The naive answer (energy threshold) fails on any noise — traffic, keyboards, crowd babble. The 2026 answer: Silero VAD (open, deep-learned) + a turn-detection model (semantic endpointing) + a VAD-calibrated silence hangover. VAD cascade: energy → Silero → turn-detector → flush trick Tier 1: energy gate. Cheapest. Threshold RMS at -40 dBFS. Filters obvious silence but fires on any noise above the threshold. Tier 2: Silero VAD (2020-2026, MIT). 1M parameters. Trained on 6000+ languages. Runs in 1 ms per 30 ms chunk on a single CPU thread. 87.7% TPR at 5% FPR. The open-source default. Tier 3: semantic turn detector. LiveKit's turn-detection model (2024-2026) or your own small classifier. Distinguishes "pause mid-sentence" from "done talking." Uses linguistic context (intonation + recent words), not just silence. Threshold. Silero outputs a probability; classify speech at > 0.5…
Voice Activity Detection & Turn-Taking — Silero, Cobra, and the Flush Trick: Every voice agent lives or dies on two decisions: is the user speaking now, and…
This free lesson is part of the AI Engineering from Scratch curriculum. Read the full explanation, run the lesson code, and verify the result in the interactive reader or from the repository source.
Browse the complete course catalog or open this lesson on GitHub.