Voice Agents: Pipecat and LiveKit
Voice agents are a first-class production category in 2026. Pipecat gives you a Python frame-based pipeline (VAD → STT → LLM → TTS → transport). LiveKit Agents bridges AI models to users over WebRTC. Production latency targets land at 450–600ms end-to-end for premium stacks. Describe Pipecat's frame-based pipeline: DOWNSTREAM (source→sink) and UPSTREAM (control). Name the canonical voice pipeline stages and which transports Pipecat supports. Explain LiveKit Agents' two voice agent classes (MultimodalAgent, VoicePipelineAgent) and when each fits. Summarize 2026 production latency expectations and how they drive architecture choices. Voice agents are not a text loop with TTS bolted on. Latency budgets are brutal (600ms), partial audio is the default, turn detection is a model, and transports range from telephony SIP to WebRTC. Either you build a frame-based pipeline (Pipecat) or you lean on a platform (LiveKit). Python frame-based pipeline framework. Frame → FrameProcessor chain. Two flow directions:. DOWNSTREAM — source → sink (audio in, TTS out). UPSTREAM — feedback and control (cancellation, metrics, barge-in). PipelineTask manages lifecycle with events (onpipelinestarted, onpipelinefinished, onidletimeout) and observers for metrics/tracing/RTVI. Typical pipeline: Transports: Daily, LiveKit, SmallWebRTCTransport, FastAPI WebSocket, WhatsApp. Pipecat Flows adds structured conversations (state machines). Pipecat Cloud is the managed runtime. Bridges AI models to users over WebRTC. Key concepts: Agent, AgentSession, entrypoint, AgentServer. Two voice agent classes:. MultimodalAgent — direct audio via…
Voice Agents: Pipecat and LiveKit: Voice agents are a first-class production category in 2026. Pipecat gives you a Python frame-based pipeline (VAD → STT →…
This free lesson is part of the AI Engineering from Scratch curriculum. Read the full explanation, run the lesson code, and verify the result in the interactive reader or from the repository source.
Browse the complete course catalog or open this lesson on GitHub.