Agent Observability: Langfuse, Phoenix, Opik
Three open-source agent observability platforms dominate 2026. Langfuse (MIT) — 6M+ installs/month, tracing + prompt management + evals + session replay. Arize Phoenix (Elastic 2.0) — deep agent-specific evals, RAG relevancy, OpenInference auto-instrumentation. Comet Opik (Apache 2.0) — automated prompt optimization, guardrails, LLM-judge hallucination detection. Name the three top open-source agent observability platforms and their licenses. Distinguish what each one is strongest at: Langfuse (prompt mgmt + sessions), Phoenix (RAG + auto-instrumentation), Opik (optimization + guardrails). Explain why 89% of organizations report having agent observability in place by 2026. Implement a stdlib trace-to-dashboard pipeline with LLM-judge evaluation. OTel GenAI (Lesson 23) gives you the schema. You still need the platform that ingests spans, runs evaluations, stores prompt versions, and surfaces regressions. The three contenders each emphasize different parts of the lifecycle. 6M+ SDK installs/month, 19k+ GitHub stars. Features: tracing, prompt management with versioning + playground, evaluations (LLM-as-judge, user feedback, custom), session replays. June 2025: formerly commercial modules (LLM-as-a-judge, annotation queues, prompt experiments, Playground) open-sourced under MIT. Strongest for: end-to-end observability with tight prompt-management loop. Deeper agent-specific evaluation: trace clustering, anomaly detection, retrieval relevancy for RAG. Native OpenInference auto-instrumentation. Pairs with managed Arize AX for production. No prompt versioning — positioned as a drift/behavioral-regression tool alongside broader platforms. Strongest for: RAG relevancy, behavioral drift, anomaly detection. Automated prompt optimization through A/B…
Agent Observability: Langfuse, Phoenix, Opik: Three open-source agent observability platforms dominate 2026. Langfuse (MIT) — 6M+ installs/month, tracing +…
This free lesson is part of the AI Engineering from Scratch curriculum. Read the full explanation, run the lesson code, and verify the result in the interactive reader or from the repository source.
Browse the complete course catalog or open this lesson on GitHub.