Capstone Lesson 28: Observability with OTel GenAI Spans and Prometheus Metrics
An agent harness without observability is a black box that costs money. This lesson hand-rolls a span builder that emits records compliant with the OpenTelemetry GenAI semantic conventions, writes them to a JSON-Lines file one span per line, and exposes counters and histograms in Prometheus text format. The whole thing is stdlib Python and runs offline. Build a span data class shaped to the OpenTelemetry GenAI semantic conventions. Implement a JSONL exporter that writes one self-contained span per line. Build counters and histograms with labels and Prometheus text-format exposition. Wrap any callable in a span context manager that records duration, status, and exceptions. Verify that the emitted spans roundtrip through json.loads and match the spec shape. A coding agent in production produces three classes of artifact every turn: a model call, a tool execution, and a verification gate decision. None of these are useful without structured telemetry. The first failure mode is the missing trace. Something went wrong on Tuesday but the only record is a 500-line chat log. There is no record of which tool ran, how long it took, how many tokens went into the prompt, or whether the gate refused anything. The agent author has to guess. The second failure mode is the unparseable trace. The harness wrote spans but used its own ad-hoc field names. Nothing…
Capstone Lesson 28: Observability with OTel GenAI Spans and Prometheus Metrics: An agent harness without observability is a black box that costs money. This…
This free lesson is part of the AI Engineering from Scratch curriculum. Read the full explanation, run the lesson code, and verify the result in the interactive reader or from the repository source.
Browse the complete course catalog or open this lesson on GitHub.