04.7.8 · Skill in AI-native engineering

AI observability

Tracing and inspecting what models and agents actually did in production.

AI observability records prompts, outputs, tool calls, latency and cost for each request. Traces show why an agent went wrong. Production traces are also the best source of new eval cases.

Your status · AI observability

Not startedLearningComfortable

Saved in this browser only. It colours this skill's tile on your map.

Resources

articleStart here
An introduction to observability for LLM-based applications ↗
OpenTelemetry · 2024 · 11 min
What to record for model calls (prompts, tokens, latency, cost) using standard tracing.
articleGo deeper
A field guide to rapidly improving AI products ↗
Hamel Husain · 2025 · 28 min
Error analysis first: read real traces, find what fails, then measure it.

Tools

ToolOpen source
Langfuse
langfuse.com ↗
Open-source tracing, evals and prompt management for LLM applications.