Observability
Agent observability connects user outcomes to model turns, tool calls, handoffs, guardrails, latency, token use, and errors. Useful telemetry makes an end-to-end workflow explainable while controlling sensitive prompt, output, argument, and result data.
Questions to answer
Resolve these before adding tools, frameworks, or automation.
- Can one user-visible outcome be followed across models, tools, agents, and queues?
- Which content is sensitive and should be redacted, sampled, or excluded?
- Which metrics reveal quality, latency, cost, retries, and operational saturation?
Implementation lifecycle
Build the evidence in this order
- 01
Instrument
Create end-to-end trace identities and spans for model, tool, handoff, and guardrail operations.
- 02
Protect
Classify sensitive telemetry and apply redaction, access, retention, and sampling rules.
- 03
Learn
Connect operational signals to eval failures, incidents, cost, and product outcomes.
Artifacts to maintain
- trace
- span conventions
- dashboard
- alert policy
Continue through AgentAtlas
Evidence ledger
Primary documentation behind this guide
OpenTelemetry
OpenTelemetry semantic conventions
Standard names, attributes, traces, metrics, logs, and generative-AI operations for interoperable telemetry.
OpenAI
OpenAI Agents SDK tracing
Tracing model generations, tools, handoffs, guardrails, custom events, workflow grouping, and sensitive-data controls.
Next topic
Deployment and operations