What Is AI Agent Observability Tools?
AI agent observability tools capture traces of what an agent did, step by step: model calls, tool calls, retrievals, handoffs, cost and latency. Most of them build on OpenTelemetry or accept OpenTelemetry data, so the same instrumentation can feed an open-source platform, a commercial one or an existing APM suite. This page lists the standards and instrumentation libraries, open-source platforms, commercial platforms, APM vendors and cloud agent services in one place, with the latest release shown for tools that publish releases on GitHub. It is a directory, not a ranking.
Standards & Instrumentation
5 entriesVendor-neutral conventions and libraries that emit traces from agent code.
Open-Source & Self-Hostable Platforms
9 entriesTracing, evaluation and prompt tools you can run yourself.
Commercial LLM Observability Platforms
5 entriesHosted platforms for agent traces, evaluation and monitoring.
APM & Observability Vendors
4 entriesExisting observability suites that now cover LLM and agent workloads.
Cloud Agent Platform Observability
3 entriesTracing built into the major cloud agent runtimes.
Related Pages
5 entriesRelated pages on this site.
Frequently Asked Questions
What is AI agent observability?
It is the practice of recording what an agent actually did: each model call, tool call, retrieval and decision, with inputs, outputs, latency, cost and errors. A trace of one run lets you debug failures, check behaviour and measure quality, which ordinary application logs rarely capture.
How is it different from traditional application monitoring?
Traditional monitoring checks that a service is up and fast. An agent can be up and fast yet still give wrong answers or take unsafe actions, so agent observability also records prompts, tool choices and outputs and often scores them with evaluators.
Is there an open standard for agent traces?
Yes. OpenTelemetry has GenAI semantic conventions, including agent spans, and libraries such as OpenInference and OpenLLMetry emit data in that style. The conventions are still evolving, so check the current specification before depending on exact attribute names.
Should I choose an open-source or commercial platform?
Open-source tools such as Langfuse, Phoenix, Opik and Laminar can be self-hosted, which helps when prompts contain sensitive data. Commercial and APM vendors suit teams that already use them for other telemetry. Instrumenting with OpenTelemetry keeps the choice reversible.
What should I watch out for?
Traces can hold personal data, secrets and customer content, so decide what to redact and how long to keep it. Also budget for volume, because agents create many spans per request, and sample where you can.