Observability Rankings

MLflow

Observability· mlflow.org

Open source

MLflow is an open-source AI engineering platform with GenAI evaluation, tracing, monitoring, and optimization workflows for agents, LLM apps, and models.

AgentOps

Observability· agentops.ai

Freemium

Developer platform for tracing, debugging, and deploying AI agents with SDK-based observability, session replay, and framework integrations.

Promptfoo

Observability· promptfoo.dev

Open source

Open-source testing, red-teaming, and evaluation framework for prompts, RAG, and agents.

Opik

Observability· comet.com

Open source

Open-source LLM observability and evaluation platform from Comet for tracing, prompt management, testing, and agent optimization.

Evidently

Observability· evidentlyai.com

Freemium

Open-source observability and evaluation framework for monitoring ML and LLM applications with tests and metrics.

Guardrails AI

Observability· guardrailsai.com

Freemium

LLM validation and guardrail framework for checking structured outputs, policy compliance, and production AI failure modes before release.

Giskard

Observability· giskard.ai

Freemium

Open-source evaluation and testing library for LLM agents, RAG systems, and AI risk checks.

AnomalyArmor

Observability· anomalyarmor.ai

Freemium

Data observability platform with hosted and local MCP access for alerts, freshness, schema drift, lineage, and quality checks.

Helicone

Observability· helicone.ai

Freemium

Open-source AI gateway and LLM observability platform for logging, tracing, routing, evaluation, caching, and cost monitoring across model providers and agent stacks.

BoundFlow

Observability· boundflow.github.io

Open source

Open-source control plane for governed AI agent execution with lifecycle policies, approvals, durable runs, audit receipts, and observability.

Future AGI

Observability· futureagi.com

Freemium

Open-source agent engineering platform that combines evaluations, observability, experiments, and guardrails so teams can ship and improve production AI agents faster.

Kiln

Observability· kiln.tech

Open source

Open-source AI engineering workbench for model evaluation, dataset curation, fine-tuning, and app reliability workflows.

OpenPipe ART

Observability· art.openpipe.ai

Open source

Agent Reinforcement Trainer for improving LLM agent reliability with GRPO-based reinforcement learning workflows.

Ragas

Observability· ragas.io

Open source

Evaluation framework for testing RAG and LLM applications with metrics, synthetic test data, and CI-friendly quality checks.

Portkey

Observability· portkey.ai

Freemium

AI gateway and observability layer for routing, guardrails, caching, logging, and spend control across LLM providers behind one API.

Inspect AI

Observability· inspect.aisi.org.uk

Open source

Framework for large language model evaluations, task definitions, solvers, scoring, and reproducible eval runs.

Laminar

Observability· laminar.sh

Freemium

Open-source observability, tracing, and evaluation platform built for AI agents.

Phoenix

Observability· arize.com

Open source

Open source AI observability and evaluation platform for tracing LLM applications, running evals, and debugging agent behavior with OpenTelemetry.

Braintrust

Observability· braintrustdata.com

Freemium

AI observability and evals platform for tracing production systems, running experiments, and catching regressions.

LangWatch

Observability· langwatch.ai

Freemium

The complete LLMOps platform for agent testing, evaluations, prompt management, and production observability across AI applications.

TensorZero

Observability· tensorzero.com

Open source

Open-source LLM application stack for observability, evaluations, prompt optimization, and controlled model gateway workflows.

MCPJam Inspector

Observability· mcpjam.com

Freemium

Testing and debugging workbench for MCP servers, MCP apps, and ChatGPT apps.

OpenLIT

Observability· openlit.io

Freemium

OpenTelemetry-native AI engineering platform for LLM observability, evaluations, guardrails, and prompt operations.

OpenInference

Observability· arize-ai.github.io

Open source

OpenTelemetry semantic conventions and instrumentation packages for tracing LLM calls, agent steps, tool use, and retrieval workloads.

DeepEval

Observability· deepeval.com

Open source

Open-source evaluation framework for testing LLM apps, agents, RAG systems, and prompts with repeatable metrics, regression checks, and CI-friendly workflows.

Traceloop OpenLLMetry

Observability· traceloop.com

Open source

OpenTelemetry-based open-source instrumentation for tracing, monitoring, and debugging LLM applications across frameworks and providers.

Pydantic Logfire

Observability· logfire-us.pydantic.dev

Freemium

Observability platform for production Python, LLM, and agent systems from the Pydantic team.

TruLens

Observability· trulens.org

Open source

Open-source evaluation and tracking toolkit for measuring LLM app quality, feedback functions, and RAG behavior.

Auralogs

Observability· auralogs.ai

Freemium

Agent-focused observability platform that exposes structured production logs and AI analyses to Codex, Claude, Cursor, and other clients through read-only MCP.

Alluvia

Observability· github.com

Open source

Local-first CLI that mines Claude Code, Cursor, and ChatGPT histories into searchable engineering insights without uploading conversations.

Scorable by Root Signals

Observability· scorable.ai

Freemium

LLM and agent evaluation platform with reusable scorers, test datasets, and SDKs for measuring production AI behavior.

Snyk Agent Scan

Observability· github.com

Open source

Open-source Snyk scanner for AI agents, MCP servers, and agent skills that detects prompt injection, tool poisoning, toxic flows, and agent supply-chain risks.

mcpsnoop

Observability· github.com

Open source

Transparent MCP proxy and terminal inspector for observing tool calls, payloads, latency, and failures between clients and servers.

Weights & Biases Weave

Observability· wandb.ai

Open source

LLM observability and evaluation toolkit for tracing, debugging, and improving production agents and AI applications.