Comet

Opik

Open source platform for tracing, evaluating and optimizing LLM apps and agents, self-hosted or on Comet's cloud.

Apache-2.0PythonTypeScriptv2.2.96

What it supports

  • Evaluates multi-step agents and tool calls: supportedEvaluates multi-step agents and tool callsDocs
  • LLM as a judge scoring: supportedLLM as a judge scoringDocs
  • Tracing: supportedTracingDocs
  • Runs in CI: supportedRuns in CIDocs

Install

pip install opik

Links

Official documentationgithub.com/comet-ml/opikHosted option: Opik CloudPricing

Other eval frameworks

Inspect AIpromptfooDeepEvalPydantic EvalsHarborLangfuseMLflow GenAI EvaluationArize PhoenixW&B WeaveRagasGoogle ADK EvaluationBraintrustLangSmith EvaluationMicrosoft Foundry EvaluationsGen AI Evaluation Service

Get the weekly agent stack update

New official MCP servers, spec changes and harness releases, checked against the source. One email a week, no fluff.

Reviewed Oct 9, 2026.