Opik: Open-Source LLM Observability, Evaluation & AI Agent Tracing

Opik is the open-source LLM observability and evaluation platform for AI agent tracing, LLM evaluation, prompt management, and production monitoring. Built by Comet. Apache-2.0 licensed, free to self-host the full platform, with 20,000+ GitHub stars.

Website • Slack Community • Twitter • Changelog • Documentation

_{Last updated: 2026-07-17}

🚀 What is Opik?

Opik covers the full LLM application lifecycle, from the first trace in development to production monitoring, for teams building LLM apps and AI agents. Key offerings include:

• AI Agent Tracing & Observability: Deep tracing of LLM calls, conversation logging, and agent activity, with full trace trees for multi-step agents and tool calls.
• LLM Evaluation: Datasets, experiments, and LLM-as-a-judge metrics for hallucination detection, moderation, and RAG assessment.
• Prompt & Agent Optimization: The Opik Agent Optimizer SDK to improve prompts and agents.
• Production-Ready Monitoring: Scalable dashboards and online evaluation rules.
• Opik Guardrails: Features to help you implement safe and responsible AI practices.
• CI/CD Evaluation: A PyTest integration to test LLM pipelines on every commit.

Key capabilities include:

• Development & Tracing:

  • Track all LLM calls and traces with detailed context during development and in production (Quickstart).
  • Extensive 3rd-party integrations for easy observability: Seamlessly integrate with a growing list of frameworks, supporting many of the largest and most popular ones natively (including recent additions like Google ADK, Autogen, and Flowise AI). (Integrations)
  • Annotate traces and spans with feedback scores via the Python SDK or the UI.
  • Experiment with prompts and models in the Prompt Playground.

• Evaluation & Testing:

  • Automate your LLM application evaluation with Datasets and Experiments.
  • Leverage powerful LLM-as-a-judge metrics for complex tasks like hallucination detection, moderation, and RAG assessment (Answer Relevance, Context Precision).
  • Integrate evaluations into your CI/CD pipeline with our PyTest integration.

• Production Monitoring & Optimization:

  • Log high volumes of production traces: Opik is designed for scale (40M+ traces/day).
  • Monitor feedback scores, trace counts, and token usage over time in the Opik Dashboard.
  • Utilize Online Evaluation Rules with LLM-as-a-Judge metrics to identify production issues.
  • Leverage Opik Agent Optimizer and Opik Guardrails to continuously improv