evaluation repositories

Trending repositories tagged evaluation, ranked by trending score. Updated daily.

  1. 1
    NVIDIA/SkillEvaluator

    Multi-tier framework for evaluating AI agent skills and plugins with quality gates, security scanning, harness compatibility checks, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior. Ships reusable CI templates that bring every check to any repository.

    GitHub · Python · 4,195 stars · 546 forks · +794 stars in 24 hours · Trending score 5.61

  2. 2
    Tencent/WeKnora

    Open-source LLM knowledge platform: turn raw documents into a queryable RAG, an autonomous reasoning agent, and a self-maintaining Wiki.

    GitHub · Go · 37,864 stars · 5,280 forks · +151 stars in 24 hours · Trending score 4.32

  3. 3
    langfuse/langfuse

    🪢 Open source AI engineering platform: AI Gateway, agent observability, evals, and prompt/skill management, built on OpenTelemetry. One open platform for every AI call and every agent. Self-host or use Langfuse Cloud.

    GitHub · TypeScript · 37,664 stars · 4,277 forks · +85 stars in 24 hours · Trending score 3.85

  4. 4
    promptfoo/promptfoo

    Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

    GitHub · TypeScript · 27,483 stars · 2,800 forks · +65 stars in 24 hours · Trending score 3.79

  5. 5
    mlflow/mlflow

    The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

    GitHub · Python · 28,905 stars · 6,640 forks · +40 stars in 24 hours · Trending score 3.23

  6. 6
    alchaincyf/darwin-skill

    达尔文.skill —— 一个让你的Skill无限进化的系统:评估→改进→测试→保留或回滚 | Autoresearch-inspired autonomous skill optimization for Claude Code. Evaluate, improve, test, keep or revert.

    GitHub · HTML · 7,004 stars · 725 forks · +39 stars in 24 hours · Trending score 3.08

  7. 7
    comet-ml/opik

    Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

    GitHub · Java · 23,460 stars · 1,964 forks · +10 stars in 24 hours · Trending score 2.61

  8. 8
    zaxiu88888/issue-rag

    RAG over GitHub issues, with a hand-reviewed eval set and an audit of the judge itself. Reports the interval, not just the best number.

    GitHub · Python · 483 stars · 6 forks · +16 stars in 24 hours · Trending score 2.33

  9. 9
    BUAASubnet/UBAA

    更好的智慧北航. Make BUAA Great Again!

    GitHub · Dart · 368 stars · 26 forks · +13 stars in 24 hours · Trending score 2.29

  10. 10
    open-compass/opencompass

    OpenCompass is an LLM evaluation platform, supporting a wide range of models from OpenAI, Anthropic, Gemini, Qwen, GLM, DeepSeek, etc, across 100+ datasets covering knowledge, reasoning, coding, science, language, long-context, and safety.

    GitHub · Python · 7,601 stars · 915 forks · +17 stars in 24 hours · Trending score 2.26

  11. 11
    modelscope/evalscope

    A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.

    GitHub · Python · 3,714 stars · 585 forks · +5 stars in 24 hours · Trending score 2.01

  12. 12
    didi/helixharness

    HelixHarness 提供完整的 AI Coding 工程化基础能力,还支持不同企业和团队结合自身的业务知识、工程规范和研发流程,持续构建和演进属于自己的专属 Harness

    GitHub · Python · 453 stars · 62 forks · +8 stars in 24 hours · Trending score 1.99

  13. 13
    lmnr-ai/lmnr

    Laminar - open-source observability platform purpose-built for AI agents. YC S24.

    GitHub · TypeScript · 3,510 stars · 262 forks · +8 stars in 24 hours · Trending score 1.85

  14. 14
    Valen-Team/Roller

    Turn agent runs into training data. Roller rolls out Claude Code, Codex, OpenClaw and four other Harnesses in disposable Docker containers, grades benchmark tasks in place, and exports SFT-ready trajectories that keep reasoning, images and compacted-context segments.

    GitHub · Python · 106 stars · 0 forks · +37 stars in 24 hours · Trending score 1.78

  15. 15
    run-llama/ParseBench

    ParseBench - A Document Parsing Benchmark for AI Agents

    GitHub · Python · 648 stars · 140 forks · +5 stars in 24 hours · Trending score 1.76

  16. 16
    NVIDIA-NeMo/Gym

    Evaluate and improve models and agents using environments

    GitHub · Python · 1,346 stars · 485 forks · +3 stars in 24 hours · Trending score 1.72

  17. 17
    MCPJam/inspector

    Testing and evaluation platform to chat, inspect, and debug MCP servers, MCP apps, and ChatGPT apps.

    GitHub · TypeScript · 2,331 stars · 305 forks · +3 stars in 24 hours · Trending score 1.57

  18. 18
    langwatch/langwatch

    Open-Source Agent Observability & Evals for AI Platform teams. Trace, test, route and govern every agent & coding assistant in the company.

    GitHub · TypeScript · 5,047 stars · 434 forks · +3 stars in 24 hours · Trending score 1.55

  19. 19
    strands-agents/evals

    A comprehensive evaluation framework for AI agents and LLM applications.

    GitHub · Python · 261 stars · 87 forks · +3 stars in 24 hours · Trending score 1.47

  20. 20
    intel/dokimi

    Dokimi (Greek δοκιμή, "test, proof") measures whether a skill, MCP doc source, agent profile or repository guidance actually helps a coding agent: paired on/off arm comparisons, novel-vs-known content analysis, and repository scans with graded agent runs.

    GitHub · Python · 39 stars · 13 forks · +1 stars in 24 hours · Trending score 1.30

  21. 21
    beipo-ai/NorthCUA

    Turn intent into work. Open-source Desktop/Web agent harness with long-horizon memory, live evaluation and recoverable batches.

    GitHub · Python · 128 stars · 7 forks · +4 stars in 24 hours · Trending score 1.28

  22. 22
    Helicone/helicone

    🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

    GitHub · TypeScript · 6,278 stars · 704 forks · 0 stars in 24 hours · Trending score 1.25

  23. 23
    Kiln-AI/Kiln

    Build, Evaluate, and Optimize AI Systems. Includes evals, RAG, agents, fine-tuning, synthetic data generation, dataset management, MCP, and more.

    GitHub · Python · 5,286 stars · 406 forks · +1 stars in 24 hours · Trending score 1.23

  24. 24
    awslabs/uaef

    Universal Agent Evaluation Framework (UAEF) is a framework-agnostic evaluation system for AI agents. Invoke any agent and score it on multiple metrics spanning tool calling, response quality, safety, performance, and reasoning. Track experiments against baselines, catch regressions automatically, and get LLM-generated insights.

    GitHub · Python · 93 stars · 8 forks · 0 stars in 24 hours · Trending score 1.22