agent-evaluation repositories

Trending repositories tagged agent-evaluation, ranked by trending score. Updated daily.

  1. 1
    NVIDIA/SkillEvaluator

    Multi-tier framework for evaluating AI agent skills and plugins with quality gates, security scanning, harness compatibility checks, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior. Ships reusable CI templates that bring every check to any repository.

    GitHub · Python · 4,234 stars · 549 forks · +794 stars in 24 hours · Trending score 5.61

  2. 2
    ifixai-ai/iFixAi

    Independent Auditing of AI Agents. Run by human or the agent itself, to answer the most crucial question in the AI Agent Economy. Is the agent doing what is supposed to do? With iFixAi you can have this answer in less than 120 seconds.

    GitHub · Python · 46,345 stars · 1,828 forks · +76 stars in 24 hours · Trending score 4.01

  3. 3
    reacher-z/Awesome-CUA

    Open, source-backed catalog of computer-use agents: reviewed papers, GUI models, datasets, benchmarks, tools, and reusable JSONL/SQLite data.

    GitHub · Python · 1,043 stars · 150 forks · +54 stars in 24 hours · Trending score 3.38

  4. 4
    TIGER-AI-Lab/WebsiteBench

    Can AI agents rebuild websites through browser exploration?

    GitHub · Python · 560 stars · 49 forks · +39 stars in 24 hours · Trending score 3.25

  5. 5
    alibaba/skill-up

    Evaluate Agent Skills and environments. Evolve Skills with evidence.

    GitHub · Go · 1,508 stars · 134 forks · +8 stars in 24 hours · Trending score 2.20

  6. 6
    oriliz/agent-crop

    给 AI 编程助手一根马鞭:本地记录时间去向,把复盘写回 agent 指令/技能/工具;改没改好,用数据说话。 | A whip for coding agents: local timing, red-line nudges, and an evidence-backed, auditable self-improvement loop. DSH / Claude Code / Codex.

    GitHub · Python · 168 stars · 0 forks · +45 stars in 24 hours · Trending score 1.86

  7. 7
    Valen-Team/Roller

    Turn agent runs into training data. Roller rolls out Claude Code, Codex, OpenClaw and four other Harnesses in disposable Docker containers, grades benchmark tasks in place, and exports SFT-ready trajectories that keep reasoning, images and compacted-context segments.

    GitHub · Python · 109 stars · 0 forks · +37 stars in 24 hours · Trending score 1.78

  8. 8
    dolev31/ProactiveInquirer

    Asking for What Was Never Requested: horizontal and vertical proactivity in LLM agents. Need-graph metrics (no LLM judge) and Q&D, which trains a questioner from the consequences of its questions.

    GitHub · Python · 22 stars · 0 forks · +10 stars in 24 hours · Trending score 1.63

  9. 9
    future-agi/future-agi

    Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.

    GitHub · Python · 2,288 stars · 725 forks · +2 stars in 24 hours · Trending score 1.55

  10. 10
    truera/trulens

    Evaluation and Tracking for LLM Experiments and AI Agents

    GitHub · Python · 3,659 stars · 397 forks · 0 stars in 24 hours · Trending score 1.28

  11. 11
    scaleapi/agentenv-framework

    Creating realistic RL environments requires collaboration between researchers, engineers, and domain experts across many dimensions: artifacts, environments tools, dynamism of the environment, reproducibility, and more. There is no open source framework for building these environments effectively. Until now.

    GitHub · Python · 447 stars · 64 forks · +1 stars in 24 hours · Trending score 1.25

  12. 12
    NVIDIA-NeMo/nemo-helix

    NeMo Helix: Make the agents you ship faster, more accurate, and safer.

    GitHub · Python · 150 stars · 60 forks · +1 stars in 24 hours · Trending score 1.11

  13. 13
    hidai25/eval-view

    Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

    GitHub · Python · 142 stars · 26 forks · +3 stars in 24 hours · Trending score 1.00

  14. 14
    chirpz-ai/pandaprobe

    open source agent engineering platform: traces, evals, and metrics to debug and improve your AI agents. Integrates with LangGraph, CrewAI, Claude Agent SDK, and more.

    GitHub · Python · 808 stars · 122 forks · 0 stars in 24 hours · Trending score 0.97

  15. 15
    anhermon/awesome-agent-observability

    Curated list of LLM and AI agent observability and evaluation tools: tracing, evals, LLM-as-judge, guardrails, OpenTelemetry GenAI, MCP.

    GitHub · Python · 65 stars · 36 forks · +1 stars in 24 hours · Trending score 0.82

  16. 16
    amitshekhariitbhu/ai-agents-tutorial

    Learn AI Agents step by step, from scratch - from function calling to agent loops to multi-agent systems, orchestration, and evaluation.

    GitHub · 214 stars · 51 forks · +1 stars in 24 hours · Trending score 0.82

  17. 17
    OpenBMB/EC2-Bench

    EC2-Bench (EC²-Bench): benchmark for edge-cloud collaboration in long-horizon LLM agents — task performance, resource usage & privacy exposure. arXiv 2610.17816

    GitHub · Python · 24 stars · 0 forks · +2 stars in 24 hours · Trending score 0.81

  18. 18
    ORO-AI/oro

    ORO AI is an applied AI research lab. This repo is its open agent competition on Bittensor Subnet 15, scored on ORO Bench: validator and miner code.

    GitHub · Python · 24 stars · 27 forks · +1 stars in 24 hours · Trending score 0.77

  19. 19
    coze-dev/coze-loop

    Next-generation AI Agent Optimization Platform: Cozeloop addresses challenges in AI agent development by providing full-lifecycle management capabilities from development, debugging, and evaluation to monitoring.

    GitHub · Go · 5,800 stars · 804 forks · +3 stars in 24 hours · Trending score 0.70

  20. 20
    Giskard-AI/giskard-oss

    🐢 Open-Source Evaluation & Testing library for LLM Agents

    GitHub · Python · 5,960 stars · 577 forks · 0 stars in 24 hours · Trending score 0.49

  21. 21
    yzhao062/awesome-auditable-ai

    Auditing AI agents: a curated list of papers, tools, datasets, benchmarks, and standards covering reliability, monitoring, failure attribution, and decision records.

    GitHub · Python · 162 stars · 53 forks · 0 stars in 24 hours · Trending score 0.45

  22. 22
    ARSI-LAB/ARSI-Evaluator

    Many ways to evaluate. One framework to bring them together. Unify benchmarks, agent evaluations, and performance testing across coding, terminal, office, web, and more.

    GitHub · TypeScript · 41 stars · 7 forks · 0 stars in 24 hours · Trending score 0.41

  23. 23
    ChaoYue0307/awesome-loop-engineering

    🔁 Build reliable recurring AI-agent systems: 1022 resources, 22 operational patterns, 22 loop contracts, 8 runtime starters, an interactive atlas, and a structured dataset.

    GitHub · Python · 60 stars · 9 forks · 0 stars in 24 hours · Trending score 0.31

  24. 24
    eigent-ai/agenticrl-bench

    Testing agents on real agentic-RL work, from debugging to training.

    GitHub · Python · 18 stars · 0 forks · 0 stars in 24 hours · Trending score 0.27