ai-evaluation repositories

Trending repositories tagged ai-evaluation, ranked by trending score. Updated daily.

  1. 1
    ifixai-ai/iFixAi

    Independent Auditing of AI Agents. Run by human or the agent itself, to answer the most crucial question in the AI Agent Economy. Is the agent doing what is supposed to do? With iFixAi you can have this answer in less than 120 seconds.

    GitHub · Python · 46,346 stars · 1,828 forks · +76 stars in 24 hours · Trending score 4.01

  2. 2
    microsoft/PyRIT

    Open-source AI red teaming toolkit from Microsoft. Test generative AI systems with automated scans, adaptive multi-turn attacks, and a web UI.

    GitHub · Python · 4,992 stars · 1,092 forks · +42 stars in 24 hours · Trending score 3.34

  3. 3
    DawitLemmaBalcha/code-review-bench

    Review benchmark for model-generated code patches: six tasks across Python, JavaScript, Go and Java with hidden tests, a fixed rubric and a Docker runner.

    GitHub · Python · 14 stars · 2 forks · +7 stars in 24 hours · Trending score 1.07

  4. 4
    huggingface/cadgenbench

    A benchmark for AI-driven CAD generation and editing

    GitHub · Python · 156 stars · 14 forks · 0 stars in 24 hours · Trending score 1.06

  5. 5
    adhit-r/fairmind

    Open-source AI assurance workbench for models and agents: map governance frameworks (AIUC-1, NIST AI RMF, EU AI Act, DPDP) to controls, collect evaluation evidence, review findings and produce audit-ready evidence trails.

    GitHub · Python · 17 stars · 16 forks · +1 stars in 24 hours · Trending score 0.80

  6. 6
    ARSI-LAB/ARSI-Evaluator

    Many ways to evaluate. One framework to bring them together. Unify benchmarks, agent evaluations, and performance testing across coding, terminal, office, web, and more.

    GitHub · TypeScript · 41 stars · 7 forks · 0 stars in 24 hours · Trending score 0.41

  7. 7
    mahmoudrabie/agentic-ai

    Agentic AI research papers, benchmarks, frameworks, and tools curated across 24 domains.

    GitHub · 192 stars · 7 forks · 0 stars in 24 hours · Trending score 0.38

  8. 8
    microsoft/scope

    Scope | Agentic Experience evaluation platform (Research preview)

    GitHub · TypeScript · 90 stars · 32 forks · 0 stars in 24 hours · Trending score 0.36

  9. 9
    OpenAgentHQ/openagent-eval

    Local-first evaluation framework for RAG systems and AI Agents. 18+ metrics, CLI + SDK, framework-agnostic. The pytest of AI evaluation.

    GitHub · Python · 16 stars · 32 forks · 0 stars in 24 hours · Trending score 0.32

  10. 10
    rudycelekli/rudycelekli

    Profile: proof-driven AI systems, evaluation, and agent reliability

    GitHub · Python · 5 stars · 1 forks · 0 stars in 24 hours · Trending score 0.19

  11. 11
    shsridhar-beep/svgap

    Evidence profiles for AI-generated RTL that expose what functional benchmarks miss. Seeking collaborators, design partners, and validation

    GitHub · Python · 21 stars · 10 forks · 0 stars in 24 hours · Trending score 0.10

  12. 12
    humane-intelligence/hi-oss-ai-red-teaming

    A platform for running human-centered red teaming evaluations on AI models and systems.

    GitHub · Python · 19 stars · 3 forks · 0 stars in 24 hours · Trending score 0.10

  13. 13
    Claire1217/benchmark-radar

    Track newly released AI benchmarks and their real-world adoption momentum.

    GitHub · HTML · 11 stars · 0 forks · −1 stars in 24 hours · Trending score 0.09

  14. 14
    SRCarlo/genai-course

    A hands-on GenAI Engineering roadmap from LLM fundamentals to production AI systems. Covers prompt engineering, RAG, Agentic AI, LLM evaluation, observability, AI security, LLMOps, production deployment, CI/CD, multimodal AI, MCP, cloud, and AI system design through practical projects and real-world architecture.

    GitHub · JavaScript · 5 stars · 0 forks · 0 stars in 24 hours · Trending score 0.08

  15. 15
    Pro-GenAI/Agent-Action-Guard

    🛡️ Safe AI Agents through Action Classifier

    GitHub · Python · 7 stars · 8 forks · −2 stars in 24 hours · Trending score 0.08

  16. 16
    MrKyaw/MrKyaw

    Principal AI Engineer & Forward-Deployed AI Architect | Enterprise & Solution Architecture | Agentic AI, Multi-Agent Systems, RAG & MCP | MLOps/LLMOps, Evaluation, Observability & Responsible AI | Customer-focused discovery, hands-on delivery, measurable business outcomes. PRIME: build, measure, operate.

    GitHub · 5 stars · 1 forks · 0 stars in 24 hours · Trending score 0.08

  17. 17
    techtheist/knowledgedrift

    KnowledgeDrift - Second Generation AI Memory benchmark

    GitHub · Rust · 17 stars · 1 forks · 0 stars in 24 hours · Trending score 0.07

  18. 18
    ARTPARK-SAHAI-ORG/calibrate

    Core engine behind Calibrate, a framework for evaluating AI agents: speech-to-text, text-to-speech, LLM evaluation, end-to-end simulations

    GitHub · JavaScript · 24 stars · 7 forks · 0 stars in 24 hours · Trending score 0.04

  19. 19
    cvs-health/uqlm

    [JMLR 2026] UQLM: A Python package for LLM hallucination detection using uncertainty quantification

    GitHub · Python · 1,217 stars · 140 forks · 0 stars in 24 hours · Trending score 0.02

  20. 20
    letscooktech-studio/outcomelab

    Open-source CLI for local HTTP API regression testing. Check webhook idempotency and invoice persistence in development and CI.

    GitHub · Python · 7 stars · 2 forks · 0 stars in 24 hours · Trending score 0.00

  21. 21
    evaleval/every_eval_ever

    Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

    GitHub · Python · 138 stars · 53 forks · 0 stars in 24 hours · Trending score 0.00

  22. 22
    kwadwoadu/modelfit

    Benchmark LLMs on your own codebase. Repo-specific probes, blind rubric-based judging, and correctness-first rankings.

    GitHub · Shell · 6 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00

  23. 23
    cimo-labs/cje

    Causal Judge Evaluation: calibrate LLM-as-judge scores against oracle labels with valid uncertainty.

    GitHub · Python · 46 stars · 8 forks · 0 stars in 24 hours · Trending score 0.00

  24. 24
    taoAIGC/AICompare

    one click to open multi AI sites | 一键打开多个 AI 站点,查看 AI 结果

    GitHub · JavaScript · 126 stars · 15 forks · 0 stars in 24 hours · Trending score 0.00