ai-evaluation repositories
Trending repositories tagged ai-evaluation, ranked by trending score. Updated daily.
-
1
ifixai-ai/iFixAi
Independent Auditing of AI Agents. Run by human or the agent itself, to answer the most crucial question in the AI Agent Economy. Is the agent doing what is supposed to do? With iFixAi you can have this answer in less than 120 seconds.
GitHub · Python · 46,346 stars · 1,828 forks · +76 stars in 24 hours · Trending score 4.01
-
2
microsoft/PyRIT
Open-source AI red teaming toolkit from Microsoft. Test generative AI systems with automated scans, adaptive multi-turn attacks, and a web UI.
GitHub · Python · 4,992 stars · 1,092 forks · +42 stars in 24 hours · Trending score 3.34
-
3
DawitLemmaBalcha/code-review-bench
Review benchmark for model-generated code patches: six tasks across Python, JavaScript, Go and Java with hidden tests, a fixed rubric and a Docker runner.
GitHub · Python · 14 stars · 2 forks · +7 stars in 24 hours · Trending score 1.07
-
4
huggingface/cadgenbench
A benchmark for AI-driven CAD generation and editing
GitHub · Python · 156 stars · 14 forks · 0 stars in 24 hours · Trending score 1.06
-
5
adhit-r/fairmind
Open-source AI assurance workbench for models and agents: map governance frameworks (AIUC-1, NIST AI RMF, EU AI Act, DPDP) to controls, collect evaluation evidence, review findings and produce audit-ready evidence trails.
GitHub · Python · 17 stars · 16 forks · +1 stars in 24 hours · Trending score 0.80
-
6
ARSI-LAB/ARSI-Evaluator
Many ways to evaluate. One framework to bring them together. Unify benchmarks, agent evaluations, and performance testing across coding, terminal, office, web, and more.
GitHub · TypeScript · 41 stars · 7 forks · 0 stars in 24 hours · Trending score 0.41
-
7
mahmoudrabie/agentic-ai
Agentic AI research papers, benchmarks, frameworks, and tools curated across 24 domains.
GitHub · 192 stars · 7 forks · 0 stars in 24 hours · Trending score 0.38
-
8
microsoft/scope
Scope | Agentic Experience evaluation platform (Research preview)
GitHub · TypeScript · 90 stars · 32 forks · 0 stars in 24 hours · Trending score 0.36
-
9
OpenAgentHQ/openagent-eval
Local-first evaluation framework for RAG systems and AI Agents. 18+ metrics, CLI + SDK, framework-agnostic. The pytest of AI evaluation.
GitHub · Python · 16 stars · 32 forks · 0 stars in 24 hours · Trending score 0.32
-
10
rudycelekli/rudycelekli
Profile: proof-driven AI systems, evaluation, and agent reliability
GitHub · Python · 5 stars · 1 forks · 0 stars in 24 hours · Trending score 0.19
-
11
shsridhar-beep/svgap
Evidence profiles for AI-generated RTL that expose what functional benchmarks miss. Seeking collaborators, design partners, and validation
GitHub · Python · 21 stars · 10 forks · 0 stars in 24 hours · Trending score 0.10
-
12
humane-intelligence/hi-oss-ai-red-teaming
A platform for running human-centered red teaming evaluations on AI models and systems.
GitHub · Python · 19 stars · 3 forks · 0 stars in 24 hours · Trending score 0.10
-
13
Claire1217/benchmark-radar
Track newly released AI benchmarks and their real-world adoption momentum.
GitHub · HTML · 11 stars · 0 forks · −1 stars in 24 hours · Trending score 0.09
-
14
SRCarlo/genai-course
A hands-on GenAI Engineering roadmap from LLM fundamentals to production AI systems. Covers prompt engineering, RAG, Agentic AI, LLM evaluation, observability, AI security, LLMOps, production deployment, CI/CD, multimodal AI, MCP, cloud, and AI system design through practical projects and real-world architecture.
GitHub · JavaScript · 5 stars · 0 forks · 0 stars in 24 hours · Trending score 0.08
-
15
Pro-GenAI/Agent-Action-Guard
🛡️ Safe AI Agents through Action Classifier
GitHub · Python · 7 stars · 8 forks · −2 stars in 24 hours · Trending score 0.08
-
16
MrKyaw/MrKyaw
Principal AI Engineer & Forward-Deployed AI Architect | Enterprise & Solution Architecture | Agentic AI, Multi-Agent Systems, RAG & MCP | MLOps/LLMOps, Evaluation, Observability & Responsible AI | Customer-focused discovery, hands-on delivery, measurable business outcomes. PRIME: build, measure, operate.
GitHub · 5 stars · 1 forks · 0 stars in 24 hours · Trending score 0.08
-
17
techtheist/knowledgedrift
KnowledgeDrift - Second Generation AI Memory benchmark
GitHub · Rust · 17 stars · 1 forks · 0 stars in 24 hours · Trending score 0.07
-
18
ARTPARK-SAHAI-ORG/calibrate
Core engine behind Calibrate, a framework for evaluating AI agents: speech-to-text, text-to-speech, LLM evaluation, end-to-end simulations
GitHub · JavaScript · 24 stars · 7 forks · 0 stars in 24 hours · Trending score 0.04
-
19
cvs-health/uqlm
[JMLR 2026] UQLM: A Python package for LLM hallucination detection using uncertainty quantification
GitHub · Python · 1,217 stars · 140 forks · 0 stars in 24 hours · Trending score 0.02
-
20
letscooktech-studio/outcomelab
Open-source CLI for local HTTP API regression testing. Check webhook idempotency and invoice persistence in development and CI.
GitHub · Python · 7 stars · 2 forks · 0 stars in 24 hours · Trending score 0.00
-
21
evaleval/every_eval_ever
Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.
GitHub · Python · 138 stars · 53 forks · 0 stars in 24 hours · Trending score 0.00
-
22
kwadwoadu/modelfit
Benchmark LLMs on your own codebase. Repo-specific probes, blind rubric-based judging, and correctness-first rankings.
GitHub · Shell · 6 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00
-
23
cimo-labs/cje
Causal Judge Evaluation: calibrate LLM-as-judge scores against oracle labels with valid uncertainty.
GitHub · Python · 46 stars · 8 forks · 0 stars in 24 hours · Trending score 0.00
-
24
taoAIGC/AICompare
one click to open multi AI sites | 一键打开多个 AI 站点,查看 AI 结果
GitHub · JavaScript · 126 stars · 15 forks · 0 stars in 24 hours · Trending score 0.00