evaluations repositories
Trending repositories tagged evaluations, ranked by trending score. Updated daily.
-
1
aws-samples/sample-gen-ai-evaluations-workshop
This workshop teaches systematic approaches to evaluating Generative AI workloads for production use. You'll learn to build evaluation frameworks that go beyond basic metrics to ensure reliable model performance while optimizing cost and performance.
GitHub · Jupyter Notebook · 82 stars · 33 forks · +16 stars in 24 hours · Trending score 2.27
-
2
ktwu01/benchmark-radar
Track 50,331+ AI benchmark, eval, dataset, and data-quality records from 41 public sources, with linked evidence and daily updates.
GitHub · Python · 409 stars · 65 forks · 0 stars in 24 hours · Trending score 1.16
-
3
AgentEvalHQ/AgentEval
AgentEval is the comprehensive .NET toolkit for AI agent evaluation—tool usage validation, RAG quality metrics, stochastic evaluation, and model comparison—built first for Microsoft Agent Framework (MAF) and Microsoft.Extensions.AI. What RAGAS, PromptFoo and DeepEval do for Python, AgentEval does for .NET
GitHub · C# · 160 stars · 16 forks · 0 stars in 24 hours · Trending score 0.27
-
4
RailtownAI/railtracks
An agentic framework that helps developers build resilient agentic systems
GitHub · Python · 234 stars · 73 forks · −2 stars in 24 hours · Trending score 0.25
-
5
justinwetch/SkillEval
A visual workbench for A/B testing AI skills. Upload two skill files, run them through a batch of test prompts, and let an AI judge score the results.
GitHub · JavaScript · 35 stars · 3 forks · 0 stars in 24 hours · Trending score 0.22
-
6
lexmount/webstone
Benchmark browsers and web tools for AI agents: Browser Trajectory Replay, Web Fetch / Search / Extract / Scraping / Crawling.
GitHub · Python · 10 stars · 1 forks · 0 stars in 24 hours · Trending score 0.09
-
7
dynatrace-oss/dt-evals
AI evaluators CLI for your AI apps and Agents - Dynatrace AI Observability
GitHub · TypeScript · 53 stars · 5 forks · 0 stars in 24 hours · Trending score 0.08
-
8
evaleval/every_eval_ever
Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.
GitHub · Python · 138 stars · 53 forks · 0 stars in 24 hours · Trending score 0.00
-
9
priyankark/jev-state
Build and regression-test conversational state machines powered by Jev. Inspect decisions, capture failing conversations as tests, and export runnable TypeScript for your app.
GitHub · TypeScript · 10 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00
-
10
dreadnode/AIRTBench-Code
Code Repository for: AIRTBench: Measuring Autonomous AI Red Teaming Capabilities in Language Models
GitHub · Jupyter Notebook · 111 stars · 16 forks · 0 stars in 24 hours · Trending score 0.00