evaluations repositories

Trending repositories tagged evaluations, ranked by trending score. Updated daily.

  1. 1
    aws-samples/sample-gen-ai-evaluations-workshop

    This workshop teaches systematic approaches to evaluating Generative AI workloads for production use. You'll learn to build evaluation frameworks that go beyond basic metrics to ensure reliable model performance while optimizing cost and performance.

    GitHub · Jupyter Notebook · 82 stars · 33 forks · +16 stars in 24 hours · Trending score 2.27

  2. 2
    ktwu01/benchmark-radar

    Track 50,331+ AI benchmark, eval, dataset, and data-quality records from 41 public sources, with linked evidence and daily updates.

    GitHub · Python · 409 stars · 65 forks · 0 stars in 24 hours · Trending score 1.16

  3. 3
    AgentEvalHQ/AgentEval

    AgentEval is the comprehensive .NET toolkit for AI agent evaluation—tool usage validation, RAG quality metrics, stochastic evaluation, and model comparison—built first for Microsoft Agent Framework (MAF) and Microsoft.Extensions.AI. What RAGAS, PromptFoo and DeepEval do for Python, AgentEval does for .NET

    GitHub · C# · 160 stars · 16 forks · 0 stars in 24 hours · Trending score 0.27

  4. 4
    RailtownAI/railtracks

    An agentic framework that helps developers build resilient agentic systems

    GitHub · Python · 234 stars · 73 forks · −2 stars in 24 hours · Trending score 0.25

  5. 5
    justinwetch/SkillEval

    A visual workbench for A/B testing AI skills. Upload two skill files, run them through a batch of test prompts, and let an AI judge score the results.

    GitHub · JavaScript · 35 stars · 3 forks · 0 stars in 24 hours · Trending score 0.22

  6. 6
    lexmount/webstone

    Benchmark browsers and web tools for AI agents: Browser Trajectory Replay, Web Fetch / Search / Extract / Scraping / Crawling.

    GitHub · Python · 10 stars · 1 forks · 0 stars in 24 hours · Trending score 0.09

  7. 7
    dynatrace-oss/dt-evals

    AI evaluators CLI for your AI apps and Agents - Dynatrace AI Observability

    GitHub · TypeScript · 53 stars · 5 forks · 0 stars in 24 hours · Trending score 0.08

  8. 8
    evaleval/every_eval_ever

    Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

    GitHub · Python · 138 stars · 53 forks · 0 stars in 24 hours · Trending score 0.00

  9. 9
    priyankark/jev-state

    Build and regression-test conversational state machines powered by Jev. Inspect decisions, capture failing conversations as tests, and export runnable TypeScript for your app.

    GitHub · TypeScript · 10 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00

  10. 10
    dreadnode/AIRTBench-Code

    Code Repository for: AIRTBench: Measuring Autonomous AI Red Teaming Capabilities in Language Models

    GitHub · Jupyter Notebook · 111 stars · 16 forks · 0 stars in 24 hours · Trending score 0.00