llm-evaluation repositories

Trending repositories tagged llm-evaluation, ranked by trending score. Updated daily.

  1. 1
    Continuum-AI-Corp/OrcaReplay

    OrcaReplay — Time travel for AI agents. Record, replay, fork, and debug any agent run with any model. Built by the OrcaRouter.ai team.

    GitHub · TypeScript · 1,038 stars · 166 forks · +730 stars in 24 hours · Trending score 5.14

  2. 2
    NVIDIA/garak

    the LLM vulnerability scanner

    GitHub · Python · 10,091 stars · 1,491 forks · +185 stars in 24 hours · Trending score 4.03

  3. 3
    ifixai-ai/iFixAi

    Independent Auditing of AI Agents. Run by human or the agent itself, to answer the most crucial question in the AI Agent Economy. Is the agent doing what is supposed to do? With iFixAi you can have this answer in less than 120 seconds.

    GitHub · Python · 46,281 stars · 1,828 forks · +76 stars in 24 hours · Trending score 4.01

  4. 4
    langfuse/langfuse

    🪢 Open source AI engineering platform: AI Gateway, agent observability, evals, and prompt/skill management, built on OpenTelemetry. One open platform for every AI call and every agent. Self-host or use Langfuse Cloud.

    GitHub · TypeScript · 37,664 stars · 4,277 forks · +85 stars in 24 hours · Trending score 3.85

  5. 5
    promptfoo/promptfoo

    Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

    GitHub · TypeScript · 27,483 stars · 2,800 forks · +65 stars in 24 hours · Trending score 3.79

  6. 6
    jackson-cmd/risk-aware-trading-agents

    AI trading agents for US stocks with news backtesting that can't see the future: a public LLM trading agent leaderboard (rta-bench-1), frozen point-in-time 2020/2022/2026 markets (prices, VIX, news), a look-ahead check on every run, LLM agents over MCP, and paper/live trading on 10 brokers. 风险感知美股交易智能体 · 新闻回测 · LLM 榜单

    GitHub · Python · 317 stars · 80 forks · +62 stars in 24 hours · Trending score 3.67

  7. 7
    confident-ai/deepeval

    The LLM Evaluation Framework

    GitHub · Python · 19,534 stars · 2,219 forks · +84 stars in 24 hours · Trending score 3.50

  8. 8
    mlflow/mlflow

    The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

    GitHub · Python · 28,905 stars · 6,640 forks · +40 stars in 24 hours · Trending score 3.23

  9. 9
    Tencent/AI-Infra-Guard

    A full-stack AI Red Teaming platform securing AI ecosystems via Agent Scan, Skills Scan, MCP scan, AI Infra scan and LLM jailbreak evaluation.

    GitHub · Python · 7,594 stars · 780 forks · +45 stars in 24 hours · Trending score 3.20

  10. 10
    Arize-ai/phoenix

    AI Observability & Evaluation

    GitHub · Python · 12,363 stars · 1,319 forks · +26 stars in 24 hours · Trending score 3.00

  11. 11
    comet-ml/opik

    Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

    GitHub · Java · 23,460 stars · 1,964 forks · +10 stars in 24 hours · Trending score 2.61

  12. 12
    bottomless/agent-duel

    A desktop app for head-to-head coding agent battles, with single-agent mode and free inference. Pick the winning agent and continue from its work.

    GitHub · TypeScript · 69 stars · 21 forks · +8 stars in 24 hours · Trending score 2.18

  13. 13
    phuryn/bug-hunt-bench

    Bug Hunt Bench: 105 real bugs in two production repos, frontier coding models (GPT-6, Claude, Grok, Gemini, DeepSeek...) find and fix them in their own CLI, graded blind. Live leaderboard + every receipt.

    GitHub · JavaScript · 219 stars · 10 forks · +14 stars in 24 hours · Trending score 2.17

  14. 14
    wanghao-SunSky/cs329z-chinese

    斯坦福 Stanford CS329Z Engineering AI Agents 中文教程:面向 Python 开发者的 AI 智能体课程,19 讲与 4 项实践,涵盖 LLM、RAG、工具调用、记忆、多智能体、评测与安全。完整 Markdown、在线与离线 HTML。Chinese course notes.

    GitHub · HTML · 637 stars · 78 forks · +8 stars in 24 hours · Trending score 2.08

  15. 15
    Future-Proof-DS/interactive-ai-ml-tools

    Free interactive tools for data scientists to learn MLOps, ML system design and agentic analytics: a production ML system map, ML project structure, data drift simulator, agentic analytics system map and AI evals.

    GitHub · 295 stars · 28 forks · +5 stars in 24 hours · Trending score 1.98

  16. 16
    lmnr-ai/lmnr

    Laminar - open-source observability platform purpose-built for AI agents. YC S24.

    GitHub · TypeScript · 3,510 stars · 262 forks · +8 stars in 24 hours · Trending score 1.85

  17. 17
    Q00/ouroboros

    Agent OS: the agent gets smarter on its own. We just hold the line: Interview-gated, staged evaluation, budgeted evolution loop. MCP server, 14 runtimes: Claude Code, Codex CLI, Gemini CLI, OpenCode, Copilot, Kiro and more.

    GitHub · Python · 6,385 stars · 635 forks · +8 stars in 24 hours · Trending score 1.78

  18. 18
    Wendystar0628/AI-High-Pressure-Challenge

    AI十项全能·定时挑战赛 | AI Decathlon · Timed Challenge: a timed benchmark for AI coding agents — 13 tasks across 10 categories (Lean proofs, competition math, CUDA/CPU optimization, game AI, 3D/web apps, nested OS), 60–100 min each, no human help.

    GitHub · JavaScript · 65 stars · 4 forks · +24 stars in 24 hours · Trending score 1.59

  19. 19
    future-agi/future-agi

    Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.

    GitHub · Python · 2,288 stars · 725 forks · +2 stars in 24 hours · Trending score 1.55

  20. 20
    jeinlee1991/chinese-llm-benchmark

    非线智能 NoneLinear - ReLE评测:中文AI大模型能力评测(持续更新):目前已囊括374个大模型,覆盖chatgpt、gpt-5.4、谷歌gemini-3.1-pro、Claude-4.6、文心ERNIE-X1.1、ERNIE-5.0、qwen3.6-max、qwen3.6-plus、百川、讯飞星火、商汤senseChat等商用模型, 以及step3.5-flash、kimi-k2.6、ernie4.5、MiniMax-M2.7、deepseek-v4、Qwen3.6、llama4、智谱GLM-5.1、MiMo-V2、LongCat、gemma4、mistral等开源大模型。不仅提供排行榜,也提供规模超200万的大模型缺陷库!方便广大社区研究分析、改进大模型。

    GitHub · 6,522 stars · 261 forks · +2 stars in 24 hours · Trending score 1.31

  21. 21
    intel/dokimi

    Dokimi (Greek δοκιμή, "test, proof") measures whether a skill, MCP doc source, agent profile or repository guidance actually helps a coding agent: paired on/off arm comparisons, novel-vs-known content analysis, and repository scans with graded agent runs.

    GitHub · Python · 39 stars · 13 forks · +1 stars in 24 hours · Trending score 1.30

  22. 22
    truera/trulens

    Evaluation and Tracking for LLM Experiments and AI Agents

    GitHub · Python · 3,659 stars · 397 forks · 0 stars in 24 hours · Trending score 1.28

  23. 23
    ljxpython/ai-agent-platform

    企业级 AI Agent 平台底座 · 基于 LangGraph 生态二次开发,开箱即用(FastAPI + Vue 3 + 持续评测飞轮 + 原生知识中台 + 全链路可观测 + 智能排障助手 + 沙箱工作区)

    GitHub · Python · 159 stars · 29 forks · +3 stars in 24 hours · Trending score 1.27

  24. 24
    overmind-core/overmind

    The platform for continuously improving AI agents.

    GitHub · Python · 766 stars · 76 forks · +4 stars in 24 hours · Trending score 1.26