llm-evaluation repositories
Trending repositories tagged llm-evaluation, ranked by trending score. Updated daily.
-
1
Continuum-AI-Corp/OrcaReplay
OrcaReplay — Time travel for AI agents. Record, replay, fork, and debug any agent run with any model. Built by the OrcaRouter.ai team.
GitHub · TypeScript · 1,038 stars · 166 forks · +730 stars in 24 hours · Trending score 5.14
-
2
NVIDIA/garak
the LLM vulnerability scanner
GitHub · Python · 10,091 stars · 1,491 forks · +185 stars in 24 hours · Trending score 4.03
-
3
ifixai-ai/iFixAi
Independent Auditing of AI Agents. Run by human or the agent itself, to answer the most crucial question in the AI Agent Economy. Is the agent doing what is supposed to do? With iFixAi you can have this answer in less than 120 seconds.
GitHub · Python · 46,281 stars · 1,828 forks · +76 stars in 24 hours · Trending score 4.01
-
4
langfuse/langfuse
🪢 Open source AI engineering platform: AI Gateway, agent observability, evals, and prompt/skill management, built on OpenTelemetry. One open platform for every AI call and every agent. Self-host or use Langfuse Cloud.
GitHub · TypeScript · 37,664 stars · 4,277 forks · +85 stars in 24 hours · Trending score 3.85
-
5
promptfoo/promptfoo
Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.
GitHub · TypeScript · 27,483 stars · 2,800 forks · +65 stars in 24 hours · Trending score 3.79
-
6
jackson-cmd/risk-aware-trading-agents
AI trading agents for US stocks with news backtesting that can't see the future: a public LLM trading agent leaderboard (rta-bench-1), frozen point-in-time 2020/2022/2026 markets (prices, VIX, news), a look-ahead check on every run, LLM agents over MCP, and paper/live trading on 10 brokers. 风险感知美股交易智能体 · 新闻回测 · LLM 榜单
GitHub · Python · 317 stars · 80 forks · +62 stars in 24 hours · Trending score 3.67
-
7
confident-ai/deepeval
The LLM Evaluation Framework
GitHub · Python · 19,534 stars · 2,219 forks · +84 stars in 24 hours · Trending score 3.50
-
8
mlflow/mlflow
The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.
GitHub · Python · 28,905 stars · 6,640 forks · +40 stars in 24 hours · Trending score 3.23
-
9
Tencent/AI-Infra-Guard
A full-stack AI Red Teaming platform securing AI ecosystems via Agent Scan, Skills Scan, MCP scan, AI Infra scan and LLM jailbreak evaluation.
GitHub · Python · 7,594 stars · 780 forks · +45 stars in 24 hours · Trending score 3.20
-
10
Arize-ai/phoenix
AI Observability & Evaluation
GitHub · Python · 12,363 stars · 1,319 forks · +26 stars in 24 hours · Trending score 3.00
-
11
comet-ml/opik
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
GitHub · Java · 23,460 stars · 1,964 forks · +10 stars in 24 hours · Trending score 2.61
-
12
bottomless/agent-duel
A desktop app for head-to-head coding agent battles, with single-agent mode and free inference. Pick the winning agent and continue from its work.
GitHub · TypeScript · 69 stars · 21 forks · +8 stars in 24 hours · Trending score 2.18
-
13
phuryn/bug-hunt-bench
Bug Hunt Bench: 105 real bugs in two production repos, frontier coding models (GPT-6, Claude, Grok, Gemini, DeepSeek...) find and fix them in their own CLI, graded blind. Live leaderboard + every receipt.
GitHub · JavaScript · 219 stars · 10 forks · +14 stars in 24 hours · Trending score 2.17
-
14
wanghao-SunSky/cs329z-chinese
斯坦福 Stanford CS329Z Engineering AI Agents 中文教程:面向 Python 开发者的 AI 智能体课程,19 讲与 4 项实践,涵盖 LLM、RAG、工具调用、记忆、多智能体、评测与安全。完整 Markdown、在线与离线 HTML。Chinese course notes.
GitHub · HTML · 637 stars · 78 forks · +8 stars in 24 hours · Trending score 2.08
-
15
Future-Proof-DS/interactive-ai-ml-tools
Free interactive tools for data scientists to learn MLOps, ML system design and agentic analytics: a production ML system map, ML project structure, data drift simulator, agentic analytics system map and AI evals.
GitHub · 295 stars · 28 forks · +5 stars in 24 hours · Trending score 1.98
-
16
lmnr-ai/lmnr
Laminar - open-source observability platform purpose-built for AI agents. YC S24.
GitHub · TypeScript · 3,510 stars · 262 forks · +8 stars in 24 hours · Trending score 1.85
-
17
Q00/ouroboros
Agent OS: the agent gets smarter on its own. We just hold the line: Interview-gated, staged evaluation, budgeted evolution loop. MCP server, 14 runtimes: Claude Code, Codex CLI, Gemini CLI, OpenCode, Copilot, Kiro and more.
GitHub · Python · 6,385 stars · 635 forks · +8 stars in 24 hours · Trending score 1.78
-
18
Wendystar0628/AI-High-Pressure-Challenge
AI十项全能·定时挑战赛 | AI Decathlon · Timed Challenge: a timed benchmark for AI coding agents — 13 tasks across 10 categories (Lean proofs, competition math, CUDA/CPU optimization, game AI, 3D/web apps, nested OS), 60–100 min each, no human help.
GitHub · JavaScript · 65 stars · 4 forks · +24 stars in 24 hours · Trending score 1.59
-
19
future-agi/future-agi
Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.
GitHub · Python · 2,288 stars · 725 forks · +2 stars in 24 hours · Trending score 1.55
-
20
jeinlee1991/chinese-llm-benchmark
非线智能 NoneLinear - ReLE评测:中文AI大模型能力评测(持续更新):目前已囊括374个大模型,覆盖chatgpt、gpt-5.4、谷歌gemini-3.1-pro、Claude-4.6、文心ERNIE-X1.1、ERNIE-5.0、qwen3.6-max、qwen3.6-plus、百川、讯飞星火、商汤senseChat等商用模型, 以及step3.5-flash、kimi-k2.6、ernie4.5、MiniMax-M2.7、deepseek-v4、Qwen3.6、llama4、智谱GLM-5.1、MiMo-V2、LongCat、gemma4、mistral等开源大模型。不仅提供排行榜,也提供规模超200万的大模型缺陷库!方便广大社区研究分析、改进大模型。
GitHub · 6,522 stars · 261 forks · +2 stars in 24 hours · Trending score 1.31
-
21
intel/dokimi
Dokimi (Greek δοκιμή, "test, proof") measures whether a skill, MCP doc source, agent profile or repository guidance actually helps a coding agent: paired on/off arm comparisons, novel-vs-known content analysis, and repository scans with graded agent runs.
GitHub · Python · 39 stars · 13 forks · +1 stars in 24 hours · Trending score 1.30
-
22
truera/trulens
Evaluation and Tracking for LLM Experiments and AI Agents
GitHub · Python · 3,659 stars · 397 forks · 0 stars in 24 hours · Trending score 1.28
-
23
ljxpython/ai-agent-platform
企业级 AI Agent 平台底座 · 基于 LangGraph 生态二次开发,开箱即用(FastAPI + Vue 3 + 持续评测飞轮 + 原生知识中台 + 全链路可观测 + 智能排障助手 + 沙箱工作区)
GitHub · Python · 159 stars · 29 forks · +3 stars in 24 hours · Trending score 1.27
-
24
overmind-core/overmind
The platform for continuously improving AI agents.
GitHub · Python · 766 stars · 76 forks · +4 stars in 24 hours · Trending score 1.26