llm-testing repositories
Trending repositories tagged llm-testing, ranked by trending score. Updated daily.
-
1
faiscadev/fakecloud
Free, open-source AWS emulator. LocalStack alternative: 107 services, 7,729 operations, true 100% Smithy conformance (260,001/260,001 variants pass). No account, no auth token, no paid tier.
GitHub · Rust · 798 stars · 63 forks · 0 stars in 24 hours · Trending score 0.63
-
2
rhesis-ai/rhesis
The collaboration layer for teams building AI agents. Test, simulate, and trace your agents, while domain experts annotate behavior and engineers improve the agent from what they find.
GitHub · Python · 405 stars · 43 forks · 0 stars in 24 hours · Trending score 0.39
-
3
TaimoorKhan10/replayd
Turn failed AI agent runs into replayable regression tests. Catch regressions before you ship.
GitHub · Python · 19 stars · 2 forks · 0 stars in 24 hours · Trending score 0.28
-
4
ProofAgent-ai/proofagent-harness
Open-source test harness for AI agents. Stress-test production agents with adversarial multi-turn scenarios in CI
GitHub · Python · 40 stars · 10 forks · 0 stars in 24 hours · Trending score 0.18
-
5
AltairaLabs/PromptKit
Test, red-team, and deploy LLM applications with confidence. Multi-provider support (OpenAI, Anthropic, Gemini), MCP integration, self-play testing, and production SDK.
GitHub · Go · 12 stars · 3 forks · 0 stars in 24 hours · Trending score 0.10
-
6
vonavikon/qa-agent-evals
Acceptance testing for LLM systems via role subagents (Claude Code plugin)
GitHub · Python · 7 stars · 0 forks · 0 stars in 24 hours · Trending score 0.08
-
7
tastewise/mcp-whichtool
Does the model pick the right tool? Tool-routing tests for your MCP server: run them on Claude and OpenAI models, see where each request lands, and fix the descriptions that lose.
GitHub · TypeScript · 13 stars · 0 forks · 0 stars in 24 hours · Trending score 0.06
-
8
Free-AI-Things/g4f-working
g4f-working is a daily-updated list of working no-auth AI providers and models from @xtekky/gpt4free. It helps developers, testers, and AI enthusiasts instantly find which models are currently online and accessible without any API keys, tokens, or cookies.
GitHub · Python · 153 stars · 18 forks · 0 stars in 24 hours · Trending score 0.05
-
9
nano-step/eval-harness
Typed evaluation for OpenCode skills and AI agents: capability, regression and product checks; provider-neutral evidence grading, provenance, and CI reports.
GitHub · Shell · 15 stars · 6 forks · 0 stars in 24 hours · Trending score 0.00
-
10
vieval-dev/vieval
🧪💥 Evaluation framework based on Vitest, the testing framework you familiar with, for agents, models, and more.
GitHub · TypeScript · 71 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00
-
11
zgrai-com/llm-api-test
A tool for testing and comparing the performance of different Large Language Model APIs. 一个用于测试和比较不同大语言模型API性能的工具。
GitHub · JavaScript · 48 stars · 4 forks · 0 stars in 24 hours · Trending score 0.00
-
12
evalshift/evalshift-cli
Open-source LLM migration and regression testing for AI agents. Compare models, detect tool-call regressions, and gate model changes in CI.
GitHub · Python · 8 stars · 1 forks · 0 stars in 24 hours · Trending score 0.00
-
13
EvilFreelancer/tgfake
Offline Telegram Bot API for testing bots, Mini Apps and LLM integrations.
GitHub · Go · 18 stars · 3 forks · 0 stars in 24 hours · Trending score 0.00