llm-infrastructure repositories
Trending repositories tagged llm-infrastructure, ranked by trending score. Updated daily.
-
1
monaccode/astromesh
Multi-model AI agent runtime. Define agents in YAML, route each role to a model, orchestrate with 7 patterns (ReAct, Plan & Execute, Fan-Out, Pipeline, Supervisor, Swarm, Glyph), and deploy as a REST/WebSocket API with RAG, memory, MCP tools, guardrails and OpenTelemetry observability.
GitHub · Python · 39 stars · 9 forks · 0 stars in 24 hours · Trending score 0.05
-
2
diguike/book-llm-infra
LLM Infra 工程实战 — 面向应用层工程师的 LLM 基础设施
GitHub · 19 stars · 5 forks · 0 stars in 24 hours · Trending score 0.00
-
3
VectorInstitute/vector-inference
Efficient LLM inference on Slurm clusters.
GitHub · Python · 110 stars · 14 forks · 0 stars in 24 hours · Trending score 0.00
-
4
alphaparkinc/genpark-llm-token-bucket-sliding-window-rate-limiter-skill
Adaptive multi-tier TPM and RPM rate limiter with smooth leaky bucket queuing and predictive burst wait estimation
GitHub · Python · 7 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00
-
5
lupingQAQ/inkblade
An agentic penetration testing framework focused on LLM training and inference infrastructure (K8s + GPU)
GitHub · Python · 5 stars · 1 forks · 0 stars in 24 hours · Trending score 0.00