llm-infrastructure repositories

Trending repositories tagged llm-infrastructure, ranked by trending score. Updated daily.

  1. 1
    monaccode/astromesh

    Multi-model AI agent runtime. Define agents in YAML, route each role to a model, orchestrate with 7 patterns (ReAct, Plan & Execute, Fan-Out, Pipeline, Supervisor, Swarm, Glyph), and deploy as a REST/WebSocket API with RAG, memory, MCP tools, guardrails and OpenTelemetry observability.

    GitHub · Python · 39 stars · 9 forks · 0 stars in 24 hours · Trending score 0.05

  2. 2
    diguike/book-llm-infra

    LLM Infra 工程实战 — 面向应用层工程师的 LLM 基础设施

    GitHub · 19 stars · 5 forks · 0 stars in 24 hours · Trending score 0.00

  3. 3
    VectorInstitute/vector-inference

    Efficient LLM inference on Slurm clusters.

    GitHub · Python · 110 stars · 14 forks · 0 stars in 24 hours · Trending score 0.00

  4. 4
    alphaparkinc/genpark-llm-token-bucket-sliding-window-rate-limiter-skill

    Adaptive multi-tier TPM and RPM rate limiter with smooth leaky bucket queuing and predictive burst wait estimation

    GitHub · Python · 7 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00

  5. 5
    lupingQAQ/inkblade

    An agentic penetration testing framework focused on LLM training and inference infrastructure (K8s + GPU)

    GitHub · Python · 5 stars · 1 forks · 0 stars in 24 hours · Trending score 0.00