sglang repositories

Trending repositories tagged sglang, ranked by trending score. Updated daily.

  1. 1
    datawhalechina/zero-to-sglang

    Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.

    GitHub · Python · 2,295 stars · 188 forks · +103 stars in 24 hours · Trending score 3.69

  2. 2
    sgl-project/sglang-omni

    High Performance Serving Runtime For Interactive Super Intelligence System

    GitHub · Python · 1,827 stars · 762 forks · +15 stars in 24 hours · Trending score 2.83

  3. 3
    kvcache-ai/Mooncake

    Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

    GitHub · C++ · 7,088 stars · 1,507 forks · +22 stars in 24 hours · Trending score 2.80

  4. 4
    ai-dynamo/dynamo

    A Datacenter Scale Distributed Inference Serving Framework

    GitHub · Rust · 8,537 stars · 1,882 forks · +13 stars in 24 hours · Trending score 2.63

  5. 5
    loxilb-io/loxilb-inference-gateway

    Inference-aware L4/L7 load balancer for LLM serving fleets, forked from loxilb

    GitHub · Go · 71 stars · 14 forks · +18 stars in 24 hours · Trending score 2.62

  6. 6
    SemiAnalysisAI/InferenceX

    Open Source AI Accelerator Research Platform Standard / 开源推理研究平台

    GitHub · Python · 2,069 stars · 382 forks · +15 stars in 24 hours · Trending score 2.47

  7. 7
    gpustack/gpustack

    A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

    GitHub · Python · 6,086 stars · 746 forks · +11 stars in 24 hours · Trending score 2.42

  8. 8
    meshapplied/goethe

    Serve your models fast on your own infra in 3 steps.

    GitHub · Rust · 81 stars · 8 forks · +17 stars in 24 hours · Trending score 2.35

  9. 9
    AMD-AGI/Hyperloom

    An agentic system that auto-optimizes LLM workloads on AMD GPUs.

    GitHub · Python · 282 stars · 89 forks · +7 stars in 24 hours · Trending score 1.88

  10. 10
    xorbitsai/inference

    Open-source inference platform for language, vision, embedding, reranking, image, audio, and video models. Deploy with multiple engines across heterogeneous hardware, from a laptop to a distributed cluster. Serve models through OpenAI-compatible APIs.

    GitHub · Python · 9,667 stars · 901 forks · +7 stars in 24 hours · Trending score 1.82

  11. 11
    bertholomus/spark-attic

    More memory for CUDA on an NVIDIA GB10 (DGX Spark, ASUS Ascent GX10): lend the unused display carveout to vLLM, SGLang and TensorFold as KV cache

    GitHub · Python · 34 stars · 1 forks · +7 stars in 24 hours · Trending score 1.81

  12. 12
    Zyphra/ZONOS3-Mini

    compact zero-shot voice-cloning TTS with state-of-the-art performance, leading voice control, real-time on-device.

    GitHub · Python · 254 stars · 28 forks · +32 stars in 24 hours · Trending score 1.68

  13. 13
    hasso5703/alzeo

    Alzeo is an open-source installer that turns an NVIDIA DGX Spark, or any GB10 machine (ASUS, Dell, HP, Lenovo, MSI, GIGABYTE, Acer), into a private AI server in one command: measured model lanes, OpenAI and Anthropic APIs, and a local cockpit.

    GitHub · Python · 270 stars · 24 forks · +6 stars in 24 hours · Trending score 1.60

  14. 14
    alvarobartt/hf-mem

    A CLI to estimate inference memory requirements for Hugging Face models, written in Python.

    GitHub · Python · 956 stars · 85 forks · +9 stars in 24 hours · Trending score 1.60

  15. 15
    smg-project/smg

    Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.

    GitHub · Rust · 734 stars · 230 forks · +2 stars in 24 hours · Trending score 1.57

  16. 16
    HarvardMadSys/chutes_workload

    one-year LLM serving trace from Chutes

    GitHub · Python · 42 stars · 1 forks · +9 stars in 24 hours · Trending score 1.52

  17. 17
    ForceInjection/AI-fundamentals

    AI 基础知识 - GPU 架构、CUDA 编程、大模型基础及AI Agent 相关知识。

    GitHub · HTML · 3,440 stars · 525 forks · 0 stars in 24 hours · Trending score 1.41

  18. 18
    MemTensor/MemT-1.0

    Official repository for MemT-1.0

    GitHub · Python · 545 stars · 44 forks · +3 stars in 24 hours · Trending score 1.41

  19. 19
    modelsphere/modelsphere

    ModelSphere is an open-source LLM inference infrastructure designed to make production-grade model serving simple, efficient, and continuously optimized.

    GitHub · Shell · 195 stars · 11 forks · +2 stars in 24 hours · Trending score 1.31

  20. 20
    davanstrien/saturate

    Batch inference for datasets: rows in, any OpenAI-compatible endpoint, resumable parquet out. Congestion-aware concurrency — no tuning.

    GitHub · Python · 56 stars · 2 forks · +2 stars in 24 hours · Trending score 1.22

  21. 21
    runzhliu/aik8s

    AI/LLM on Kubernetes 基础设施知识库:涵盖 GPU、大数据、调度、分布式训练、LLM 推理、RAG、Agent 与生产运维。| AI/LLM on Kubernetes Infrastructure Knowledge Base: Covering GPUs, big data, scheduling, distributed training, LLM inference, RAG, agents, and production operations.

    GitHub · Python · 44 stars · 4 forks · +4 stars in 24 hours · Trending score 1.22

  22. 22
    sgl-project/SpecForge

    Train speculative decoding models effortlessly and port them smoothly to SGLang serving.

    GitHub · Python · 1,290 stars · 398 forks · +1 stars in 24 hours · Trending score 1.19

  23. 23
    ModelCloud/GPTQModel

    LLM model quantization (compression) toolkit with HW acceleration support for Nvidia CUDA, AMD ROCm, Intel XPU, Tenstorrent, Huawei Ascent NPU, and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.

    GitHub · Python · 1,294 stars · 220 forks · +2 stars in 24 hours · Trending score 1.11

  24. 24
    ome-projects/ome

    Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton

    GitHub · Go · 528 stars · 107 forks · +3 stars in 24 hours · Trending score 1.10