sglang repositories
Trending repositories tagged sglang, ranked by trending score. Updated daily.
-
1
datawhalechina/zero-to-sglang
Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.
GitHub · Python · 2,295 stars · 188 forks · +103 stars in 24 hours · Trending score 3.69
-
2
sgl-project/sglang-omni
High Performance Serving Runtime For Interactive Super Intelligence System
GitHub · Python · 1,827 stars · 762 forks · +15 stars in 24 hours · Trending score 2.83
-
3
kvcache-ai/Mooncake
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
GitHub · C++ · 7,088 stars · 1,507 forks · +22 stars in 24 hours · Trending score 2.80
-
4
ai-dynamo/dynamo
A Datacenter Scale Distributed Inference Serving Framework
GitHub · Rust · 8,537 stars · 1,882 forks · +13 stars in 24 hours · Trending score 2.63
-
5
loxilb-io/loxilb-inference-gateway
Inference-aware L4/L7 load balancer for LLM serving fleets, forked from loxilb
GitHub · Go · 71 stars · 14 forks · +18 stars in 24 hours · Trending score 2.62
-
6
SemiAnalysisAI/InferenceX
Open Source AI Accelerator Research Platform Standard / 开源推理研究平台
GitHub · Python · 2,069 stars · 382 forks · +15 stars in 24 hours · Trending score 2.47
-
7
gpustack/gpustack
A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.
GitHub · Python · 6,086 stars · 746 forks · +11 stars in 24 hours · Trending score 2.42
-
8
meshapplied/goethe
Serve your models fast on your own infra in 3 steps.
GitHub · Rust · 81 stars · 8 forks · +17 stars in 24 hours · Trending score 2.35
-
9
AMD-AGI/Hyperloom
An agentic system that auto-optimizes LLM workloads on AMD GPUs.
GitHub · Python · 282 stars · 89 forks · +7 stars in 24 hours · Trending score 1.88
-
10
xorbitsai/inference
Open-source inference platform for language, vision, embedding, reranking, image, audio, and video models. Deploy with multiple engines across heterogeneous hardware, from a laptop to a distributed cluster. Serve models through OpenAI-compatible APIs.
GitHub · Python · 9,667 stars · 901 forks · +7 stars in 24 hours · Trending score 1.82
-
11
bertholomus/spark-attic
More memory for CUDA on an NVIDIA GB10 (DGX Spark, ASUS Ascent GX10): lend the unused display carveout to vLLM, SGLang and TensorFold as KV cache
GitHub · Python · 34 stars · 1 forks · +7 stars in 24 hours · Trending score 1.81
-
12
Zyphra/ZONOS3-Mini
compact zero-shot voice-cloning TTS with state-of-the-art performance, leading voice control, real-time on-device.
GitHub · Python · 254 stars · 28 forks · +32 stars in 24 hours · Trending score 1.68
-
13
hasso5703/alzeo
Alzeo is an open-source installer that turns an NVIDIA DGX Spark, or any GB10 machine (ASUS, Dell, HP, Lenovo, MSI, GIGABYTE, Acer), into a private AI server in one command: measured model lanes, OpenAI and Anthropic APIs, and a local cockpit.
GitHub · Python · 270 stars · 24 forks · +6 stars in 24 hours · Trending score 1.60
-
14
alvarobartt/hf-mem
A CLI to estimate inference memory requirements for Hugging Face models, written in Python.
GitHub · Python · 956 stars · 85 forks · +9 stars in 24 hours · Trending score 1.60
-
15
smg-project/smg
Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.
GitHub · Rust · 734 stars · 230 forks · +2 stars in 24 hours · Trending score 1.57
-
16
HarvardMadSys/chutes_workload
one-year LLM serving trace from Chutes
GitHub · Python · 42 stars · 1 forks · +9 stars in 24 hours · Trending score 1.52
-
17
ForceInjection/AI-fundamentals
AI 基础知识 - GPU 架构、CUDA 编程、大模型基础及AI Agent 相关知识。
GitHub · HTML · 3,440 stars · 525 forks · 0 stars in 24 hours · Trending score 1.41
-
18
MemTensor/MemT-1.0
Official repository for MemT-1.0
GitHub · Python · 545 stars · 44 forks · +3 stars in 24 hours · Trending score 1.41
-
19
modelsphere/modelsphere
ModelSphere is an open-source LLM inference infrastructure designed to make production-grade model serving simple, efficient, and continuously optimized.
GitHub · Shell · 195 stars · 11 forks · +2 stars in 24 hours · Trending score 1.31
-
20
davanstrien/saturate
Batch inference for datasets: rows in, any OpenAI-compatible endpoint, resumable parquet out. Congestion-aware concurrency — no tuning.
GitHub · Python · 56 stars · 2 forks · +2 stars in 24 hours · Trending score 1.22
-
21
runzhliu/aik8s
AI/LLM on Kubernetes 基础设施知识库:涵盖 GPU、大数据、调度、分布式训练、LLM 推理、RAG、Agent 与生产运维。| AI/LLM on Kubernetes Infrastructure Knowledge Base: Covering GPUs, big data, scheduling, distributed training, LLM inference, RAG, agents, and production operations.
GitHub · Python · 44 stars · 4 forks · +4 stars in 24 hours · Trending score 1.22
-
22
sgl-project/SpecForge
Train speculative decoding models effortlessly and port them smoothly to SGLang serving.
GitHub · Python · 1,290 stars · 398 forks · +1 stars in 24 hours · Trending score 1.19
-
23
ModelCloud/GPTQModel
LLM model quantization (compression) toolkit with HW acceleration support for Nvidia CUDA, AMD ROCm, Intel XPU, Tenstorrent, Huawei Ascent NPU, and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.
GitHub · Python · 1,294 stars · 220 forks · +2 stars in 24 hours · Trending score 1.11
-
24
ome-projects/ome
Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton
GitHub · Go · 528 stars · 107 forks · +3 stars in 24 hours · Trending score 1.10