vllm repositories
Trending repositories tagged vllm, ranked by trending score. Updated daily.
-
1
autonomous-ai/Personal-AI-Computer
Open-source AI computer builds, from 2-GPU desktops to 8-GPU servers: CAD, BOM, BIOS and build guides. Get the case kit with PCIe risers, or ask us for a free fit check.
GitHub · 1,600 stars · 178 forks · +348 stars in 24 hours · Trending score 4.26
-
2
modelscope/FunASR
Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.
GitHub · Python · 21,084 stars · 2,140 forks · +73 stars in 24 hours · Trending score 3.55
-
3
adventists-ai/sophon-realtime
Open full-duplex voice models that hear your tone and speak with one. One GPU with vLLM-Omni, or fully offline on a Mac with MLX. Zero-shot voice cloning, OpenAI Realtime / GPT-Live compatible, free hosted API trial. Apache-2.0.
GitHub · Python · 499 stars · 43 forks · +43 stars in 24 hours · Trending score 3.26
-
4
vllm-project/vllm-ascend
Community maintained hardware plugin for vLLM on Huawei Ascend
GitHub · Python · 3,126 stars · 2,793 forks · +18 stars in 24 hours · Trending score 2.84
-
5
kvcache-ai/Mooncake
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
GitHub · C++ · 7,090 stars · 1,510 forks · +21 stars in 24 hours · Trending score 2.82
-
6
ai-dynamo/dynamo
A Datacenter Scale Distributed Inference Serving Framework
GitHub · Rust · 8,540 stars · 1,883 forks · +18 stars in 24 hours · Trending score 2.76
-
7
huawei-csl/MACaQ
SOTA model-agnostic quantization for GGUF, local inference and beyond. Quantize and run text, image, video and audio models with llama.cpp, vLLM, Diffusers and PyTorch.
GitHub · C++ · 381 stars · 22 forks · +15 stars in 24 hours · Trending score 2.72
-
8
verl-project/verl-omni
Multimodal RL training framework for diffusion & omni models
GitHub · Python · 1,554 stars · 336 forks · +17 stars in 24 hours · Trending score 2.66
-
9
loxilb-io/loxilb-inference-gateway
Inference-aware L4/L7 load balancer for LLM serving fleets, forked from loxilb
GitHub · Go · 72 stars · 14 forks · +15 stars in 24 hours · Trending score 2.54
-
10
vllm-project/semantic-router
An open, programmable decision layer for models and compute.
GitHub · Go · 6,514 stars · 1,141 forks · +10 stars in 24 hours · Trending score 2.44
-
11
AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-DFlash
Fully uncensored, capability-enhanced abliteration of Qwen3.6-27B. NVFP4 + z-lab DFlash speculative decoding (n=12) on the unified ghcr.io/aeon-7/aeon-vllm-ultimate:latest container, tuned for long-context draft acceptance on DGX Spark. 6 HF variants (BF16/NVFP4/MTP/MTP-XS), docker-compose, and QuickStart.
GitHub · Python · 500 stars · 57 forks · +27 stars in 24 hours · Trending score 2.44
-
12
SemiAnalysisAI/InferenceX
Open Source AI Accelerator Research Platform Standard / 开源推理研究平台
GitHub · Python · 2,072 stars · 384 forks · +13 stars in 24 hours · Trending score 2.43
-
13
LMCache/LMCache
LMCache: Supercharge Your LLM with the Fastest KV Cache Layer
GitHub · Python · 12,400 stars · 2,171 forks · +11 stars in 24 hours · Trending score 2.36
-
14
5p00kyy/club-5060ti
Practical local LLM recipes and benchmarks for RTX 5060 Ti setups
GitHub · Python · 209 stars · 13 forks · +30 stars in 24 hours · Trending score 2.25
-
15
budtmo/docker-android
Android in docker solution with noVNC supported, video recording, mcp server and AI-agent
GitHub · Python · 16,164 stars · 1,796 forks · +8 stars in 24 hours · Trending score 2.23
-
16
kserve/kserve
Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes
GitHub · Go · 6,252 stars · 1,801 forks · +7 stars in 24 hours · Trending score 2.13
-
17
ovg-project/kvcached
Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond
GitHub · Python · 2,051 stars · 243 forks · +15 stars in 24 hours · Trending score 2.12
-
18
christopherowen/spark-ds41f
Reproducible Docker/vLLM deployment, tuning, and benchmarks for DeepSeek V4.1 Flash on a switchless three-node or four-node DGX Spark fabric.
GitHub · Python · 235 stars · 58 forks · +6 stars in 24 hours · Trending score 2.07
-
19
vllm-project/vllm-metal
Community maintained hardware plugin for vLLM on Apple Silicon
GitHub · Python · 2,008 stars · 365 forks · +8 stars in 24 hours · Trending score 2.05
-
20
christopherowen/dgx-spark-networking
Switchless RoCE collectives, NCCL profiles and fabric tooling for DGX Spark inference recipes
GitHub · Python · 44 stars · 4 forks · +14 stars in 24 hours · Trending score 1.91
-
21
StartLuxLabs/StartLux-Decision
Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine for vLLM, llama.cpp and MLX.
GitHub · Python · 1,778 stars · 169 forks · +7 stars in 24 hours · Trending score 1.87
-
22
nokia-applied-research/AnyJev
Turn any LLM into a Jev-style decision model: typed decisions, real probabilities, no training. (continue updating, welcome any issue and PR request)
GitHub · Python · 1,632 stars · 181 forks · +9 stars in 24 hours · Trending score 1.85
-
23
smg-project/smg
Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.
GitHub · Rust · 735 stars · 230 forks · +4 stars in 24 hours · Trending score 1.76
-
24
hasso5703/alzeo
Alzeo is an open-source installer that turns an NVIDIA DGX Spark, or any GB10 machine (ASUS, Dell, HP, Lenovo, MSI, GIGABYTE, Acer), into a private AI server in one command: measured model lanes, OpenAI and Anthropic APIs, and a local cockpit.
GitHub · Python · 270 stars · 24 forks · +6 stars in 24 hours · Trending score 1.60