vllm repositories

Trending repositories tagged vllm, ranked by trending score. Updated daily.

  1. 1
    autonomous-ai/Personal-AI-Computer

    Open-source AI computer builds, from 2-GPU desktops to 8-GPU servers: CAD, BOM, BIOS and build guides. Get the case kit with PCIe risers, or ask us for a free fit check.

    GitHub · 1,600 stars · 178 forks · +348 stars in 24 hours · Trending score 4.26

  2. 2
    modelscope/FunASR

    Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.

    GitHub · Python · 21,084 stars · 2,140 forks · +73 stars in 24 hours · Trending score 3.55

  3. 3
    adventists-ai/sophon-realtime

    Open full-duplex voice models that hear your tone and speak with one. One GPU with vLLM-Omni, or fully offline on a Mac with MLX. Zero-shot voice cloning, OpenAI Realtime / GPT-Live compatible, free hosted API trial. Apache-2.0.

    GitHub · Python · 499 stars · 43 forks · +43 stars in 24 hours · Trending score 3.26

  4. 4
    vllm-project/vllm-ascend

    Community maintained hardware plugin for vLLM on Huawei Ascend

    GitHub · Python · 3,126 stars · 2,793 forks · +18 stars in 24 hours · Trending score 2.84

  5. 5
    kvcache-ai/Mooncake

    Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

    GitHub · C++ · 7,090 stars · 1,510 forks · +21 stars in 24 hours · Trending score 2.82

  6. 6
    ai-dynamo/dynamo

    A Datacenter Scale Distributed Inference Serving Framework

    GitHub · Rust · 8,540 stars · 1,883 forks · +18 stars in 24 hours · Trending score 2.76

  7. 7
    huawei-csl/MACaQ

    SOTA model-agnostic quantization for GGUF, local inference and beyond. Quantize and run text, image, video and audio models with llama.cpp, vLLM, Diffusers and PyTorch.

    GitHub · C++ · 381 stars · 22 forks · +15 stars in 24 hours · Trending score 2.72

  8. 8
    verl-project/verl-omni

    Multimodal RL training framework for diffusion & omni models

    GitHub · Python · 1,554 stars · 336 forks · +17 stars in 24 hours · Trending score 2.66

  9. 9
    loxilb-io/loxilb-inference-gateway

    Inference-aware L4/L7 load balancer for LLM serving fleets, forked from loxilb

    GitHub · Go · 72 stars · 14 forks · +15 stars in 24 hours · Trending score 2.54

  10. 10
    vllm-project/semantic-router

    An open, programmable decision layer for models and compute.

    GitHub · Go · 6,514 stars · 1,141 forks · +10 stars in 24 hours · Trending score 2.44

  11. 11
    AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-DFlash

    Fully uncensored, capability-enhanced abliteration of Qwen3.6-27B. NVFP4 + z-lab DFlash speculative decoding (n=12) on the unified ghcr.io/aeon-7/aeon-vllm-ultimate:latest container, tuned for long-context draft acceptance on DGX Spark. 6 HF variants (BF16/NVFP4/MTP/MTP-XS), docker-compose, and QuickStart.

    GitHub · Python · 500 stars · 57 forks · +27 stars in 24 hours · Trending score 2.44

  12. 12
    SemiAnalysisAI/InferenceX

    Open Source AI Accelerator Research Platform Standard / 开源推理研究平台

    GitHub · Python · 2,072 stars · 384 forks · +13 stars in 24 hours · Trending score 2.43

  13. 13
    LMCache/LMCache

    LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

    GitHub · Python · 12,400 stars · 2,171 forks · +11 stars in 24 hours · Trending score 2.36

  14. 14
    5p00kyy/club-5060ti

    Practical local LLM recipes and benchmarks for RTX 5060 Ti setups

    GitHub · Python · 209 stars · 13 forks · +30 stars in 24 hours · Trending score 2.25

  15. 15
    budtmo/docker-android

    Android in docker solution with noVNC supported, video recording, mcp server and AI-agent

    GitHub · Python · 16,164 stars · 1,796 forks · +8 stars in 24 hours · Trending score 2.23

  16. 16
    kserve/kserve

    Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes

    GitHub · Go · 6,252 stars · 1,801 forks · +7 stars in 24 hours · Trending score 2.13

  17. 17
    ovg-project/kvcached

    Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

    GitHub · Python · 2,051 stars · 243 forks · +15 stars in 24 hours · Trending score 2.12

  18. 18
    christopherowen/spark-ds41f

    Reproducible Docker/vLLM deployment, tuning, and benchmarks for DeepSeek V4.1 Flash on a switchless three-node or four-node DGX Spark fabric.

    GitHub · Python · 235 stars · 58 forks · +6 stars in 24 hours · Trending score 2.07

  19. 19
    vllm-project/vllm-metal

    Community maintained hardware plugin for vLLM on Apple Silicon

    GitHub · Python · 2,008 stars · 365 forks · +8 stars in 24 hours · Trending score 2.05

  20. 20
    christopherowen/dgx-spark-networking

    Switchless RoCE collectives, NCCL profiles and fabric tooling for DGX Spark inference recipes

    GitHub · Python · 44 stars · 4 forks · +14 stars in 24 hours · Trending score 1.91

  21. 21
    StartLuxLabs/StartLux-Decision

    Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine for vLLM, llama.cpp and MLX.

    GitHub · Python · 1,778 stars · 169 forks · +7 stars in 24 hours · Trending score 1.87

  22. 22
    nokia-applied-research/AnyJev

    Turn any LLM into a Jev-style decision model: typed decisions, real probabilities, no training. (continue updating, welcome any issue and PR request)

    GitHub · Python · 1,632 stars · 181 forks · +9 stars in 24 hours · Trending score 1.85

  23. 23
    smg-project/smg

    Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.

    GitHub · Rust · 735 stars · 230 forks · +4 stars in 24 hours · Trending score 1.76

  24. 24
    hasso5703/alzeo

    Alzeo is an open-source installer that turns an NVIDIA DGX Spark, or any GB10 machine (ASUS, Dell, HP, Lenovo, MSI, GIGABYTE, Acer), into a private AI server in one command: measured model lanes, OpenAI and Anthropic APIs, and a local cockpit.

    GitHub · Python · 270 stars · 24 forks · +6 stars in 24 hours · Trending score 1.60