atdev-company/atdev-serve
Up to 14.2x faster Qwen3.8-27B inference on a single A100 - a drop-in plugin for vLLM 0.31.0. Models: huggingface.co/atdev-company
GitHub repository with 32 stars and 5 forks.
Language: Python
Topics: a100, cuda, llm-inference, quantization, qwen, speculative-decoding, vllm