GitHub · Python
halt95/merlin-engine
vLLM 0.33 fork for consumer GPUs: 3 x 262K sessions on Ampere,Ada and more! Qwen 3.8 Flash-Next at 192 tok/s at full 262K context, up to 17 sessions on bigger cards. Eight checkpoints incl. Qwen3.8-27B, Nemotron 3.5 Super VL, Ornith 1.5 and Hy4 Flash (new). One command: merlin serve.
- Stars
- 53
- Forks
- 8
- Stars gained in 24 hours
- +11
- Forks gained in 24 hours
- +1
Topics: consumer-gpu, fp8, llm-inference, long-context, nemotron, qwen, rtx-3090, rtx-4090, speculative-decoding, vllm
Latest data point: Nov 22, 2026 · 53 stars · 8 forks