GitHub · C++
omgitsbase/llmash
Ollama-compatible LLM server. Beats vllm by 40% on single stream decode, and by 20% on multi stream decode, and beats ollama by much 2-4x. Made possible with custom CUDA kernels, MTP, speculation and its own 3-bit builds. Uncensors models in one command. Windows and Linux, NVIDIA. Optimized to not eat up ram.
- Stars
- 37
- Forks
- 7
- Stars gained in 24 hours
- 0
- Forks gained in 24 hours
- 0
Topics: llm, ollama, ollama-api, local-first, qwen, abliteration, cuda, gemma, gguf, inference-engine
Latest data point: Nov 22, 2026 · 37 stars · 7 forks