GitHub · C++
1115714829/llama.cpp-v100
llama.cpp optimized for NVIDIA Tesla V100 (SM70). Qwen3.8-Flash-Next (MoE) with MTP on 1/2/3/6x V100-16GB at 262K context (6 GPUs: prefill 3402 tok/s at 200K; 2 GPUs on one socket: 2152 tok/s). Qwen3.8-27B with DFlash2 on 1-6 GPUs, up to 512K context. Multimodal, thinking, concurrent serving, built-in web UI.
- Stars
- 84
- Forks
- 17
- Stars gained in 24 hours
- 0
- Forks gained in 24 hours
- 0
Topics: cuda, gguf, llama-cpp, llm-inference, local-llm, long-context, multi-gpu, multimodal, qwen, qwen3
Latest data point: Nov 22, 2026 · 84 stars · 17 forks