kv-cache-offloading repositories
Trending repositories tagged kv-cache-offloading, ranked by trending score. Updated daily.
-
1
Leonccaa/ReVolta
Stable, high-throughput inference server for multi-GPU NVIDIA V100 (SM70): Qwen3.8 Flash-Next with AWQ / NVFP4, multimodal input, CPU/disk KV offload, and MTP or DFlash2 speculative decoding.
GitHub · Python · 64 stars · 2 forks · +2 stars in 24 hours · Trending score 1.14
-
2
OrbitInfra/OrbitKV
OrbitKV is a High Performance KV Cache System for LLM Inference.
GitHub · Rust · 284 stars · 19 forks · +1 stars in 24 hours · Trending score 0.82