hicache repositories
Trending repositories tagged hicache, ranked by trending score. Updated daily.
-
1
awesome-pro/hiqcache
INT8-quantized hierarchical KV cache for SGLang HiCache. Compresses KV evicted from GPU L1 into the CPU L2 tier, holding 1.78x more tokens in the same host memory. Measured on Qwen3-8B against a BF16 baseline.
GitHub · Python · 11 stars · 0 forks · 0 stars in 24 hours · Trending score 0.02
-
2
jpezzulli/sglang-rtxpro6000
Optimized SGLang runtime for Qwen3.8-27B FP8 with DFlash2 and Qwen3.8 Flash-Next NVFP4 with FR-Spec on one NVIDIA RTX PRO 6000 Blackwell 96 GB GPU (SM120): 524K context, HiCache and NIXL.
GitHub · Python · 179 stars · 26 forks · 0 stars in 24 hours · Trending score 0.00