kael-odin/2080ti-22G-kvmem-qwen3.8-27B-256K-36.6toks
RTX 2080 Ti 22G(sm_75)单卡本地长上下文最优解:KVMem 主线 262K 上下文 / 36.6 tok/s 实测(IQ3+MTP+视觉),KV 流式无损备档;完整源码复现链 + 构建/补丁/测试工具
GitHub repository with 5 stars and 1 forks.
Language: Python
Topics: cuda, inference, kv-cache, kvmem, llama-cpp, llm, local-llm, long-context, mtp, rtx-2080-ti