alibaba/tair-kvcache
Alibaba Cloud's high-performance KVCache system for LLM inference, with components for global cache management, inference simulation(HiSim), and more.
GitHub repository with 185 stars and 35 forks.
Language: C++
Topics: hisim, kv-cache, kvcache, llm, simulator