neshat73/proxycache
⚡ Accelerate chat and IDE workflows with a proxy for llama.cpp, managing slots and cached context for efficient, low-latency interactions.
GitHub repository with 5 stars and 3 forks.
Language: Python
Topics: cache, file-cache, google-cloud, google-cloud-storage, nodejs, proxy, proxy-cache, proxycache