lhadzic377-netizen/TokiRecur
Toki Recur - recycled-prefill LLM toolkit: benchmarks prefill-reuse gains (TTFT, prefill tps, token savings), in-memory KV routing, model registry with per-stack llama-server+daemon. Cross-platform: ROCm/CUDA/Vulkan/Metal/CPU.
GitHub repository with 8 stars and 0 forks.
Language: Python