byte-od/mixture-of-scars
Breaking the barrier between LLMs and hardware. Mixture of Scars (MoS) static weight compression & SCache O(1) constant KV-cache replacement.
GitHub repository with 9 stars and 0 forks.
Language: Python
Topics: apple-silicon, artificial-intelligence, deep-learning, edge-ai, fast-weights, huggingface, inference, kv-cache, llm, machine-learning