wici-ai/inferssd
Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.
GitHub repository with 262 stars and 41 forks.
Language: C++
Topics: apple-silicon, cuda, gguf, inference, llama-cpp, llm, metal, mixture-of-experts, ssd