Betelgeuse-AI/Betelgeuse
A custom fork for frontier-scale mixture-of-experts inference on an RTX 3060 with 12 GB VRAM, 16 GB system RAM and local NVMe.** Betelgeuse keeps dense computation on the GPU and brings routed experts into a bounded VRAM pool only when needed.
GitHub repository with 85 stars and 1 forks.
Language: C++