zhangshuibai/molten
Run a coding model at 300+ tok/s on your Mac. Native Rust + Metal inference engine for LiquidAI LFM2.5-24B-A2B (MLX 4-bit) on Apple Silicon.
GitHub repository with 23 stars and 2 forks.
Language: Metal
Topics: apple-silicon, gpu-kernels, inference-engine, inference-optimization, lfm2, liquid-ai, llama-cpp, llm, llm-inference, llm-runtime