gkjuwon-tech/ouroboros
Runs multi-B LLMs on a 2GB laptop MX150 (beating RTX 4060 in token decode) & hits 23.35 TFLOPS on A100 — zero-dependency single-file Rust GPU compiler
GitHub repository with 21 stars and 2 forks.
Language: Rust
Topics: compiler, cuda, deep-learning, e-graphs, gpu, inference, llm, machine-learning, ptx, quantization