The-School-of-AI/LLM
LightningLM 0.1V — Reference training pipeline for the LightningLM family. 2B dense seed → 5B MoE → 9B MoE → 120B sparse MoE through TurboQuant-PreTraining on a single eight-GPU node. Companion code for *Reversible Foundations*.
GitHub repository with 54 stars and 7 forks.
Language: Python
Topics: language-models, mixture-of-experts, pretraining, pytorch, quantized-training, reproducibility, sparse-moe, brahmic-tokenizer, lightninglm, reversible-models