JChen-318/gaudi2-single-card-strata
Qwen3.8-Flash-Next (512-expert MoE, 256k context) on a single Intel Gaudi2: native SynapseAI + custom TPC kernels, DFlash2 speculative decoding, OpenAI/Anthropic-compatible API, Docker. 单卡 Gaudi2 原生推理,decode ~155 tok/s,prefill ~3.5k tok/s。
GitHub repository with 10 stars and 3 forks.
Language: C++
Topics: docker, gaudi2, habana, intel-gaudi, llm-inference, moe, qwen, speculative-decoding, tpc, synapseai