fp8 repositories
Trending repositories tagged fp8, ranked by trending score. Updated daily.
-
1
NVIDIA/TransformerEngine
A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit and 4-bit floating point (FP8 and FP4) precision on Hopper, Ada and Blackwell GPUs, to provide better performance with lower memory utilization in both training and inference.
GitHub · Python · 3,609 stars · 901 forks · +2 stars in 24 hours · Trending score 1.27
-
2
IxMxAMAR/SageAttention-RDNA4
Drop-in SageAttention for AMD RX 9070 / 9070 XT (RDNA4, gfx1201): custom HIP attention kernels for fp16 and bf16, 1.06-2.2x faster than PR #368's gfx12 port over the full call. Prebuilt Windows wheels; ComfyUI uses it through --use-sage-attention.
GitHub · Python · 15 stars · 1 forks · +3 stars in 24 hours · Trending score 0.93
-
3
NVIDIA/cudnn-frontend
cuDNN Frontend is NVIDIA's modern, open-source entry point to the cuDNN library and a growing collection of high-performance open-source kernels.
GitHub · Cuda · 1,007 stars · 342 forks · 0 stars in 24 hours · Trending score 0.87
-
4
MerkyorLynn/lynn-engine
Lynn 原生 LLM 推理引擎 · W4A8/NVFP4 量化 · 自写 CUDA/Triton kernel · MoE · 投机解码 | Lynn-native LLM inference engine for NVIDIA Blackwell
GitHub · Python · 28 stars · 0 forks · 0 stars in 24 hours · Trending score 0.48
-
5
05yuki/deepseek-v4.1-flash-1017-ktransformers
Run DeepSeek-V4.1-Flash-1017 with SGLang + KTransformers (CPU/GPU hybrid): one setup script, a launcher that reads the machine, and the FP8 step 1017 needs
GitHub · Python · 7 stars · 4 forks · 0 stars in 24 hours · Trending score 0.06
-
6
hanxiao/DeepSeek-V4.1-Flash-1017-FP8-abliterated-4x-RTX-PRO-6000
Abliterated DeepSeek-V4.1-Flash-1017 (FP8 dense, FP4 experts) on 4x RTX PRO 6000 with vLLM-Moet r21, DSpark + FLy verification and top_p 1.0: 463 tok/s single-stream agent decode at 30k-120k context, 1082 tok/s over 8 streams, 1M context, Flex-start GCE VM
GitHub · Python · 57 stars · 10 forks · 0 stars in 24 hours · Trending score 0.04
-
7
amasu/dgx-spark-qwen38
Docker compose configs for serving Qwen3.8-27B on a DGX Spark (GB10): vLLM+MTP current, SGLang+DSPARK + FP8 rollback stacks
GitHub · Shell · 13 stars · 0 forks · 0 stars in 24 hours · Trending score 0.02
-
8
vipshop/ffpa-attn
Kernel Library for Large Headdim Attention (64~1024, BF16/FP8/FP4), 1.5x~15x↑ vs PyTorch SDPA.
GitHub · Python · 341 stars · 29 forks · 0 stars in 24 hours · Trending score 0.00
-
9
jpezzulli/sglang-rtxpro6000
Optimized SGLang runtime for Qwen3.8-27B FP8 with DFlash2 and Qwen3.8 Flash-Next NVFP4 with FR-Spec on one NVIDIA RTX PRO 6000 Blackwell 96 GB GPU (SM120): 524K context, HiCache and NIXL.
GitHub · Python · 179 stars · 26 forks · 0 stars in 24 hours · Trending score 0.00
-
10
kk-pcl/sglang-qwen38-dual-3080-patch
Minimal SGLang patch for Qwen3.8-27B + DFlash2 on dual RTX 3080: TP-sharded Draft fc and static per-head FP8 KV.
GitHub · Python · 18 stars · 4 forks · 0 stars in 24 hours · Trending score 0.00
-
11
yeungtuzi/Xiaotu-LLM
XTU = X Transformers Unity — hybrid (CPU-expert + GPU) inference acceleration for vLLM: mainline plugin + CPU MoE engine (AVX512-VNNI/BF16 MXFP4/FP8), long-prefill GPU streaming, SM80 (A100) port.
GitHub · Python · 16 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00
-
12
ncannings/fastconformer-trt
Parakeet / FastConformer ASR as an FP8 TensorRT engine: 4,903x real time on a DGX Spark, 25,073x on a B200, same accuracy as stock NeMo
GitHub · Python · 5 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00
-
13
halt95/merlin-engine
vLLM 0.32 fork for consumer GPUs: 3 x 262K sessions on 4x RTX 3090, Qwen 3.8 Flash-Next at 197 tok/s, up to 17 on bigger cards. Seven checkpoints incl. Qwen3.8-27B, Nemotron 3.5 Super VL (1M) and Ornith 1.5. One command: merlin serve.
GitHub · Python · 42 stars · 7 forks · 0 stars in 24 hours · Trending score 0.00
-
14
astding/mmaemu
Bit-exact emulation of NVIDIA tensor-core FP8 GEMMs (torch._scaled_mm, Transformer Engine) as an H100 PCIe or RTX 4060 computes them, on the CPU or another CUDA GPU
GitHub · Python · 60 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00
-
15
spinfinc/spinf-serve
Bulk classification and decision inference on one GPU: the content is read once, every question about it is scored in the same pass. OpenAI-compatible, Apache 2.0
GitHub · Python · 21 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00
-
16
albond/SingleSpark-Qwen3.8-Flash-Next
Experimental: Qwen3.8-Flash-Next (180B/6B MoE) on one NVIDIA DGX Spark at 44 tok/s — 4-bit NVFP4 + FP8 hybrid under vLLM, with the quality cost measured and published
GitHub · Python · 12 stars · 0 forks · −1 stars in 24 hours · Trending score -0.01