fp8 repositories

Trending repositories tagged fp8, ranked by trending score. Updated daily.

  1. 1
    NVIDIA/TransformerEngine

    A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit and 4-bit floating point (FP8 and FP4) precision on Hopper, Ada and Blackwell GPUs, to provide better performance with lower memory utilization in both training and inference.

    GitHub · Python · 3,609 stars · 901 forks · +2 stars in 24 hours · Trending score 1.27

  2. 2
    IxMxAMAR/SageAttention-RDNA4

    Drop-in SageAttention for AMD RX 9070 / 9070 XT (RDNA4, gfx1201): custom HIP attention kernels for fp16 and bf16, 1.06-2.2x faster than PR #368's gfx12 port over the full call. Prebuilt Windows wheels; ComfyUI uses it through --use-sage-attention.

    GitHub · Python · 15 stars · 1 forks · +3 stars in 24 hours · Trending score 0.93

  3. 3
    NVIDIA/cudnn-frontend

    cuDNN Frontend is NVIDIA's modern, open-source entry point to the cuDNN library and a growing collection of high-performance open-source kernels.

    GitHub · Cuda · 1,007 stars · 342 forks · 0 stars in 24 hours · Trending score 0.87

  4. 4
    MerkyorLynn/lynn-engine

    Lynn 原生 LLM 推理引擎 · W4A8/NVFP4 量化 · 自写 CUDA/Triton kernel · MoE · 投机解码 | Lynn-native LLM inference engine for NVIDIA Blackwell

    GitHub · Python · 28 stars · 0 forks · 0 stars in 24 hours · Trending score 0.48

  5. 5
    05yuki/deepseek-v4.1-flash-1017-ktransformers

    Run DeepSeek-V4.1-Flash-1017 with SGLang + KTransformers (CPU/GPU hybrid): one setup script, a launcher that reads the machine, and the FP8 step 1017 needs

    GitHub · Python · 7 stars · 4 forks · 0 stars in 24 hours · Trending score 0.06

  6. 6
    hanxiao/DeepSeek-V4.1-Flash-1017-FP8-abliterated-4x-RTX-PRO-6000

    Abliterated DeepSeek-V4.1-Flash-1017 (FP8 dense, FP4 experts) on 4x RTX PRO 6000 with vLLM-Moet r21, DSpark + FLy verification and top_p 1.0: 463 tok/s single-stream agent decode at 30k-120k context, 1082 tok/s over 8 streams, 1M context, Flex-start GCE VM

    GitHub · Python · 57 stars · 10 forks · 0 stars in 24 hours · Trending score 0.04

  7. 7
    amasu/dgx-spark-qwen38

    Docker compose configs for serving Qwen3.8-27B on a DGX Spark (GB10): vLLM+MTP current, SGLang+DSPARK + FP8 rollback stacks

    GitHub · Shell · 13 stars · 0 forks · 0 stars in 24 hours · Trending score 0.02

  8. 8
    vipshop/ffpa-attn

    Kernel Library for Large Headdim Attention (64~1024, BF16/FP8/FP4), 1.5x~15x↑ vs PyTorch SDPA.

    GitHub · Python · 341 stars · 29 forks · 0 stars in 24 hours · Trending score 0.00

  9. 9
    jpezzulli/sglang-rtxpro6000

    Optimized SGLang runtime for Qwen3.8-27B FP8 with DFlash2 and Qwen3.8 Flash-Next NVFP4 with FR-Spec on one NVIDIA RTX PRO 6000 Blackwell 96 GB GPU (SM120): 524K context, HiCache and NIXL.

    GitHub · Python · 179 stars · 26 forks · 0 stars in 24 hours · Trending score 0.00

  10. 10
    kk-pcl/sglang-qwen38-dual-3080-patch

    Minimal SGLang patch for Qwen3.8-27B + DFlash2 on dual RTX 3080: TP-sharded Draft fc and static per-head FP8 KV.

    GitHub · Python · 18 stars · 4 forks · 0 stars in 24 hours · Trending score 0.00

  11. 11
    yeungtuzi/Xiaotu-LLM

    XTU = X Transformers Unity — hybrid (CPU-expert + GPU) inference acceleration for vLLM: mainline plugin + CPU MoE engine (AVX512-VNNI/BF16 MXFP4/FP8), long-prefill GPU streaming, SM80 (A100) port.

    GitHub · Python · 16 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00

  12. 12
    ncannings/fastconformer-trt

    Parakeet / FastConformer ASR as an FP8 TensorRT engine: 4,903x real time on a DGX Spark, 25,073x on a B200, same accuracy as stock NeMo

    GitHub · Python · 5 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00

  13. 13
    halt95/merlin-engine

    vLLM 0.32 fork for consumer GPUs: 3 x 262K sessions on 4x RTX 3090, Qwen 3.8 Flash-Next at 197 tok/s, up to 17 on bigger cards. Seven checkpoints incl. Qwen3.8-27B, Nemotron 3.5 Super VL (1M) and Ornith 1.5. One command: merlin serve.

    GitHub · Python · 42 stars · 7 forks · 0 stars in 24 hours · Trending score 0.00

  14. 14
    astding/mmaemu

    Bit-exact emulation of NVIDIA tensor-core FP8 GEMMs (torch._scaled_mm, Transformer Engine) as an H100 PCIe or RTX 4060 computes them, on the CPU or another CUDA GPU

    GitHub · Python · 60 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00

  15. 15
    spinfinc/spinf-serve

    Bulk classification and decision inference on one GPU: the content is read once, every question about it is scored in the same pass. OpenAI-compatible, Apache 2.0

    GitHub · Python · 21 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00

  16. 16
    albond/SingleSpark-Qwen3.8-Flash-Next

    Experimental: Qwen3.8-Flash-Next (180B/6B MoE) on one NVIDIA DGX Spark at 44 tok/s — 4-bit NVFP4 + FP8 hybrid under vLLM, with the quality cost measured and published

    GitHub · Python · 12 stars · 0 forks · −1 stars in 24 hours · Trending score -0.01