nvfp4 repositories

Trending repositories tagged nvfp4, ranked by trending score. Updated daily.

  1. 1
    sergqwer/strata-nvfp4

    Strata fork: Qwen3.8-Flash-Next 125B MoE in NVFP4 on one RTX 20-50 card (12 GB+) and 64 GB of RAM or more. Setup installs our GPTQ quants (huihui abliterated, OrcaRouter) from Hugging Face. CPU experts on AVX-512 or AVX2 (+AVX-VNNI), all experts in RAM from 92 GB, a low-RAM mode below, vision, W4A8 prompts. Windows / CUDA 13.

    GitHub · C++ · 251 stars · 31 forks · +23 stars in 24 hours · Trending score 2.42

  2. 2
    meshapplied/goethe

    Serve your models fast on your own infra in 3 steps.

    GitHub · Rust · 81 stars · 8 forks · +17 stars in 24 hours · Trending score 2.35

  3. 3
    NVlabs/Sana

    SANA & Sol: Efficient Generative Models and Full-Stack Training & Inference Engines for Image, Video, and World Modeling

    GitHub · Python · 9,820 stars · 805 forks · 0 stars in 24 hours · Trending score 1.84

  4. 4
    TandemLLM/TandemEngine

    TandemEngine: local inference that drafts and verifies its own tokens, behind an OpenAI-compatible API.

    GitHub · Python · 82 stars · 17 forks · +8 stars in 24 hours · Trending score 1.73

  5. 5
    xl03-bit/sm120-nvfp4-training

    Faster, more memory-efficient NVFP4 training

    GitHub · Python · 111 stars · 13 forks · +6 stars in 24 hours · Trending score 1.62

  6. 6
    hasso5703/alzeo

    Alzeo is an open-source installer that turns an NVIDIA DGX Spark, or any GB10 machine (ASUS, Dell, HP, Lenovo, MSI, GIGABYTE, Acer), into a private AI server in one command: measured model lanes, OpenAI and Anthropic APIs, and a local cockpit.

    GitHub · Python · 270 stars · 24 forks · +6 stars in 24 hours · Trending score 1.60

  7. 7
    NVIDIA/cudnn-frontend

    cuDNN Frontend is NVIDIA's modern, open-source entry point to the cuDNN library and a growing collection of high-performance open-source kernels.

    GitHub · Cuda · 1,007 stars · 340 forks · +3 stars in 24 hours · Trending score 1.47

  8. 8
    Leonccaa/ReVolta

    Stable, high-throughput inference server for multi-GPU NVIDIA V100 (SM70): Qwen3.8 Flash-Next with AWQ / NVFP4, multimodal input, CPU/disk KV offload, and MTP or DFlash2 speculative decoding.

    GitHub · Python · 64 stars · 2 forks · +2 stars in 24 hours · Trending score 1.14

  9. 9
    Bizuayeu/GLM-5.3-Flash-DGX-Sparks-BIZ

    GLM-5.3-Flash BIZ for DGX Spark-class GB10 systems. 3.x serves Jammed XSFP4 at TP=2 and XSFP4 at TP=3 and TP=4 with TensorFold's native Zig/CUDA engine (MTP drafts equal serial, tool calls, structured output, images, video). Apache-2.0 code, weights fetched separately. BIZ = business-use intent, not support or certification.

    GitHub · Python · 16 stars · 3 forks · +2 stars in 24 hours · Trending score 0.78

  10. 10
    gpillon/ignis

    An inference engine for one NVIDIA RTX card, built for a coding agent and its subagents, and a companion that listens, reads and draws: Qwen3.8 27B and 125B Flash-Next MoE, Qwen-Image-2.1, Qwen3-ASR on the CPU. OpenAI-compatible API and Playground in one binary.

    GitHub · Rust · 67 stars · 9 forks · +1 stars in 24 hours · Trending score 0.72

  11. 11
    intel/auto-round

    PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度

    GitHub · Python · 1,705 stars · 205 forks · 0 stars in 24 hours · Trending score 0.69

  12. 12
    avifenesh/memra

    Memra is a Rust + CUDA LLM inference engine with hand-written kernels, NVFP4 and mixed-precision execution, and correctness gates scoped to each model and GPU

    GitHub · Rust · 7 stars · 1 forks · +1 stars in 24 hours · Trending score 0.63

  13. 13
    headpiece747/ninfer-5090-windows

    Native Windows port of NInfer engine for RTX 5090. Features Qwen3.8-27B with QUASAR and NInfer models, MTP/DFlash2 with vision and 262,144 context

    GitHub · C++ · 91 stars · 18 forks · 0 stars in 24 hours · Trending score 0.59

  14. 14
    atakann/dgx-spark-recipes

    Measured recipes for the NVIDIA DGX Spark, on one or two Sparks: LLM serving, image, video and speech

    GitHub · Python · 19 stars · 0 forks · 0 stars in 24 hours · Trending score 0.57

  15. 15
    MerkyorLynn/lynn-engine

    Lynn 原生 LLM 推理引擎 · W4A8/NVFP4 量化 · 自写 CUDA/Triton kernel · MoE · 投机解码 | Lynn-native LLM inference engine for NVIDIA Blackwell

    GitHub · Python · 28 stars · 0 forks · 0 stars in 24 hours · Trending score 0.48

  16. 16
    Avarok-Cybersecurity/atlas

    Pure Rust Inference Engine

    GitHub · Rust · 717 stars · 106 forks · 0 stars in 24 hours · Trending score 0.31

  17. 17
    Enntity/sparkscore

    Typed, calibrated decisions on one NVIDIA DGX Spark: d3-ultra 27B NVFP4 on Atlas, JevBench public 90.9% at p50 146 ms, every question in one engine pass.

    GitHub · Python · 12 stars · 0 forks · 0 stars in 24 hours · Trending score 0.31

  18. 18
    evanwtf/local-llm

    Which model + engine + agent harness runs a coding agent best on local hardware? Real-repo agent benchmarks, pass rates and wall time, measured on an M5 Max MacBook Pro and a two-node DGX Spark cluster. OpenCode is the primary harness.

    GitHub · Python · 6 stars · 0 forks · 0 stars in 24 hours · Trending score 0.23

  19. 19
    kekzl/imp

    Run large AI chat models privately on your own NVIDIA RTX 50 or RTX PRO graphics card. On one RTX 5090, Qwen3.8-27B writes about 70 words per second, 1.4x faster than vLLM. Even a model almost four times larger than the card's memory runs. Works with apps made for ChatGPT or Claude. Starts with one docker command.

    GitHub · C++ · 79 stars · 6 forks · −4 stars in 24 hours · Trending score 0.10

  20. 20
    Cedriceuh/freetoken-rdna3

    Fast local LLM inference on AMD Radeon RX 7900 XTX / XT (RDNA3, ROCm): big Mixture-of-Experts models like Qwen3.8-Flash-Next on one or two consumer GPUs. Up to 4 parallel agents with 262k context each, or one 1M-token context, OpenAI/Anthropic API, NVFP4 and EXL3 checkpoints. A tuned build of FreeToken.

    GitHub · Python · 12 stars · 0 forks · 0 stars in 24 hours · Trending score 0.07

  21. 21
    ProprietaryLegal/Volta-Max-uPLIft-FP4-DeepSeek-V4.1-Flash-1017-NVFP4-8xV100-SXM2-PP2xTP4

    Volta Max uPLIft: DeepSeek-V4.1-Flash-1017 on 8x V100-SXM2 (2018) with as much of the routed work at NVIDIA's NVFP4 precision as fits and every routed expert on the cards. One Hugging Face package, the code, every measurement. PLI Labs.

    GitHub · Python · 21 stars · 0 forks · 0 stars in 24 hours · Trending score 0.06

  22. 22
    AEON-7/comfyui-aeon-spark

    Bleeding-edge ComfyUI for NVIDIA DGX Spark (GB10/Blackwell/sm_121a). CUDA 13 + SageAttention v3 (sm_121a) + NVFP4 + 14 custom-node packs + Flux 2 Dev / LTX 2.3 22B / ACE-Step v1.5 XL Turbo pre-bundled with abliterated text-encoder paths.

    GitHub · Shell · 90 stars · 18 forks · 0 stars in 24 hours · Trending score 0.05

  23. 23
    omgitsbase/llmash

    Ollama-compatible LLM server. Beats vllm by 40% on single stream decode, and by 20% on multi stream decode, and beats ollama by much 2-4x. Made possible with custom CUDA kernels, MTP, speculation and its own 3-bit builds. Uncensors models in one command. Windows and Linux, NVIDIA. Optimized to not eat up ram.

    GitHub · C++ · 37 stars · 7 forks · 0 stars in 24 hours · Trending score 0.05

  24. 24
    actypedef/ARCQuant

    [ACL 2026 Main] Code for the paper "ARCQuant: Boosting NVFP4 Quantization with Augmented Residual Channels for LLMs"

    GitHub · Cuda · 35 stars · 9 forks · 0 stars in 24 hours · Trending score 0.04