nvfp4 repositories
Trending repositories tagged nvfp4, ranked by trending score. Updated daily.
-
1
sergqwer/strata-nvfp4
Strata fork: Qwen3.8-Flash-Next 125B MoE in NVFP4 on one RTX 20-50 card (12 GB+) and 64 GB of RAM or more. Setup installs our GPTQ quants (huihui abliterated, OrcaRouter) from Hugging Face. CPU experts on AVX-512 or AVX2 (+AVX-VNNI), all experts in RAM from 92 GB, a low-RAM mode below, vision, W4A8 prompts. Windows / CUDA 13.
GitHub · C++ · 251 stars · 31 forks · +23 stars in 24 hours · Trending score 2.42
-
2
meshapplied/goethe
Serve your models fast on your own infra in 3 steps.
GitHub · Rust · 81 stars · 8 forks · +17 stars in 24 hours · Trending score 2.35
-
3
NVlabs/Sana
SANA & Sol: Efficient Generative Models and Full-Stack Training & Inference Engines for Image, Video, and World Modeling
GitHub · Python · 9,820 stars · 805 forks · 0 stars in 24 hours · Trending score 1.84
-
4
TandemLLM/TandemEngine
TandemEngine: local inference that drafts and verifies its own tokens, behind an OpenAI-compatible API.
GitHub · Python · 82 stars · 17 forks · +8 stars in 24 hours · Trending score 1.73
-
5
xl03-bit/sm120-nvfp4-training
Faster, more memory-efficient NVFP4 training
GitHub · Python · 111 stars · 13 forks · +6 stars in 24 hours · Trending score 1.62
-
6
hasso5703/alzeo
Alzeo is an open-source installer that turns an NVIDIA DGX Spark, or any GB10 machine (ASUS, Dell, HP, Lenovo, MSI, GIGABYTE, Acer), into a private AI server in one command: measured model lanes, OpenAI and Anthropic APIs, and a local cockpit.
GitHub · Python · 270 stars · 24 forks · +6 stars in 24 hours · Trending score 1.60
-
7
NVIDIA/cudnn-frontend
cuDNN Frontend is NVIDIA's modern, open-source entry point to the cuDNN library and a growing collection of high-performance open-source kernels.
GitHub · Cuda · 1,007 stars · 340 forks · +3 stars in 24 hours · Trending score 1.47
-
8
Leonccaa/ReVolta
Stable, high-throughput inference server for multi-GPU NVIDIA V100 (SM70): Qwen3.8 Flash-Next with AWQ / NVFP4, multimodal input, CPU/disk KV offload, and MTP or DFlash2 speculative decoding.
GitHub · Python · 64 stars · 2 forks · +2 stars in 24 hours · Trending score 1.14
-
9
Bizuayeu/GLM-5.3-Flash-DGX-Sparks-BIZ
GLM-5.3-Flash BIZ for DGX Spark-class GB10 systems. 3.x serves Jammed XSFP4 at TP=2 and XSFP4 at TP=3 and TP=4 with TensorFold's native Zig/CUDA engine (MTP drafts equal serial, tool calls, structured output, images, video). Apache-2.0 code, weights fetched separately. BIZ = business-use intent, not support or certification.
GitHub · Python · 16 stars · 3 forks · +2 stars in 24 hours · Trending score 0.78
-
10
gpillon/ignis
An inference engine for one NVIDIA RTX card, built for a coding agent and its subagents, and a companion that listens, reads and draws: Qwen3.8 27B and 125B Flash-Next MoE, Qwen-Image-2.1, Qwen3-ASR on the CPU. OpenAI-compatible API and Playground in one binary.
GitHub · Rust · 67 stars · 9 forks · +1 stars in 24 hours · Trending score 0.72
-
11
intel/auto-round
PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度
GitHub · Python · 1,705 stars · 205 forks · 0 stars in 24 hours · Trending score 0.69
-
12
avifenesh/memra
Memra is a Rust + CUDA LLM inference engine with hand-written kernels, NVFP4 and mixed-precision execution, and correctness gates scoped to each model and GPU
GitHub · Rust · 7 stars · 1 forks · +1 stars in 24 hours · Trending score 0.63
-
13
headpiece747/ninfer-5090-windows
Native Windows port of NInfer engine for RTX 5090. Features Qwen3.8-27B with QUASAR and NInfer models, MTP/DFlash2 with vision and 262,144 context
GitHub · C++ · 91 stars · 18 forks · 0 stars in 24 hours · Trending score 0.59
-
14
atakann/dgx-spark-recipes
Measured recipes for the NVIDIA DGX Spark, on one or two Sparks: LLM serving, image, video and speech
GitHub · Python · 19 stars · 0 forks · 0 stars in 24 hours · Trending score 0.57
-
15
MerkyorLynn/lynn-engine
Lynn 原生 LLM 推理引擎 · W4A8/NVFP4 量化 · 自写 CUDA/Triton kernel · MoE · 投机解码 | Lynn-native LLM inference engine for NVIDIA Blackwell
GitHub · Python · 28 stars · 0 forks · 0 stars in 24 hours · Trending score 0.48
-
16
Avarok-Cybersecurity/atlas
Pure Rust Inference Engine
GitHub · Rust · 717 stars · 106 forks · 0 stars in 24 hours · Trending score 0.31
-
17
Enntity/sparkscore
Typed, calibrated decisions on one NVIDIA DGX Spark: d3-ultra 27B NVFP4 on Atlas, JevBench public 90.9% at p50 146 ms, every question in one engine pass.
GitHub · Python · 12 stars · 0 forks · 0 stars in 24 hours · Trending score 0.31
-
18
evanwtf/local-llm
Which model + engine + agent harness runs a coding agent best on local hardware? Real-repo agent benchmarks, pass rates and wall time, measured on an M5 Max MacBook Pro and a two-node DGX Spark cluster. OpenCode is the primary harness.
GitHub · Python · 6 stars · 0 forks · 0 stars in 24 hours · Trending score 0.23
-
19
kekzl/imp
Run large AI chat models privately on your own NVIDIA RTX 50 or RTX PRO graphics card. On one RTX 5090, Qwen3.8-27B writes about 70 words per second, 1.4x faster than vLLM. Even a model almost four times larger than the card's memory runs. Works with apps made for ChatGPT or Claude. Starts with one docker command.
GitHub · C++ · 79 stars · 6 forks · −4 stars in 24 hours · Trending score 0.10
-
20
Cedriceuh/freetoken-rdna3
Fast local LLM inference on AMD Radeon RX 7900 XTX / XT (RDNA3, ROCm): big Mixture-of-Experts models like Qwen3.8-Flash-Next on one or two consumer GPUs. Up to 4 parallel agents with 262k context each, or one 1M-token context, OpenAI/Anthropic API, NVFP4 and EXL3 checkpoints. A tuned build of FreeToken.
GitHub · Python · 12 stars · 0 forks · 0 stars in 24 hours · Trending score 0.07
-
21
ProprietaryLegal/Volta-Max-uPLIft-FP4-DeepSeek-V4.1-Flash-1017-NVFP4-8xV100-SXM2-PP2xTP4
Volta Max uPLIft: DeepSeek-V4.1-Flash-1017 on 8x V100-SXM2 (2018) with as much of the routed work at NVIDIA's NVFP4 precision as fits and every routed expert on the cards. One Hugging Face package, the code, every measurement. PLI Labs.
GitHub · Python · 21 stars · 0 forks · 0 stars in 24 hours · Trending score 0.06
-
22
AEON-7/comfyui-aeon-spark
Bleeding-edge ComfyUI for NVIDIA DGX Spark (GB10/Blackwell/sm_121a). CUDA 13 + SageAttention v3 (sm_121a) + NVFP4 + 14 custom-node packs + Flux 2 Dev / LTX 2.3 22B / ACE-Step v1.5 XL Turbo pre-bundled with abliterated text-encoder paths.
GitHub · Shell · 90 stars · 18 forks · 0 stars in 24 hours · Trending score 0.05
-
23
omgitsbase/llmash
Ollama-compatible LLM server. Beats vllm by 40% on single stream decode, and by 20% on multi stream decode, and beats ollama by much 2-4x. Made possible with custom CUDA kernels, MTP, speculation and its own 3-bit builds. Uncensors models in one command. Windows and Linux, NVIDIA. Optimized to not eat up ram.
GitHub · C++ · 37 stars · 7 forks · 0 stars in 24 hours · Trending score 0.05
-
24
actypedef/ARCQuant
[ACL 2026 Main] Code for the paper "ARCQuant: Boosting NVFP4 Quantization with Augmented Residual Channels for LLMs"
GitHub · Cuda · 35 stars · 9 forks · 0 stars in 24 hours · Trending score 0.04