int4 repositories

Trending repositories tagged int4, ranked by trending score. Updated daily.

  1. 1
    tuan3w/qwen-warp

    Fast Qwen3.8-27B for one RTX 3090. 200k context, 100+ tok/s. Custom CUDA decode engine, OpenAI-compatible API.

    GitHub · Python · 50 stars · 6 forks · +1 stars in 24 hours · Trending score 0.83

  2. 2
    intel/auto-round

    PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度

    GitHub · Python · 1,706 stars · 205 forks · −2 stars in 24 hours · Trending score 0.64

  3. 3
    intel/neural-compressor

    SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime

    GitHub · Python · 2,730 stars · 333 forks · 0 stars in 24 hours · Trending score 0.03

  4. 4
    Dicklesworthstone/franken_ocr

    Pure-Rust, CPU-only OCR engine for Baidu Unlimited-OCR (a DeepSeek-OCR-derived 3B MoE VLM). Five-model zoo, custom int8 kernels, no ML framework, no Python, no GPU.

    GitHub · Rust · 347 stars · 41 forks · 0 stars in 24 hours · Trending score 0.02

  5. 5
    lokutor-ai/oido

    Oído: open-vocabulary speech recognition on a $5 ESP32-S3, faster than real time. No cloud, no NPU. English + Spanish.

    GitHub · C · 141 stars · 13 forks · 0 stars in 24 hours · Trending score 0.00

  6. 6
    DominikBucko/qwen38-flash-next-3090

    Run Qwen3.8-Flash-Next on ONE RTX 3090 (24 GB) + 64 GB RAM: 128K context, up to 2,100 tok/s prefill, 43–51 tok/s decode. vLLM runtime with hot MoE experts on the GPU and cold experts computed on the CPU, INT8 KV cache, Docker, OpenAI-compatible API.

    GitHub · Python · 16 stars · 3 forks · 0 stars in 24 hours · Trending score 0.00