exllamav3 repositories

Trending repositories tagged exllamav3, ranked by trending score. Updated daily.

  1. 1
    Yamz-Labs/kyojin

    Kyojin: open-source engine that runs 100 GB-class open models (GLM-5.3, Qwen3.8, DeepSeek V4.1) on one desktop machine. AMD Strix Halo, NVIDIA RTX and Apple Mac. OpenAI API, built on ExLlamaV3.

    GitHub · Python · 245 stars · 39 forks · +3 stars in 24 hours · Trending score 1.32

  2. 2
    vcruz305/vllm-exl3

    Serve EXL3 (ExLlamaV3 trellis) quantized models on vLLM fork runtimes — any architecture, mixed per-layer bitrates, composable with source-format non-routed weights

    GitHub · Python · 27 stars · 14 forks · +1 stars in 24 hours · Trending score 0.61

  3. 3
    lna-lab/flash-next-8gb

    Run a 177B model (Qwen3.8-Flash-Next) on an 8 GB laptop GPU. Measured: 6.6 GiB VRAM, 47.8 GiB RAM, 34-35 tok/s. The n-gram table stays on disk; the experts run on CPU.

    GitHub · Python · 36 stars · 10 forks · 0 stars in 24 hours · Trending score 0.49

  4. 4
    r0b0tlab/qwen38-exl3-dflash2

    Qwen3.8-27B EXL3 (4.00 bpw) + DFlash2 speculative decoding for ExLlamaV3, validated at 262k context on a 24 GB RTX 3090

    GitHub · Python · 25 stars · 3 forks · 0 stars in 24 hours · Trending score 0.00

  5. 5
    architectds/collabosm

    A local app that rents a GPU in your own Colab account and serves Qwen3.8 on it: 27B on an A100-40G, or Flash-Next (125B-A6B MoE) on an A100-80G. The cost is shown before it starts; chat with images; Codex or any OpenAI client uses a fixed local /v1. Auto-stop, CU ledger, live prefill/decode. Windows, macOS, Linux.

    GitHub · Python · 103 stars · 4 forks · 0 stars in 24 hours · Trending score 0.00

  6. 6
    parth-thakre/exl3-rdna4-recipe

    Qwen3.8-27B (EXL3) on an AMD RX 9070 XT: ~150 tok/s for code, 96k context, one-line install (container, any Linux distro)

    GitHub · Shell · 10 stars · 1 forks · 0 stars in 24 hours · Trending score 0.00

  7. 7
    vcruz305/MiniMax-M3.1-Flash-EXL3-Spark-recipe

    MiniMax-M3.1-Flash EXL3 (SAGE 2.15 bpw) on ONE NVIDIA DGX Spark: pinned vLLM + vllm-exl3 recipe, q4 KV 32K, measured

    GitHub · Shell · 6 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00

  8. 8
    thefahien/elpis-on-7900xtx

    Qwen3.8-27B EXL3 + DFlash2 speculative decoding on an AMD RX 7900 XTX (exllamav3-rocm): recipe, patches, benchmarks

    GitHub · Python · 22 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00