exllamav3 repositories
Trending repositories tagged exllamav3, ranked by trending score. Updated daily.
-
1
Yamz-Labs/kyojin
Kyojin: open-source engine that runs 100 GB-class open models (GLM-5.3, Qwen3.8, DeepSeek V4.1) on one desktop machine. AMD Strix Halo, NVIDIA RTX and Apple Mac. OpenAI API, built on ExLlamaV3.
GitHub · Python · 245 stars · 39 forks · +3 stars in 24 hours · Trending score 1.32
-
2
vcruz305/vllm-exl3
Serve EXL3 (ExLlamaV3 trellis) quantized models on vLLM fork runtimes — any architecture, mixed per-layer bitrates, composable with source-format non-routed weights
GitHub · Python · 27 stars · 14 forks · +1 stars in 24 hours · Trending score 0.61
-
3
lna-lab/flash-next-8gb
Run a 177B model (Qwen3.8-Flash-Next) on an 8 GB laptop GPU. Measured: 6.6 GiB VRAM, 47.8 GiB RAM, 34-35 tok/s. The n-gram table stays on disk; the experts run on CPU.
GitHub · Python · 36 stars · 10 forks · 0 stars in 24 hours · Trending score 0.49
-
4
r0b0tlab/qwen38-exl3-dflash2
Qwen3.8-27B EXL3 (4.00 bpw) + DFlash2 speculative decoding for ExLlamaV3, validated at 262k context on a 24 GB RTX 3090
GitHub · Python · 25 stars · 3 forks · 0 stars in 24 hours · Trending score 0.00
-
5
architectds/collabosm
A local app that rents a GPU in your own Colab account and serves Qwen3.8 on it: 27B on an A100-40G, or Flash-Next (125B-A6B MoE) on an A100-80G. The cost is shown before it starts; chat with images; Codex or any OpenAI client uses a fixed local /v1. Auto-stop, CU ledger, live prefill/decode. Windows, macOS, Linux.
GitHub · Python · 103 stars · 4 forks · 0 stars in 24 hours · Trending score 0.00
-
6
parth-thakre/exl3-rdna4-recipe
Qwen3.8-27B (EXL3) on an AMD RX 9070 XT: ~150 tok/s for code, 96k context, one-line install (container, any Linux distro)
GitHub · Shell · 10 stars · 1 forks · 0 stars in 24 hours · Trending score 0.00
-
7
vcruz305/MiniMax-M3.1-Flash-EXL3-Spark-recipe
MiniMax-M3.1-Flash EXL3 (SAGE 2.15 bpw) on ONE NVIDIA DGX Spark: pinned vLLM + vllm-exl3 recipe, q4 KV 32K, measured
GitHub · Shell · 6 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00
-
8
thefahien/elpis-on-7900xtx
Qwen3.8-27B EXL3 + DFlash2 speculative decoding on an AMD RX 7900 XTX (exllamav3-rocm): recipe, patches, benchmarks
GitHub · Python · 22 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00