gguf repositories
Trending repositories tagged gguf, ranked by trending score. Updated daily.
-
1
sgaofen/humanizer-local-model
Rewrites AI drafts so they read like a person wrote them — runs locally, keeps the facts
GitHub · Python · 1,001 stars · 163 forks · +625 stars in 24 hours · Trending score 4.98
-
2
qybaihe/bla
🛡️ BLA (bla-agents / Blind LLM Agents) — Open-source Blue Defense MCP server & CLI protecting JS/TS, Electron ASAR, WASM, Python, Native Binaries, SQLite, GGUF/Safetensors & Android DEX against AI reverse-engineering agents (REA).
GitHub · TypeScript · 6,260 stars · 778 forks · +154 stars in 24 hours · Trending score 4.42
-
3
mw00/project-maya
Run GLM-5.3 (744B), DeepSeek-V4.1-Flash (748B), MiniMax-M3.1-Flash (428B), GLM-5.3-Flash (321B) and Hy4-Flash (220B) on your own GPUs - from 8 GB of VRAM, NVIDIA and AMD, one dashboard, OpenAI- and Anthropic-compatible API.
GitHub · C++ · 3,094 stars · 430 forks · +69 stars in 24 hours · Trending score 3.63
-
4
simplexai-labs/PolyStrata
Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder and MiniCPM-V on a second engine written for PolyStrata. One-click install for Windows and Linux, OpenAI/Anthropic API on localhost.
GitHub · C++ · 1,280 stars · 166 forks · +60 stars in 24 hours · Trending score 3.63
-
5
synchopate/llama-modchip
Run 284B DeepSeek-V4-Flash (<59GiB RAM: 41.1 cold pp, 9.1 N=1 tg, up to 25.3 spec tok/s) & 30B MoEs (22.4 N=1 / 111.6 spec tok/s) on 64GB CPU (0 GPUs).
GitHub · Python · 1,387 stars · 134 forks · +38 stars in 24 hours · Trending score 3.55
-
6
CapriChooms/CapriCore
Capri — an AI desktop companion for Windows with local/API models, Hugging Face, optional voice, memory and permission-based desktop tools.
GitHub · JavaScript · 259 stars · 29 forks · +74 stars in 24 hours · Trending score 3.53
-
7
wici-ai/inferssd
Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.
GitHub · C++ · 1,262 stars · 113 forks · +32 stars in 24 hours · Trending score 3.48
-
8
AlexsJones/llmfit
Hundreds of models & providers. One command to find what runs on your hardware.
GitHub · Rust · 39,127 stars · 2,536 forks · +64 stars in 24 hours · Trending score 3.41
-
9
MakazhanAlpamys/Soup
Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.
GitHub · Python · 10,666 stars · 1,777 forks · +43 stars in 24 hours · Trending score 3.38
-
10
JakeATX/llamAmpere
llama.cpp fork for significantly improved performance on Ampere generation Nvidia cards (especially RTX 3090 / 3090 Ti): example: 110+ tok/s over a 100K-token generation at temperature 1 for qwen3.8, supporting full 262K ctx. Supports low memory (12gb) configs for 200k+ ctx. 120+ TPS, 3300+ prefill for QFN 3.8.
GitHub · C++ · 403 stars · 50 forks · +87 stars in 24 hours · Trending score 3.35
-
11
Veloresearch/kinetic
The open engine for running AI on your own computer. Fast, exact, open source.
GitHub · Rust · 202 stars · 20 forks · +62 stars in 24 hours · Trending score 3.17
-
12
Helldez/BigMoeOnEdge
Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, edge. Lossless, on stock llama.cpp. An adaptive streaming optimizer. Any hardware, any OS, any model, any quantization, behind a local OpenAI-compatible server.
GitHub · C++ · 1,033 stars · 107 forks · +23 stars in 24 hours · Trending score 3.07
-
13
huawei-csl/MACaQ
SOTA model-agnostic quantization for GGUF, local inference and beyond. Quantize and run text, image, video and audio models with llama.cpp, vLLM, Diffusers and PyTorch.
GitHub · C++ · 380 stars · 22 forks · +22 stars in 24 hours · Trending score 2.98
-
14
Bai-Cute/big-blue-fish-desktop-pet
蓝色大肥鱼 Windows AI 桌宠,内置 4B 本地大模型,以鲸鱼娘人设结合天气、科技新闻和当前应用,通过文字气泡温柔陪伴。支持一键安装与离电暂停推理。Big Blue Fish desktop pet with local LLM.
GitHub · C# · 374 stars · 28 forks · +51 stars in 24 hours · Trending score 2.97
-
15
RustingStudio/RustingStrata
Pure-Rust inference engine for Qwen GGUF models (Qwen3 to Qwen3.8, dense and MoE up to 770B+). CUDA and Vulkan (NVIDIA, AMD), hybrid CPU+GPU experts, MTP speculative decoding, OpenAI and Anthropic compatible server. Up to ~1.7x llama.cpp decode on a 12 GB RTX 3060.
GitHub · Rust · 291 stars · 36 forks · +31 stars in 24 hours · Trending score 2.90
-
16
Video-Rebirth/ComfyUI-HyperFlow-Flash
ComfyUI custom nodes and workflow templates for HyperFlow H3 Flash GGUF checkpoints
GitHub · Python · 109 stars · 21 forks · +21 stars in 24 hours · Trending score 2.61
-
17
AtomicBot-ai/Atomic-Chat
Local AI app and inference engine for agents. Run open-weight LLMs locally — private, 100% offline on your computer. Join our Discord: https://discord.com/invite/8wGSsvmg4V
GitHub · TypeScript · 2,034 stars · 234 forks · +21 stars in 24 hours · Trending score 2.50
-
18
prabanta-dev/janas
37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.
GitHub · C · 735 stars · 73 forks · +11 stars in 24 hours · Trending score 2.41
-
19
off-grid-ai/OGAM
The Swiss Army Knife of Offline AI. Chat, see, speak, and generate images on your phone or Mac — GGUF LLMs, vision, Whisper speech-to-text, Stable Diffusion, tool calling, and local-network servers. Runs on your CPU, GPU, or NPU. No account, no API key, zero data leaves your device.
GitHub · TypeScript · 3,398 stars · 339 forks · +7 stars in 24 hours · Trending score 2.24
-
20
Indras-Mirror/Xolotl
Xolotl: run DeepSeek-V4.1-Flash, all 552B, as a local coding agent on one gaming PC (RTX 4090 + 110 GB RAM): ~33 tok/s on agent tasks, ~41 in fast mode, 512K context. Plus GLM-5.3-Flash, Qwen3.8 and LFM3-Omni. MoE experts tiered across VRAM, pinned RAM and NVMe, with Xolotl Direct NVMe->VRAM. OpenAI + Anthropic API.
GitHub · C++ · 545 stars · 67 forks · +12 stars in 24 hours · Trending score 2.18
-
21
scarletkc/colab-playground
A desktop app for the Google Colab CLI: accounts, GPU sessions with cost guards, port forwarding and plugins.
GitHub · Rust · 297 stars · 23 forks · +87 stars in 24 hours · Trending score 2.15
-
22
affifuddin-hazam/ghost-infer
Fast local LLM inference for the GPU you already have: one Rust binary, GGUF models, an OpenAI-compatible server and a chat page with a ghost.
GitHub · HTML · 246 stars · 17 forks · +3 stars in 24 hours · Trending score 2.15
-
23
StartLuxLabs/StartLux-Decision
Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine for vLLM, llama.cpp and MLX.
GitHub · Python · 1,776 stars · 169 forks · +9 stars in 24 hours · Trending score 2.04
-
24
nobodywho-ooo/nobodywho
NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.
GitHub · Rust · 1,691 stars · 102 forks · +16 stars in 24 hours · Trending score 2.03