turboquant repositories
Trending repositories tagged turboquant, ranked by trending score. Updated daily.
-
1
JakeATX/llamAmpere
llama.cpp fork for significantly improved performance on Ampere generation Nvidia cards (especially RTX 3090 / 3090 Ti): example: 110+ tok/s over a 100K-token generation at temperature 1 for qwen3.8, supporting full 262K ctx. Supports low memory (12gb) configs for 200k+ ctx. 120+ TPS, 3300+ prefill for QFN 3.8.
GitHub · C++ · 406 stars · 51 forks · +6 stars in 24 hours · Trending score 2.24
-
2
manjunathshiva/turboquant-mlx
Extreme weight + KV cache compression for LLMs on Apple Silicon (MLX implementation of Google's TurboQuant)
GitHub · Python · 94 stars · 23 forks · +3 stars in 24 hours · Trending score 1.00
-
3
RyanCodrai/turbovec
A vector index built on TurboQuant, written in Rust with Python bindings
GitHub · Rust · 17,505 stars · 1,483 forks · +1 stars in 24 hours · Trending score 0.94
-
4
aivrar/vllm-windows-build
Native Windows vLLM 0.33.0 (Python 3.13/CUDA 13.0), plus a Python 3.14/CUDA 13.2 build; older versions on the Releases page. PyTorch 2.14, FlashAttention/Rust, OpenAI serving, 10 KV formats, Multi-TurboQuant, experimental CPU/NVMe KV offload. No WSL or Docker.
GitHub · Python · 68 stars · 9 forks · 0 stars in 24 hours · Trending score 0.53
-
5
jaylfc/taOS
Self-hosted AI agent OS. Your memory, chat, agents, and files stay on hardware you own, offline by default, cloud by choice. Offline AI memory (taOSmd), self-hosted multi-framework group chat, a full web desktop + app store, and auto-clustering across the consumer hardware you already have (Orange/Raspberry Pi, Mac mini, gaming PC).
GitHub · Python · 609 stars · 48 forks · 0 stars in 24 hours · Trending score 0.36
-
6
jamesarslan/local-ai-coding-setup
Local AI coding pipeline on an RTX 5090: llama.cpp, Qwen3.5/3.6 MoE, Gemma 4, TurboQuant KV, OpenCode, Claude Code, MCP. 188 t/s (Qwen3.5-35B-A3B, mainline, f16 KV); ~177 t/s (Qwen3.6-35B-A3B, turbo3, TLA=2). Optional cloud GPU server; default config uses hosted Context7 docs MCP.
GitHub · Shell · 22 stars · 4 forks · 0 stars in 24 hours · Trending score 0.22
-
7
Pushkinist/rMLX
Native Rust, single-binary MLX inference server for Apple Silicon. OpenAI/Anthropic-compatible LLM serving — text, vision, audio, embeddings — with the widest weight×KV quantization matrix of any MLX server. No Python, no GGUF.
GitHub · Rust · 18 stars · 7 forks · 0 stars in 24 hours · Trending score 0.02
-
8
Sggin1/DGX-SPARK
DGX Spark research and tests - containers, benchmarks, and investigation notes for running models on GB10 (SM 12.1)
GitHub · Python · 28 stars · 3 forks · 0 stars in 24 hours · Trending score 0.00
-
9
aivrar/multi-turboquant
Unified KV-cache compression for LLM inference: 12 Python-native methods, guarded add-on composition and routing, analytical capacity simulation, Godzilla KVarN/TriAttention with SM86/SM89 qualification, CUDA weight sharing, and multi-GPU planning.
GitHub · Python · 30 stars · 8 forks · 0 stars in 24 hours · Trending score 0.00
-
10
julianmb/q38rocm
Qwen 3.8 27B ROCmFP4 on AMD Strix Halo (Ryzen AI Max+ 395). Up to 36 tok/s via MTP Speculation, TurboQuant & Mesa RADV Wave64.
GitHub · Python · 270 stars · 12 forks · 0 stars in 24 hours · Trending score 0.00
-
11
FelipeMiiller/my-memory
🧠 Memória semântica e relacional unificada (Go, SQLite-Vec, TurboQuant 4-bit, Grafo CTEs e MCP) para Agentes de IA
GitHub · Go · 7 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00
-
12
Indras-Mirror/llama.cpp-turboq-mtp
Qwen4-Exp (Qwen3.8-Flash-Next) SWA + MTP + PLE-on-iswa — bounded deep-context decode w/ long-range recall, TBQ4 4.25 bpv KV, DSpark + RotorQuant. Main (master) = Qwen4-Exp build; the Qwen35 SWA hybrid lives on the qwen35 branch. RTX 4090.
GitHub · C++ · 94 stars · 13 forks · −2 stars in 24 hours · Trending score -0.00