cpu-inference repositories
Trending repositories tagged cpu-inference, ranked by trending score. Updated daily.
-
1
FareedKhan-dev/kimi-k3-in-c
A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.
GitHub · C · 12,510 stars · 1,935 forks · +127 stars in 24 hours · Trending score 4.17
-
2
synchopate/llama-modchip
Zero-GPU CPU inference for 30B-284B MoE models (Qwen3.8 176.5B cold-cache certified in 28.26/48.35 GiB Cgroup v2, DeepSeek-V4 284B, Nemotron-30B) + RouteCrack 2.0.
GitHub · Python · 1,393 stars · 135 forks · +33 stars in 24 hours · Trending score 3.35
-
3
saurabhv749/supertonic3-voice-clone
Train voice styles for Supertone/supertonic-3 model.
GitHub · Python · 89 stars · 25 forks · +13 stars in 24 hours · Trending score 1.95
-
4
shyringo/qwen3.8-flash-next-in-c
The best model under 200B at near 10 token/s exact batch throughput on a single laptop CPU, with an automatic 8 GB RAM path. Native C, no GPU or Python. | 200B 以下最强模型,单颗笔记本 CPU 精确批处理接近 10 token/s,最低 8 GB 内存自动运行。原生 C 语言,无需 GPU 或 Python。
GitHub · C · 24 stars · 8 forks · +6 stars in 24 hours · Trending score 1.45
-
5
prabanta-dev/janas
37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.
GitHub · C · 735 stars · 73 forks · 0 stars in 24 hours · Trending score 1.28
-
6
DaggerSlicer34/koboldcpp
KoboldCpp — local llm cpu.
GitHub · 21 stars · 0 forks · +13 stars in 24 hours · Trending score 1.20
-
7
yanfei0725/dsh-qwen-paint
DSH 插件 / DeepSeek Harness:视频 / 生图 / TTS 全能 · 本机运行(QQ 交流群 454707710)。★ 0.33.0 起许可★ 0.33.0 起许可改为 BSL 1.1(BUSL-1.1,非 OSI 开源许可):除托管/代管服务与直接竞品捆绑两类「竞争性使用」外免费生产使用,2030-11-15 转 Apache-2.0;0.32.1 及更早仍 MIT、不追溯。★ 1.0.0:选模型「假成功」根治(拒绝时如实回未生效 + ★ 1.0.0:选模型「假成功」根治(拒绝时如实回未生效 + 原因,不再无条件报已选)、优化面板与素材库白条同源修复(持久化缓存
GitHub · JavaScript · 45 stars · 0 forks · +1 stars in 24 hours · Trending score 0.87
-
8
shyringo/qwen3.8-27b-in-c
Qwen3.8-27B on one laptop CPU: up to 2.52 token/s, 8 GB tested, no runtime accuracy loss. Native OpenAI-compatible API with function tools; no GPU or Python. | 单颗笔记本 CPU 运行 Qwen3.8-27B:最快 2.52 token/s,最低 8 GB 内存可运行,推理加速不牺牲准确性。原生 OpenAI 兼容接口支持函数工具;无需 GPU 或 Python。
GitHub · C · 24 stars · 6 forks · 0 stars in 24 hours · Trending score 0.42
-
9
sauravsingla/MemVanta
Low-memory C++20 LLM inference runtime for quantized GGUF models on CPU — mmap-backed weights, paged KV cache, Q4/Q8 kernels, and reproducible llama.cpp benchmarks.
GitHub · C++ · 41 stars · 40 forks · 0 stars in 24 hours · Trending score 0.37
-
10
maheshvaikri-code/snapllm
🔥 Alternative to Ollama — multi-model serving with sub-ms model switching · CPU-only 20B inference for Edge AI · llama.cpp + stablediffusion.cpp
GitHub · C++ · 44 stars · 4 forks · 0 stars in 24 hours · Trending score 0.32
-
11
DavidTorro/gdg-uam-ml-contest-2026
1D CNN for 8-class multichannel time-series classification, optimized for CPU inference with ONNX Runtime and Numba.
GitHub · Python · 7 stars · 0 forks · 0 stars in 24 hours · Trending score 0.26
-
12
siris9476/pulsarforge
GitHub · C · 24 stars · 4 forks · 0 stars in 24 hours · Trending score 0.14
-
13
Dicklesworthstone/franken_nlp
Pure-Rust, CPU-optimized local NLP toolbox built around one model (Nanbeige4.2-3B): schema-valid structured extraction, NER and entity resolution, sentiment, zero-shot classification, PII redaction, faithfulness judging, QA, and text generation. One library plus the fnlp CLI; no Python, CUDA, or GPU. Pre-release.
GitHub · Rust · 7 stars · 2 forks · 0 stars in 24 hours · Trending score 0.10
-
14
Classevelabs/rai
Run language models on your own computer's CPU: one-step installers for Windows, macOS and Linux, a local chat page and API, 4-bit models, pure Rust. No GPU, no Python, nothing leaves your machine.
GitHub · Rust · 16 stars · 1 forks · 0 stars in 24 hours · Trending score 0.08
-
15
ranjithrajv/awesome-cpu-first-ai
Curated, evidence-backed list of runtimes, formats & tools for running AI inference on CPU — start with CPU, justify the GPU.
GitHub · Python · 5 stars · 4 forks · 0 stars in 24 hours · Trending score 0.07
-
16
Endgame-Labs/understudy
A drop-in, CPU-only proxy for System One models (Jev, Rune, Clef). It learns each repeated question from the lead, then answers it itself.
GitHub · Rust · 7 stars · 1 forks · 0 stars in 24 hours · Trending score 0.05
-
17
benmaster82/picchio
A streaming Mixture-of-Experts (MoE) inference engine written in pure C, that runs models larger than your RAM on ordinary consumer hardware - MiniMax-M2 (230B) - GPT-OSS (20B and 120B) and Qwen3-MoE
GitHub · C · 11 stars · 3 forks · 0 stars in 24 hours · Trending score 0.04
-
18
ronitgupta138/Local-Jev
⚡ 100% offline, zero-cost drop-in alternative to TypeSafe Jev System One decision models in Rust (<18ms on CPU)
GitHub · Rust · 11 stars · 1 forks · 0 stars in 24 hours · Trending score 0.03
-
19
mcpware/IntentProbe
CPU activation probes for AI-agent guardrails and classification. One frozen local model pass, many independent heads. Open models, code and reproducible benchmarks.
GitHub · Python · 9 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00
-
20
brontoguana/krasis
Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware
GitHub · C++ · 522 stars · 33 forks · 0 stars in 24 hours · Trending score 0.00
-
21
feder-cr/jev
jevos: an open-source alternative to Jev and Microsoft-Decision-1. Yes/no decisions, choices and scores with probabilities, on your laptop's CPU.
GitHub · C++ · 1,345 stars · 163 forks · 0 stars in 24 hours · Trending score 0.00
-
22
shyringo/deepseek-v4-flash-0731-in-c
Run the native 284B-A13B DeepSeek-V4-Flash-0731 LLM locally on one laptop CPU: pure C, 8 GB RAM minimum, no GPU, best TPOT 0.892 s/token, resident OpenAI-compatible API with function tools. | 在笔记本单颗 CPU 上本地运行原生 284B-A13B DeepSeek-V4-Flash-0731 大模型:纯 C,最低 8 GB 内存,无需 GPU,最优 TPOT 0.892 秒/token,支持模型常驻的 OpenAI 兼容接口与函数工具。
GitHub · C · 29 stars · 5 forks · 0 stars in 24 hours · Trending score 0.00
-
23
Tahirpathan-AiLab/Evaan_Personal_Intelligence_Engine
Evaan — a fully local, offline-capable AI companion chatbot built with Python & Hugging Face Transformers, running on Qwen2.5-0.5B-Instruct (CPU-only, no fine-tuning). Features a persona-driven system prompt, rule-based mood/tone detection, and persistent JSON memory across sessions — no API key or internet needed after setup.
GitHub · Python · 23 stars · 4 forks · 0 stars in 24 hours · Trending score 0.00
-
24
IronHpc/UVR-rs
群友让我优化一下UVR5的4个常用的模型的推理速度,以及运行时环境的大小
GitHub · Rust · 5 stars · 1 forks · 0 stars in 24 hours · Trending score 0.00