ai-inference repositories
Trending repositories tagged ai-inference, ranked by trending score. Updated daily.
-
1
bentoml/BentoML
The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
GitHub · Python · 8,941 stars · 1,065 forks · 0 stars in 24 hours · Trending score 1.20
-
2
team-telnyx/telnyx-code-examples
Production-ready code examples for Telnyx AI Communications Infrastructure — Voice AI, SMS, SIP, and IoT APIs
GitHub · TypeScript · 169 stars · 11 forks · +3 stars in 24 hours · Trending score 1.10
-
3
uxlfoundation/oneDAL
oneAPI Data Analytics Library (oneDAL)
GitHub · C++ · 654 stars · 232 forks · +1 stars in 24 hours · Trending score 0.64
-
4
Genesi-OS/GenesiOS
A CachyOS-based, developer-first Linux distribution that retunes itself for local inference — dressed in a glass KDE Plasma desktop.
GitHub · Python · 14 stars · 0 forks · +1 stars in 24 hours · Trending score 0.56
-
5
IBM/vLLM-Hook
Repo for vLLM Hook, an vLLM plug-in for programming internal states of models deployed on vLLM
GitHub · Python · 172 stars · 38 forks · 0 stars in 24 hours · Trending score 0.50
-
6
MerkyorLynn/lynn-engine
Lynn 原生 LLM 推理引擎 · W4A8/NVFP4 量化 · 自写 CUDA/Triton kernel · MoE · 投机解码 | Lynn-native LLM inference engine for NVIDIA Blackwell
GitHub · Python · 28 stars · 0 forks · 0 stars in 24 hours · Trending score 0.48
-
7
andrew-serrao/bloomgauge
BloomGauge (formerly Bloomkeeper): free Darkbloom manager and earnings dashboard for Apple Silicon Macs. Source-available core (PolyForm Shield). Keeps the best-paying model loaded, recovers stalls, no Terminal needed.
GitHub · Python · 25 stars · 5 forks · 0 stars in 24 hours · Trending score 0.45
-
8
avikeid2007/KaiROS-AI
KaiROS AI— Intelligence, Precisely When It Matters.
GitHub · C# · 27 stars · 4 forks · 0 stars in 24 hours · Trending score 0.43
-
9
cipherflow-fhe/latti-ai
A framework for performing AI model inference on encrypted data.
GitHub · C++ · 10 stars · 0 forks · 0 stars in 24 hours · Trending score 0.42
-
10
surendranb/free-inference
Daily-updated catalog of free LLM inference providers — for humans and agents. Verified nightly, PRs welcome.
GitHub · Python · 6 stars · 3 forks · +1 stars in 24 hours · Trending score 0.42
-
11
uxlfoundation/scikit-learn-intelex
Extension for Scikit-learn is a seamless way to speed up your Scikit-learn application
GitHub · Python · 1,358 stars · 198 forks · 0 stars in 24 hours · Trending score 0.19
-
12
ProjectZeroDays/FreeAI_AI_Inference_Workstation
FreeAI: Unified AI Inference Workstation — local LLMs, autonomous agents, full SDLC automation, security tools, and multi-provider routing. One self-hosted stack.
GitHub · Python · 7 stars · 1 forks · 0 stars in 24 hours · Trending score 0.14
-
13
FerrumVir/arc-chain
Rust L1 testnet. A 7-billion-parameter model gave the same answer, bit for bit, on the Linux, Windows, Intel Mac and Apple Silicon CPUs we tested.
GitHub · Rust · 25 stars · 7 forks · 0 stars in 24 hours · Trending score 0.10
-
14
xflops/flame
A distributed engine for elastic workload, e.g. AI Agent, RL, Quant
GitHub · Rust · 57 stars · 13 forks · 0 stars in 24 hours · Trending score 0.09
-
15
ronitgupta138/Local-Jev
⚡ 100% offline, zero-cost drop-in alternative to TypeSafe Jev System One decision models in Rust (<18ms on CPU)
GitHub · Rust · 11 stars · 1 forks · 0 stars in 24 hours · Trending score 0.04
-
16
Scottcjn/ram-coffers
LLM infrastructure cost reduction via NUMA-aware weight banking: 147 t/s (8.8x stock llama.cpp) on refurbished enterprise POWER8. Self-hosted inference, no cloud APIs. Part of the Proof of Physical AI stack.
GitHub · Python · 173 stars · 59 forks · 0 stars in 24 hours · Trending score 0.01
-
17
Intrafere/MOTO-Autonomous-ASI
MOTO is an automated theorem generator for science. It's a creative novelty-seeking researcher with autonomous Lean 4 proof generation. Run for days at a time once pressing start - no interaction needed! Agents working in parallel from either local host LM studio, OpenRouter, OAuth or all 3. No internet required. Star us for more!
GitHub · Python · 90 stars · 14 forks · 0 stars in 24 hours · Trending score 0.00
-
18
Scottcjn/llama-cpp-power8
llama.cpp optimizations for IBM POWER8: vec_perm non-bijunctive collapse, PSE hardware entropy, DCBT prefetch. Sovereign inference. Part of the Proof of Physical AI stack.
GitHub · C · 130 stars · 18 forks · 0 stars in 24 hours · Trending score 0.00
-
19
Quazmoz/InferBridge
Windows-first OpenAI-compatible local LLM server powered by OpenVINO GenAI for Intel CPU/GPU/NPU, with chat UI, model conversion, and setup scripts.
GitHub · Python · 51 stars · 3 forks · 0 stars in 24 hours · Trending score 0.00
-
20
blace-ai/blace-ai
Cross-platform c++ sdk & model hub for cross-platform AI inference. Ready-to-deploy models including Segment Anything 3, Depth Anything 2 and Gemma.
GitHub · C++ · 76 stars · 3 forks · 0 stars in 24 hours · Trending score 0.00
-
21
Josia-doit/ComfyUI_JosiaNodes
一套功能完整、轻量稳定的 ComfyUI 实用节点集。初衷是为了平替一些断更插件的功能,会持续更新一些新开发的节点插件。有好的创意可以发Issues,我会力所能及想办法做出来。全中文界面,原生控件,不依赖第三方库。
GitHub · JavaScript · 37 stars · 5 forks · 0 stars in 24 hours · Trending score 0.00
-
22
kamalesh404/UnrestrictedLLM
Run uncensored open-source LLMs locally or on cloud. GGUF/SafeTensors support, OpenAI-compatible API, multiple backends, CLI, and web UI.
GitHub · Python · 9 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00
-
23
pixelrhino-ai/vrhino
A self-contained native runtime and model format for local AI video generation.
GitHub · C++ · 103 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00
-
24
alphaparkinc/genpark-dense-feedforward-mlp-backprop-skill
Multi-layer perceptron (MLP) feedforward inference with Xavier/Glorot initialization and activations
GitHub · Python · 7 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00