ai-inference repositories

Trending repositories tagged ai-inference, ranked by trending score. Updated daily.

  1. 1
    bentoml/BentoML

    The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

    GitHub · Python · 8,941 stars · 1,065 forks · 0 stars in 24 hours · Trending score 1.20

  2. 2
    team-telnyx/telnyx-code-examples

    Production-ready code examples for Telnyx AI Communications Infrastructure — Voice AI, SMS, SIP, and IoT APIs

    GitHub · TypeScript · 169 stars · 11 forks · +3 stars in 24 hours · Trending score 1.10

  3. 3
    uxlfoundation/oneDAL

    oneAPI Data Analytics Library (oneDAL)

    GitHub · C++ · 654 stars · 232 forks · +1 stars in 24 hours · Trending score 0.64

  4. 4
    Genesi-OS/GenesiOS

    A CachyOS-based, developer-first Linux distribution that retunes itself for local inference — dressed in a glass KDE Plasma desktop.

    GitHub · Python · 14 stars · 0 forks · +1 stars in 24 hours · Trending score 0.56

  5. 5
    IBM/vLLM-Hook

    Repo for vLLM Hook, an vLLM plug-in for programming internal states of models deployed on vLLM

    GitHub · Python · 172 stars · 38 forks · 0 stars in 24 hours · Trending score 0.50

  6. 6
    MerkyorLynn/lynn-engine

    Lynn 原生 LLM 推理引擎 · W4A8/NVFP4 量化 · 自写 CUDA/Triton kernel · MoE · 投机解码 | Lynn-native LLM inference engine for NVIDIA Blackwell

    GitHub · Python · 28 stars · 0 forks · 0 stars in 24 hours · Trending score 0.48

  7. 7
    andrew-serrao/bloomgauge

    BloomGauge (formerly Bloomkeeper): free Darkbloom manager and earnings dashboard for Apple Silicon Macs. Source-available core (PolyForm Shield). Keeps the best-paying model loaded, recovers stalls, no Terminal needed.

    GitHub · Python · 25 stars · 5 forks · 0 stars in 24 hours · Trending score 0.45

  8. 8
    avikeid2007/KaiROS-AI

    KaiROS AI— Intelligence, Precisely When It Matters.

    GitHub · C# · 27 stars · 4 forks · 0 stars in 24 hours · Trending score 0.43

  9. 9
    cipherflow-fhe/latti-ai

    A framework for performing AI model inference on encrypted data.

    GitHub · C++ · 10 stars · 0 forks · 0 stars in 24 hours · Trending score 0.42

  10. 10
    surendranb/free-inference

    Daily-updated catalog of free LLM inference providers — for humans and agents. Verified nightly, PRs welcome.

    GitHub · Python · 6 stars · 3 forks · +1 stars in 24 hours · Trending score 0.42

  11. 11
    uxlfoundation/scikit-learn-intelex

    Extension for Scikit-learn is a seamless way to speed up your Scikit-learn application

    GitHub · Python · 1,358 stars · 198 forks · 0 stars in 24 hours · Trending score 0.19

  12. 12
    ProjectZeroDays/FreeAI_AI_Inference_Workstation

    FreeAI: Unified AI Inference Workstation — local LLMs, autonomous agents, full SDLC automation, security tools, and multi-provider routing. One self-hosted stack.

    GitHub · Python · 7 stars · 1 forks · 0 stars in 24 hours · Trending score 0.14

  13. 13
    FerrumVir/arc-chain

    Rust L1 testnet. A 7-billion-parameter model gave the same answer, bit for bit, on the Linux, Windows, Intel Mac and Apple Silicon CPUs we tested.

    GitHub · Rust · 25 stars · 7 forks · 0 stars in 24 hours · Trending score 0.10

  14. 14
    xflops/flame

    A distributed engine for elastic workload, e.g. AI Agent, RL, Quant

    GitHub · Rust · 57 stars · 13 forks · 0 stars in 24 hours · Trending score 0.09

  15. 15
    ronitgupta138/Local-Jev

    ⚡ 100% offline, zero-cost drop-in alternative to TypeSafe Jev System One decision models in Rust (<18ms on CPU)

    GitHub · Rust · 11 stars · 1 forks · 0 stars in 24 hours · Trending score 0.04

  16. 16
    Scottcjn/ram-coffers

    LLM infrastructure cost reduction via NUMA-aware weight banking: 147 t/s (8.8x stock llama.cpp) on refurbished enterprise POWER8. Self-hosted inference, no cloud APIs. Part of the Proof of Physical AI stack.

    GitHub · Python · 173 stars · 59 forks · 0 stars in 24 hours · Trending score 0.01

  17. 17
    Intrafere/MOTO-Autonomous-ASI

    MOTO is an automated theorem generator for science. It's a creative novelty-seeking researcher with autonomous Lean 4 proof generation. Run for days at a time once pressing start - no interaction needed! Agents working in parallel from either local host LM studio, OpenRouter, OAuth or all 3. No internet required. Star us for more!

    GitHub · Python · 90 stars · 14 forks · 0 stars in 24 hours · Trending score 0.00

  18. 18
    Scottcjn/llama-cpp-power8

    llama.cpp optimizations for IBM POWER8: vec_perm non-bijunctive collapse, PSE hardware entropy, DCBT prefetch. Sovereign inference. Part of the Proof of Physical AI stack.

    GitHub · C · 130 stars · 18 forks · 0 stars in 24 hours · Trending score 0.00

  19. 19
    Quazmoz/InferBridge

    Windows-first OpenAI-compatible local LLM server powered by OpenVINO GenAI for Intel CPU/GPU/NPU, with chat UI, model conversion, and setup scripts.

    GitHub · Python · 51 stars · 3 forks · 0 stars in 24 hours · Trending score 0.00

  20. 20
    blace-ai/blace-ai

    Cross-platform c++ sdk & model hub for cross-platform AI inference. Ready-to-deploy models including Segment Anything 3, Depth Anything 2 and Gemma.

    GitHub · C++ · 76 stars · 3 forks · 0 stars in 24 hours · Trending score 0.00

  21. 21
    Josia-doit/ComfyUI_JosiaNodes

    一套功能完整、轻量稳定的 ComfyUI 实用节点集。初衷是为了平替一些断更插件的功能,会持续更新一些新开发的节点插件。有好的创意可以发Issues,我会力所能及想办法做出来。全中文界面,原生控件,不依赖第三方库。

    GitHub · JavaScript · 37 stars · 5 forks · 0 stars in 24 hours · Trending score 0.00

  22. 22
    kamalesh404/UnrestrictedLLM

    Run uncensored open-source LLMs locally or on cloud. GGUF/SafeTensors support, OpenAI-compatible API, multiple backends, CLI, and web UI.

    GitHub · Python · 9 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00

  23. 23
    pixelrhino-ai/vrhino

    A self-contained native runtime and model format for local AI video generation.

    GitHub · C++ · 103 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00

  24. 24
    alphaparkinc/genpark-dense-feedforward-mlp-backprop-skill

    Multi-layer perceptron (MLP) feedforward inference with Xavier/Glorot initialization and activations

    GitHub · Python · 7 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00