model-inference repositories

Trending repositories tagged model-inference, ranked by trending score. Updated daily.

  1. 1
    bentoml/OpenLLM

    Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

    GitHub · Python · 12,610 stars · 853 forks · +49 stars in 24 hours · Trending score 2.61

  2. 2
    zadazddq/lm-studio-private-ai

    LM Studio: Local LLM Desktop Runtime with Offline Inference and Model Management

    GitHub · Rust · 149 stars · 0 forks · +21 stars in 24 hours · Trending score 1.50

  3. 3
    jaedync/crucible-strix-halo

    Run Qwen3.8-Flash-Next on one AMD Strix Halo (Ryzen AI Max+ 395, gfx1151). EXL3 4.05 and 3.05 bpw packs, own ROCm/HIP kernels, OpenAI-compatible server, 4 sessions of 262K context, setup guide included.

    GitHub · Python · 35 stars · 7 forks · 0 stars in 24 hours · Trending score 0.78

  4. 4
    wangxb96/Awesome-EdgeAI

    Resources of our survey paper "Optimizing Edge AI: A Comprehensive Survey on Data, Model, and System Strategies" and "Cognitive Edge Computing: A Comprehensive Survey on Optimizing Large Models and AI Agents for Pervasive Deployment"

    GitHub · Python · 122 stars · 22 forks · 0 stars in 24 hours · Trending score 0.29

  5. 5
    hegongshan/Storage-for-AI-Paper

    Accelerating AI Training and Inference from Storage Perspective (Must-read Papers on Storage for AI)

    GitHub · 67 stars · 6 forks · 0 stars in 24 hours · Trending score 0.07

  6. 6
    IBM/vLLM-Hook

    Repo for vLLM Hook, an vLLM plug-in for programming internal states of models deployed on vLLM

    GitHub · Python · 172 stars · 38 forks · 0 stars in 24 hours · Trending score 0.07