model-inference repositories
Trending repositories tagged model-inference, ranked by trending score. Updated daily.
-
1
bentoml/OpenLLM
Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
GitHub · Python · 12,610 stars · 853 forks · +49 stars in 24 hours · Trending score 2.61
-
2
zadazddq/lm-studio-private-ai
LM Studio: Local LLM Desktop Runtime with Offline Inference and Model Management
GitHub · Rust · 149 stars · 0 forks · +21 stars in 24 hours · Trending score 1.50
-
3
jaedync/crucible-strix-halo
Run Qwen3.8-Flash-Next on one AMD Strix Halo (Ryzen AI Max+ 395, gfx1151). EXL3 4.05 and 3.05 bpw packs, own ROCm/HIP kernels, OpenAI-compatible server, 4 sessions of 262K context, setup guide included.
GitHub · Python · 35 stars · 7 forks · 0 stars in 24 hours · Trending score 0.78
-
4
wangxb96/Awesome-EdgeAI
Resources of our survey paper "Optimizing Edge AI: A Comprehensive Survey on Data, Model, and System Strategies" and "Cognitive Edge Computing: A Comprehensive Survey on Optimizing Large Models and AI Agents for Pervasive Deployment"
GitHub · Python · 122 stars · 22 forks · 0 stars in 24 hours · Trending score 0.29
-
5
hegongshan/Storage-for-AI-Paper
Accelerating AI Training and Inference from Storage Perspective (Must-read Papers on Storage for AI)
GitHub · 67 stars · 6 forks · 0 stars in 24 hours · Trending score 0.07
-
6
IBM/vLLM-Hook
Repo for vLLM Hook, an vLLM plug-in for programming internal states of models deployed on vLLM
GitHub · Python · 172 stars · 38 forks · 0 stars in 24 hours · Trending score 0.07