serving repositories
Trending repositories tagged serving, ranked by trending score. Updated daily.
-
1
ray-project/ray
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
GitHub · Python · 44,349 stars · 8,298 forks · +26 stars in 24 hours · Trending score 2.87
-
2
skyzh/tiny-llm
learn LLM inference system on Apple Silicon for systems engineers: build a tiny vLLM + Qwen
GitHub · Python · 5,747 stars · 532 forks · 0 stars in 24 hours · Trending score 1.19
-
3
volcano-sh/volcano
A Cloud Native Batch System (Project under CNCF)
GitHub · Go · 6,098 stars · 1,640 forks · 0 stars in 24 hours · Trending score 1.02
-
4
dingodb/dingo
A multi-modal vector database that supports upserts and vector queries using unified SQL (MySQL-Compatible) on structured and unstructured data, while meeting the requirements of high concurrency and ultra-low latency.
GitHub · Java · 1,706 stars · 263 forks · +3 stars in 24 hours · Trending score 0.90
-
5
openvinotoolkit/model_server
A scalable inference server for models optimized with OpenVINO™
GitHub · C++ · 961 stars · 297 forks · −1 stars in 24 hours · Trending score 0.64
-
6
pixeltable/pixeltable
The backend agents build with - Multimodal database, orchestration, and serving in one file
GitHub · Python · 1,663 stars · 245 forks · −1 stars in 24 hours · Trending score 0.53
-
7
deepjavalibrary/djl-serving
A universal scalable machine learning model deployment solution
GitHub · Java · 258 stars · 100 forks · 0 stars in 24 hours · Trending score 0.23
-
8
tensorflow/serving
A flexible, high-performance serving system for machine learning models
GitHub · C++ · 6,362 stars · 2,201 forks · −1 stars in 24 hours · Trending score 0.09
-
9
clearml/clearml-serving
ClearML - Model-Serving Orchestration and Repository Solution
GitHub · Python · 170 stars · 54 forks · 0 stars in 24 hours · Trending score 0.08
-
10
polyaxon/haupt
Lineage metadata API, artifacts streams, sandbox, API, and spaces for Polyaxon
GitHub · Python · 452 stars · 204 forks · 0 stars in 24 hours · Trending score 0.04
-
11
AI-Hypercomputer/gpu-recipes
Recipes for reproducing training and serving benchmarks for large machine learning models using GPUs on Google Cloud.
GitHub · Python · 148 stars · 94 forks · 0 stars in 24 hours · Trending score 0.03
-
12
PaddlePaddle/FastDeploy
High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle
GitHub · Python · 3,722 stars · 757 forks · 0 stars in 24 hours · Trending score 0.00
-
13
Lightning-AI/LitServe
A minimal Python framework for building custom AI inference servers with full control over logic, batching, and scaling.
GitHub · Python · 3,979 stars · 320 forks · 0 stars in 24 hours · Trending score 0.00