serving repositories

Trending repositories tagged serving, ranked by trending score. Updated daily.

  1. 1
    ray-project/ray

    Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

    GitHub · Python · 44,349 stars · 8,298 forks · +26 stars in 24 hours · Trending score 2.87

  2. 2
    skyzh/tiny-llm

    learn LLM inference system on Apple Silicon for systems engineers: build a tiny vLLM + Qwen

    GitHub · Python · 5,747 stars · 532 forks · 0 stars in 24 hours · Trending score 1.19

  3. 3
    volcano-sh/volcano

    A Cloud Native Batch System (Project under CNCF)

    GitHub · Go · 6,098 stars · 1,640 forks · 0 stars in 24 hours · Trending score 1.02

  4. 4
    dingodb/dingo

    A multi-modal vector database that supports upserts and vector queries using unified SQL (MySQL-Compatible) on structured and unstructured data, while meeting the requirements of high concurrency and ultra-low latency.

    GitHub · Java · 1,706 stars · 263 forks · +3 stars in 24 hours · Trending score 0.90

  5. 5
    openvinotoolkit/model_server

    A scalable inference server for models optimized with OpenVINO™

    GitHub · C++ · 961 stars · 297 forks · −1 stars in 24 hours · Trending score 0.64

  6. 6
    pixeltable/pixeltable

    The backend agents build with - Multimodal database, orchestration, and serving in one file

    GitHub · Python · 1,663 stars · 245 forks · −1 stars in 24 hours · Trending score 0.53

  7. 7
    deepjavalibrary/djl-serving

    A universal scalable machine learning model deployment solution

    GitHub · Java · 258 stars · 100 forks · 0 stars in 24 hours · Trending score 0.23

  8. 8
    tensorflow/serving

    A flexible, high-performance serving system for machine learning models

    GitHub · C++ · 6,362 stars · 2,201 forks · −1 stars in 24 hours · Trending score 0.09

  9. 9
    clearml/clearml-serving

    ClearML - Model-Serving Orchestration and Repository Solution

    GitHub · Python · 170 stars · 54 forks · 0 stars in 24 hours · Trending score 0.08

  10. 10
    polyaxon/haupt

    Lineage metadata API, artifacts streams, sandbox, API, and spaces for Polyaxon

    GitHub · Python · 452 stars · 204 forks · 0 stars in 24 hours · Trending score 0.04

  11. 11
    AI-Hypercomputer/gpu-recipes

    Recipes for reproducing training and serving benchmarks for large machine learning models using GPUs on Google Cloud.

    GitHub · Python · 148 stars · 94 forks · 0 stars in 24 hours · Trending score 0.03

  12. 12
    PaddlePaddle/FastDeploy

    High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle

    GitHub · Python · 3,722 stars · 757 forks · 0 stars in 24 hours · Trending score 0.00

  13. 13
    Lightning-AI/LitServe

    A minimal Python framework for building custom AI inference servers with full control over logic, batching, and scaling.

    GitHub · Python · 3,979 stars · 320 forks · 0 stars in 24 hours · Trending score 0.00