batch-inference repositories

Trending repositories tagged batch-inference, ranked by trending score. Updated daily.

  1. 1
    davanstrien/saturate

    Batch inference for datasets: rows in, any OpenAI-compatible endpoint, resumable parquet out. Congestion-aware concurrency — no tuning.

    GitHub · Python · 56 stars · 2 forks · 0 stars in 24 hours · Trending score 0.71

  2. 2
    Artain-AI/ignite-ms

    Fast self-hosted embedding engine for search, RAG, and reindexing workloads on NVIDIA GPUs. Built in Rust + TensorRT for teams that care about scale, cost, and control.

    GitHub · Rust · 9 stars · 0 forks · 0 stars in 24 hours · Trending score 0.14

  3. 3
    spinfinc/spinf-serve

    Bulk classification and decision inference on one GPU: the content is read once, every question about it is scored in the same pass. OpenAI-compatible, Apache 2.0

    GitHub · Python · 21 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00