batch-inference repositories
Trending repositories tagged batch-inference, ranked by trending score. Updated daily.
-
1
davanstrien/saturate
Batch inference for datasets: rows in, any OpenAI-compatible endpoint, resumable parquet out. Congestion-aware concurrency — no tuning.
GitHub · Python · 56 stars · 2 forks · 0 stars in 24 hours · Trending score 0.71
-
2
Artain-AI/ignite-ms
Fast self-hosted embedding engine for search, RAG, and reindexing workloads on NVIDIA GPUs. Built in Rust + TensorRT for teams that care about scale, cost, and control.
GitHub · Rust · 9 stars · 0 forks · 0 stars in 24 hours · Trending score 0.14
-
3
spinfinc/spinf-serve
Bulk classification and decision inference on one GPU: the content is read once, every question about it is scored in the same pass. OpenAI-compatible, Apache 2.0
GitHub · Python · 21 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00