spinfinc/spinf-serve
Bulk classification and decision inference on one GPU: the content is read once, every question about it is scored in the same pass. OpenAI-compatible, Apache 2.0
GitHub repository with 21 stars and 0 forks.
Language: Python
Topics: batch-inference, classification, content-moderation, decision-model, extraction, fp8, gemma, inference-engine, llm, openai-compatible