StartLuxLabs/StartLux-Prism
Fast, exact inference for StartLux-Decision models on NVIDIA GPUs: the released GGUF files on int8 and 4-bit tensor cores, behind the /v1/systemone API, with up to 5.3× lower latency than vLLM in FP8 and bit-identical answers.
GitHub repository with 254 stars and 5 forks.
Language: Python
Topics: calibration, cuda, cuda-graphs, decision-making, gguf, h200, inference-engine, int8, llm-inference, nvidia