GitHub · Rust
azrtydxb/Fastllm-proxy
The lowest-overhead LLM router. Production-ready, highly available, one OpenAI-compatible endpoint in front of 80 providers and your own vLLM/SGLang — 0.76 µs per request, no I/O on the request path, cache-affinity routing, RBAC, budgets and a 13-screen UI in the binary.
- Stars
- 109
- Forks
- 93
- Stars gained in 24 hours
- 0
- Forks gained in 24 hours
- 0
Topics: ai-gateway, inference, litellm, llm, llm-gateway, load-balancing, observability, openai-api, rbac, reverse-proxy
Latest data point: Nov 21, 2026 · 109 stars · 93 forks