1. alibaba/rtp-llm
RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.
GitHub repository with 1,181 stars and 204 forks.
Trending score: 1.09; stars gained: +9; forks gained: +0.
Language: Cuda
Topics: gpt, inference, llama, llm, llm-serving, llmops