RunanywhereAI/trnrt
The open LLM inference engine for AWS Trainium: NKI megakernels, 1M-token context, OpenAI-compatible API. By RunAnywhere.
GitHub repository with 1,585 stars and 18 forks.
Language: Python
Topics: aws, glm, inference, llm, llm-serving, megakernels, neuron, nki, openai-api, trainium