cosmonomic/pulsarlm
An LLM inference engine for persistent chat bots and agents. It allows models to forget old and unimportant tokens to keep context small and inference fast.
GitHub repository with 6 stars and 0 forks.
Language: C++
Topics: llm, llm-agents, llm-inference, llmops