rewroy/Straydog
Local LLM inference server in a single executable, with an OpenAI-compatible API. Runs DeepSeek-V4.1-Flash and Qwen3.8-Flash-Next GGUFs on AMD (ROCm) and NVIDIA (CUDA) GPUs, with exact numerics and a sandboxed engine.
GitHub repository with 6 stars and 0 forks.
Language: Rust