Air.rs 70B+ inference on consumer GPU, LLM inference in Rust
GitHub repository with 10 stars and 0 forks.
Language: Rust
Topics: inference, ggml, instruction-set, lora, open-models, open-source, qlora, apple-silicon, kernel, llama-cpp
2026-06-05: 10 stars and 0 forks.
Camelid: a Rust-native local inference backend with evidence-gated model compatibility.
GitHub repository with 53 stars and 10 forks.
Trending score: 1.25; stars gained: +17; forks gained: +2.
Language: Rust
Topics: apple-silicon, gguf, inference, llama, llm, local-first
Object-storage-native KV cache for LLM inference & RL. Cross-restart, cross-conversation, cross-engine via shared S3 bucket.
GitHub repository with 12 stars and 1 forks.
Trending score: 0.33; stars gained: +1; forks gained: +0.
Language: Rust
Topics: amd, caching, ds4, inference, kv-cache, llm
An enhanced tool for CodexApp, striving to make Codex better to use and more comfortable 一个CodexApp的增强工具，努力让Codex变得更好用更舒服
GitHub repository with 14,059 stars and 871 forks.
Trending score: 5.16; stars gained: +916; forks gained: +44.
Language: Rust
CLI proxy that reduces LLM token consumption by 60-90% on common dev commands. Single Rust binary, zero dependencies
GitHub repository with 59,182 stars and 3,643 forks.
Trending score: 4.96; stars gained: +654; forks gained: +44.
Language: Rust
Topics: agentic-coding, ai-coding, anthropic, claude-code, cli, command-line-tool
Lightweight coding agent that runs in your terminal
GitHub repository with 88,943 stars and 13,070 forks.
Trending score: 4.58; stars gained: +326; forks gained: +48.
Language: Rust
Your Personal AI super intelligence. Private, Simple and extremely powerful.
GitHub repository with 30,892 stars and 2,983 forks.
Trending score: 4.37; stars gained: +332; forks gained: +50.
Language: Rust
Codebase intelligence for TypeScript and JavaScript. Free static layer: unused code, duplication, circular deps, complexity hotspots, architecture boundaries. Optional paid runtime layer: hot-path review and cold-path deletion evidence from real production traffic. Rust-native, sub-second, zero-config framework support.
GitHub repository with 3,118 stars and 96 forks.
Trending score: 4.05; stars gained: +346; forks gained: +16.
Language: Rust
Topics: cli, code-duplication, code-quality, codebase-intelligence, copy-paste-detection, dead-code
High performance object store for fast LLM Inference and GPU Training. Feed your GPUs at blazing fast speeds
GitHub repository with 1,118 stars and 176 forks.
Trending score: 4.00; stars gained: +244; forks gained: +120.
Language: Rust
Topics: blackwell, gpt, gpu, high-performance, llm, llm-training
A high-throughput and memory-efficient inference and serving engine for LLMs
GitHub repository with 82,008 stars and 17,695 forks.
Trending score: 3.75; stars gained: +79; forks gained: +46.
Language: Python
Topics: amd, blackwell, cuda, deepseek, deepseek-v3, gpt
Community maintained hardware plugin for vLLM on Ascend
GitHub repository with 2,201 stars and 1,350 forks.
Trending score: 3.25; stars gained: +16; forks gained: +22.
Language: C++
Topics: ascend, inference, llm, llm-serving, llmops, mlops
SGLang is a high-performance serving framework for large language models and multimodal models.
GitHub repository with 28,866 stars and 6,352 forks.
Trending score: 1.72; stars gained: -55; forks gained: +18.
Language: Python
Topics: attention, blackwell, cuda, deepseek, diffusion, glm
A framework for efficient model inference with omni-modality models
GitHub repository with 4,958 stars and 1,056 forks.
Trending score: 1.61; stars gained: +49; forks gained: +18.
Language: Python
Topics: audio-generation, diffusion, image-generation, inference, model-serving, multimodal
Cross-platform, customizable ML solutions for live and streaming media.
GitHub repository with 35,492 stars and 6,003 forks.
Trending score: 1.41; stars gained: +29; forks gained: -2.
Language: C++
Topics: mediapipe, c-plus-plus, computer-vision, deep-learning, android, video-processing
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
GitHub repository with 42,472 stars and 4,850 forks.
Trending score: 1.40; stars gained: +21; forks gained: +1.
Language: Python
Topics: deep-learning, pytorch, gpu, machine-learning, billion-parameters, data-parallelism