sizzlecar/ferrum-infer-rs

Production-grade LLM inference in Rust. Single binary, OpenAI-compatible, runs on Apple Silicon and CUDA.

GitHub repository with 6 stars and 0 forks.

Language: C++

Topics: apple-silicon, cuda, inference, inference-engine, llama, llm, metal, mixture-of-experts, moe, openai-api

Open provider repository

Latest metric snapshot

2026-06-15: 6 stars and 0 forks.

Similar repositories

  1. 1. widelands/widelands

    Widelands is a free, open source real-time strategy game with singleplayer campaigns and a multiplayer mode. The game was inspired by Settlers II™ (© Bluebyte) but has significantly more variety and depth to it.

    GitHub repository with 2,857 stars and 204 forks.

    Trending score: 0.65; stars gained: +0; forks gained: +2.

    Language: C++

    Topics: apple-silicon, bsd, c-plus-plus, cmake, floss, game

Trending in C++

  1. 1. ggml-org/llama.cpp

    LLM inference in C/C++

    GitHub repository with 116,605 stars and 19,593 forks.

    Trending score: 4.92; stars gained: +285; forks gained: +59.

    Language: C++

    Topics: ggml

  2. 2. opencv/opencv

    Open Source Computer Vision Library

    GitHub repository with 89,160 stars and 56,658 forks.

    Trending score: 4.35; stars gained: +147; forks gained: +16.

    Language: C++

    Topics: c-plus-plus, computer-vision, deep-learning, image-processing, opencv

  3. 3. alibaba/zvec

    A lightweight, lightning-fast, in-process vector database

    GitHub repository with 10,064 stars and 583 forks.

    Trending score: 4.23; stars gained: +283; forks gained: +17.

    Language: C++

    Topics: agent-skills, db, embedded, faiss, hnsw, llm-memory

  4. 4. ggml-org/whisper.cpp

    Port of OpenAI's Whisper model in C/C++

    GitHub repository with 50,727 stars and 5,661 forks.

    Trending score: 3.92; stars gained: +155; forks gained: +26.

    Language: C++

    Topics: inference, openai, speech-recognition, speech-to-text, transformer, whisper

  5. 5. noctalia-dev/noctalia

    A sleek and minimal desktop shell thoughtfully crafted for Wayland.

    GitHub repository with 7,754 stars and 545 forks.

    Trending score: 3.82; stars gained: +91; forks gained: +10.

    Language: C++

    Topics: dotfiles, hyprland, linux, niri, noctalia, quickshell

  6. 6. ml-explore/mlx

    MLX: An array framework for Apple silicon

    GitHub repository with 27,010 stars and 1,908 forks.

    Trending score: 3.68; stars gained: +58; forks gained: +10.

    Language: C++

    Topics: mlx

Trending topic: apple-silicon

  1. 1. Andyyyy64/whichllm

    Find the local LLM that actually runs and performs best on your hardware. Ranked by real, recency-aware benchmarks, not parameter count. One command, run it instantly.

    GitHub repository with 4,787 stars and 263 forks.

    Trending score: 4.13; stars gained: +62; forks gained: +5.

    Language: Python

    Topics: ai, cli, llm, local-llm, command-line-tool, gguf

  2. 2. jundot/omlx

    LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

    GitHub repository with 16,627 stars and 1,411 forks.

    Trending score: 3.94; stars gained: +104; forks gained: +2.

    Language: Python

    Topics: apple-silicon, inference-server, llm, macos, mlx, openai-api

  3. 3. raullenchai/Rapid-MLX

    The fastest local AI engine for Apple Silicon. 4.2x faster than Ollama, 0.08s cached TTFT, 100% tool calling. 17 tool parsers, prompt cache, reasoning separation, cloud routing. Drop-in OpenAI replacement. Works with Claude Code, Cursor, Aider.

    GitHub repository with 2,800 stars and 342 forks.

    Trending score: 2.91; stars gained: +30; forks gained: +0.

    Language: Python

    Topics: apple-silicon, claude-code, cursor, deepseek, fastapi, hacktoberfest

  4. 4. Arthur-Ficial/apfel

    The free AI already on your Mac. CLI tool, OpenAI-compatible server, and interactive chat — all on-device via Apple Intelligence. No API keys, no cloud, no downloads.

    GitHub repository with 5,779 stars and 216 forks.

    Trending score: 2.70; stars gained: +16; forks gained: +0.

    Language: Swift

    Topics: apple-intelligence, apple-silicon, cli, foundationmodels, homebrew, llm

  5. 5. Blaizzy/mlx-vlm

    MLX-VLM is a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac using MLX.

    GitHub repository with 5,057 stars and 592 forks.

    Trending score: 2.57; stars gained: +14; forks gained: +2.

    Language: Python

    Topics: llava, llm, mlx, vision-transformer, apple-silicon, idefics

  6. 6. soniqo/speech-swift

    AI speech toolkit for Apple Silicon — ASR, TTS, speech-to-speech, VAD, and diarization powered by MLX and CoreML

    GitHub repository with 888 stars and 114 forks.

    Trending score: 2.57; stars gained: +13; forks gained: +1.

    Language: Swift

    Topics: apple-silicon, asr, coreml, ios, macos, mlx