efficient-inference repositories

Trending repositories tagged efficient-inference, ranked by trending score. Updated daily.

  1. 1
    cvlab-kaist/GRACE

    Official implementation of "GRACE: Generation-Aware Latent Compression for Efficient Video Generation" (arXiv 2610.10524)

    GitHub · Python · 34 stars · 3 forks · +9 stars in 24 hours · Trending score 1.62

  2. 2
    LynnReal-AI/LynnReal

    Hand-controlled video world models with full-sequence and three-step causal generation.

    GitHub · Python · 80 stars · 2 forks · +19 stars in 24 hours · Trending score 1.45

  3. 3
    AI-Efficiency/Awesome-Efficient-Diffusion

    A curated list of papers and code on efficient diffusion models for image, video, world modeling, and language generation. Covering acceleration, quantization, compression, caching, and distillation.

    GitHub · 212 stars · 11 forks · +3 stars in 24 hours · Trending score 1.16

  4. 4
    AI-Efficiency/Awesome-Model-Quantization

    A curated collection of papers, benchmarks, surveys, and tools for model quantization, covering low-bit networks, LLMs, multimodal and generative models, vector and lattice quantization, and efficient deployment.

    GitHub · 2,470 stars · 246 forks · 0 stars in 24 hours · Trending score 0.86

  5. 5
    LAMDA-Model-Reuse/ORBIT

    ORBIT: A Unified Toolbox for LLM Routing. Unified LLM/model routing, cost-aware inference, multimodal benchmarks, and reproducible evaluation. Accepted by Frontiers of Computer Science (2026).

    GitHub · Python · 19 stars · 1 forks · +2 stars in 24 hours · Trending score 0.72

  6. 6
    ranjithrajv/awesome-cpu-first-ai

    Curated, evidence-backed list of runtimes, formats & tools for running AI inference on CPU — start with CPU, justify the GPU.

    GitHub · Python · 5 stars · 4 forks · 0 stars in 24 hours · Trending score 0.07

  7. 7
    Picovoice/picollm

    On-device LLM Inference Powered by X-Bit Quantization

    GitHub · Python · 319 stars · 26 forks · 0 stars in 24 hours · Trending score 0.00

  8. 8
    vdev-ctrl/Adaptive-Learnable-Hierarchical-Routing-

    This repo is a alternative solution for standard self-attention. It uses tree routing with learnable functions for cheaper self-attention while not compromising accuracy.

    GitHub · Python · 43 stars · 4 forks · 0 stars in 24 hours · Trending score 0.00

  9. 9
    sejong-rcv/ST-Pruner

    Official implementation of "Don't Ask the LLM: Visual Token Pruning with Spectral Structure and Semantic Relevance" (WACV 2027)

    GitHub · Python · 23 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00

  10. 10
    KooroshRH/AReA

    Official implementation of "AReA: Adaptive Receptive-Field Attention for Efficient High-Resolution Diffusion Transformers"

    GitHub · Python · 153 stars · 5 forks · 0 stars in 24 hours · Trending score 0.00

  11. 11
    yxwang1215/Glance-ETD

    Glance, then Select: Evidence-Tempered Visual Token Selection for Extremely Compressed Multimodal LLMs

    GitHub · Python · 9 stars · 0 forks · Trending score 0.09