attention-mechanism repositories

Trending repositories tagged attention-mechanism, ranked by trending score. Updated daily.

  1. 1
    rasbt/LLMs-from-scratch

    Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

    GitHub · Jupyter Notebook · 108,771 stars · 16,816 forks · +106 stars in 24 hours · Trending score 3.96

  2. 2
    ruvnet/RuVector

    RuVector provides High Performance, Real-Time decisions and agent memory , Self-Learning Ai, Vector GNN DB built in Rust.

    GitHub · Rust · 4,713 stars · 634 forks · +15 stars in 24 hours · Trending score 2.20

  3. 3
    amitshekhariitbhu/llm-internals

    Learn LLM internals step by step - from tokenization to attention to inference optimization.

    GitHub · 1,774 stars · 202 forks · +4 stars in 24 hours · Trending score 1.13

  4. 4
    lavishguptaa/llm-architecture-atlas

    An open-source collection covering almost all major LLM architectures, built to explore, understand, and implement the building blocks of modern language models.

    GitHub · Python · 11 stars · 1 forks · +1 stars in 24 hours · Trending score 0.54

  5. 5
    skyliulu/LLM-Infra-Explorer

    Interactive visualization and exploration of large language model (LLM) systems, infrastructure, and inference workflows

    GitHub · JavaScript · 12 stars · 1 forks · 0 stars in 24 hours · Trending score 0.47

  6. 6
    billpsomas/efficient-probing

    [ICLR 2026] - Official implementation of "Attention, Please! Revisiting Attentive Probing Through the Lens of Efficiency"

    GitHub · Python · 43 stars · 0 forks · +1 stars in 24 hours · Trending score 0.45

  7. 7
    lucidrains/recurrent-transformer

    Explorations into the Recurrent Transformer proposed by Costin-Andrei Oncescu et al. of Harvard University

    GitHub · Python · 26 stars · 2 forks · 0 stars in 24 hours · Trending score 0.39

  8. 8
    alhqlearn/DeepMech

    DeepMech: A Machine Learning Framework for Chemical Reaction Mechanism Prediction

    GitHub · Python · 11 stars · 4 forks · 0 stars in 24 hours · Trending score 0.37

  9. 9
    lucidrains/x-transformers

    A concise but complete full-attention transformer with a set of promising experimental features from various papers

    GitHub · Python · 5,963 stars · 533 forks · 0 stars in 24 hours · Trending score 0.30

  10. 10
    raiyanyahya/how-to-train-your-gpt

    Build a modern LLM from scratch. Every line commented. Explained like we are five.

    GitHub · Jupyter Notebook · 4,142 stars · 503 forks · 0 stars in 24 hours · Trending score 0.00

  11. 11
    aryanap07/Timeless

    Before the prompt, there was the architecture.

    GitHub · Jupyter Notebook · 5 stars · 3 forks · 0 stars in 24 hours · Trending score 0.00

  12. 12
    zay002/SplAttN

    [ICML 2026 Spotlight] Official PyTorch implementation of “SplAttN: Bridging 2D and 3D with Gaussian Soft Splatting and Attention for Point Cloud Completion”.

    GitHub · Python · 16 stars · 1 forks · 0 stars in 24 hours · Trending score 0.00

  13. 13
    CASE-Lab-UMD/LLM-Drop

    The official implementation of the paper "Uncovering the Redundancy in Transformers via a Unified Study of Layer Dropping (TMLR)".

    GitHub · Python · 193 stars · 23 forks · 0 stars in 24 hours · Trending score 0.00

  14. 14
    LIU42/Surveillance

    面向监控视频的异常行为检测系统设计。

    GitHub · Vue · 15 stars · 2 forks · 0 stars in 24 hours · Trending score 0.00

  15. 15
    lucidrains/declarative-attention

    Implementation of the procedure in Language Models Can Control Their Own Attention, from Namgyu Ho et al. of KAIST AI

    GitHub · Python · 24 stars · 2 forks · 0 stars in 24 hours · Trending score 0.00

  16. 16
    lucidrains/learn-attention

    Sharing how attention appears in my mind's eye, for students and those interested in understanding transformers

    GitHub · HTML · 63 stars · 3 forks · 0 stars in 24 hours · Trending score 0.00

  17. 17
    FemtoRhythm/pyramis-l3d

    Pyramis-L3D: hierarchical sparse attention research prototype (code only; weights on Hugging Face)

    GitHub · Python · 26 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00

  18. 18
    lucidrains/stack-attention

    For following a line of research that augments attention with a differentiable stack, beginning with DuSell et al. at ETH Zurich

    GitHub · Python · 12 stars · 1 forks · 0 stars in 24 hours · Trending score 0.00

  19. 19
    YuvrajSingh-mist/Paper-Replications-and-Experiments

    From-scratch PyTorch replications of classic and SOTA AI/ML papers: transformers, attention, GANs, diffusion, CLIP, LoRA, MoE, plus SFT/GRPO post-training experiments.

    GitHub · Jupyter Notebook · 431 stars · 43 forks · 0 stars in 24 hours · Trending score 0.00

  20. 20
    vdev-ctrl/Adaptive-Learnable-Hierarchical-Routing-

    This repo is a alternative solution for standard self-attention. It uses tree routing with learnable functions for cheaper self-attention while not compromising accuracy.

    GitHub · Python · 44 stars · 4 forks · 0 stars in 24 hours · Trending score 0.00

  21. 21
    PaddlePaddle/VHA

    [NeurIPS2026] Official implementation of Virtual Head Attention (VHA) for efficient large language model architecture and training optimization.

    GitHub · Python · 12 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00

  22. 22
    lucidrains/attention-contact-planning

    Implementation of Attention-based Contact Planning, by Junzhe He of ETH Zürich

    GitHub · Python · 21 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00

  23. 23
    vita-epfl/social-transmotion

    [ICLR 2024] Official implementation of "Social-Transmotion: Promptable Human Trajectory Prediction" in PyTorch.

    GitHub · Python · 65 stars · 4 forks · 0 stars in 24 hours · Trending score 0.00

  24. 24
    Zguoxu/tiny-transformer-from-scratch

    A tiny Transformer built from scratch in PyTorch, with an English-to-Chinese translation demo | 用 PyTorch 从零实现的小型 Transformer,附英译中演示,帮助理解模型原理

    GitHub · Jupyter Notebook · 102 stars · 3 forks · 0 stars in 24 hours · Trending score 0.00