attention-mechanism repositories
Trending repositories tagged attention-mechanism, ranked by trending score. Updated daily.
-
1
rasbt/LLMs-from-scratch
Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
GitHub · Jupyter Notebook · 108,771 stars · 16,816 forks · +106 stars in 24 hours · Trending score 3.96
-
2
ruvnet/RuVector
RuVector provides High Performance, Real-Time decisions and agent memory , Self-Learning Ai, Vector GNN DB built in Rust.
GitHub · Rust · 4,713 stars · 634 forks · +15 stars in 24 hours · Trending score 2.20
-
3
amitshekhariitbhu/llm-internals
Learn LLM internals step by step - from tokenization to attention to inference optimization.
GitHub · 1,774 stars · 202 forks · +4 stars in 24 hours · Trending score 1.13
-
4
lavishguptaa/llm-architecture-atlas
An open-source collection covering almost all major LLM architectures, built to explore, understand, and implement the building blocks of modern language models.
GitHub · Python · 11 stars · 1 forks · +1 stars in 24 hours · Trending score 0.54
-
5
skyliulu/LLM-Infra-Explorer
Interactive visualization and exploration of large language model (LLM) systems, infrastructure, and inference workflows
GitHub · JavaScript · 12 stars · 1 forks · 0 stars in 24 hours · Trending score 0.47
-
6
billpsomas/efficient-probing
[ICLR 2026] - Official implementation of "Attention, Please! Revisiting Attentive Probing Through the Lens of Efficiency"
GitHub · Python · 43 stars · 0 forks · +1 stars in 24 hours · Trending score 0.45
-
7
lucidrains/recurrent-transformer
Explorations into the Recurrent Transformer proposed by Costin-Andrei Oncescu et al. of Harvard University
GitHub · Python · 26 stars · 2 forks · 0 stars in 24 hours · Trending score 0.39
-
8
alhqlearn/DeepMech
DeepMech: A Machine Learning Framework for Chemical Reaction Mechanism Prediction
GitHub · Python · 11 stars · 4 forks · 0 stars in 24 hours · Trending score 0.37
-
9
lucidrains/x-transformers
A concise but complete full-attention transformer with a set of promising experimental features from various papers
GitHub · Python · 5,963 stars · 533 forks · 0 stars in 24 hours · Trending score 0.30
-
10
raiyanyahya/how-to-train-your-gpt
Build a modern LLM from scratch. Every line commented. Explained like we are five.
GitHub · Jupyter Notebook · 4,142 stars · 503 forks · 0 stars in 24 hours · Trending score 0.00
-
11
aryanap07/Timeless
Before the prompt, there was the architecture.
GitHub · Jupyter Notebook · 5 stars · 3 forks · 0 stars in 24 hours · Trending score 0.00
-
12
zay002/SplAttN
[ICML 2026 Spotlight] Official PyTorch implementation of “SplAttN: Bridging 2D and 3D with Gaussian Soft Splatting and Attention for Point Cloud Completion”.
GitHub · Python · 16 stars · 1 forks · 0 stars in 24 hours · Trending score 0.00
-
13
CASE-Lab-UMD/LLM-Drop
The official implementation of the paper "Uncovering the Redundancy in Transformers via a Unified Study of Layer Dropping (TMLR)".
GitHub · Python · 193 stars · 23 forks · 0 stars in 24 hours · Trending score 0.00
-
14
LIU42/Surveillance
面向监控视频的异常行为检测系统设计。
GitHub · Vue · 15 stars · 2 forks · 0 stars in 24 hours · Trending score 0.00
-
15
lucidrains/declarative-attention
Implementation of the procedure in Language Models Can Control Their Own Attention, from Namgyu Ho et al. of KAIST AI
GitHub · Python · 24 stars · 2 forks · 0 stars in 24 hours · Trending score 0.00
-
16
lucidrains/learn-attention
Sharing how attention appears in my mind's eye, for students and those interested in understanding transformers
GitHub · HTML · 63 stars · 3 forks · 0 stars in 24 hours · Trending score 0.00
-
17
FemtoRhythm/pyramis-l3d
Pyramis-L3D: hierarchical sparse attention research prototype (code only; weights on Hugging Face)
GitHub · Python · 26 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00
-
18
lucidrains/stack-attention
For following a line of research that augments attention with a differentiable stack, beginning with DuSell et al. at ETH Zurich
GitHub · Python · 12 stars · 1 forks · 0 stars in 24 hours · Trending score 0.00
-
19
YuvrajSingh-mist/Paper-Replications-and-Experiments
From-scratch PyTorch replications of classic and SOTA AI/ML papers: transformers, attention, GANs, diffusion, CLIP, LoRA, MoE, plus SFT/GRPO post-training experiments.
GitHub · Jupyter Notebook · 431 stars · 43 forks · 0 stars in 24 hours · Trending score 0.00
-
20
vdev-ctrl/Adaptive-Learnable-Hierarchical-Routing-
This repo is a alternative solution for standard self-attention. It uses tree routing with learnable functions for cheaper self-attention while not compromising accuracy.
GitHub · Python · 44 stars · 4 forks · 0 stars in 24 hours · Trending score 0.00
-
21
PaddlePaddle/VHA
[NeurIPS2026] Official implementation of Virtual Head Attention (VHA) for efficient large language model architecture and training optimization.
GitHub · Python · 12 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00
-
22
lucidrains/attention-contact-planning
Implementation of Attention-based Contact Planning, by Junzhe He of ETH Zürich
GitHub · Python · 21 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00
-
23
vita-epfl/social-transmotion
[ICLR 2024] Official implementation of "Social-Transmotion: Promptable Human Trajectory Prediction" in PyTorch.
GitHub · Python · 65 stars · 4 forks · 0 stars in 24 hours · Trending score 0.00
-
24
Zguoxu/tiny-transformer-from-scratch
A tiny Transformer built from scratch in PyTorch, with an English-to-Chinese translation demo | 用 PyTorch 从零实现的小型 Transformer,附英译中演示,帮助理解模型原理
GitHub · Jupyter Notebook · 102 stars · 3 forks · 0 stars in 24 hours · Trending score 0.00