mechanistic-interpretability repositories
Trending repositories tagged mechanistic-interpretability, ranked by trending score. Updated daily.
-
1
lyramakesmusic/neuron-explorer
Steering-first neuron explorer for Qwen3.5-4B: push every MLP neuron, keep every rollout, browse the results
GitHub · Python · 2,030 stars · 253 forks · +29 stars in 24 hours · Trending score 2.65
-
2
chrishayuk/larql
Run, serve, query and edit AI models — a Rust model runtime built on VINDEX3, an executable, inspectable model format.
GitHub · Rust · 1,193 stars · 216 forks · +10 stars in 24 hours · Trending score 1.85
-
3
ShutongLiang/Neural-State-OS
Calibrated internal-state modulation, readout, and neural observability for local dense language models.
GitHub · Python · 215 stars · 0 forks · +43 stars in 24 hours · Trending score 1.79
-
4
stdlogits/neuron-lens
A catalog of every MLP neuron and attention head of Qwen3-8B and Gemma 4 12B, read from the weights through two Jacobian lenses (R and S), from activations, and from their exact token drivers
GitHub · Python · 57 stars · 5 forks · +4 stars in 24 hours · Trending score 1.39
-
5
google-deepmind/detectorscope
Build AI text detectors whose decisions are mechanistically grounded and human-auditable (EMNLP 2026)
GitHub · Python · 496 stars · 44 forks · +2 stars in 24 hours · Trending score 1.38
-
6
stdlogits/neuron-reflection
Swap, fuse, remove or split a concept in Qwen3-8B's weights, without training, by mirroring the neurons and attention heads that carry it
GitHub · Python · 16 stars · 2 forks · +11 stars in 24 hours · Trending score 1.30
-
7
whp199/theodolite
An atlas of all 1,290,240 neurons in Gemma-4-31B-it, a census of what they do when pushed, and the instrument to read them with.
GitHub · Python · 39 stars · 5 forks · +2 stars in 24 hours · Trending score 1.27
-
8
canivel/dynolab
Dyno Lab — an AI safety, alignment and interpretability research workbench for Apple Silicon. Explore activations, probes, SAEs and interventions with local inference, a Python SDK, APIs and MCP.
GitHub · Python · 220 stars · 162 forks · 0 stars in 24 hours · Trending score 1.24
-
9
stdlogits/neuron-lens.com
neuron-lens.com: what each neuron and attention head inside Qwen3-8B and Gemma 4 12B responds to, and what it makes the model say
GitHub · Svelte · 26 stars · 1 forks · 0 stars in 24 hours · Trending score 0.67
-
10
Miyamoto-Renya/LaplacesScissors
Laplace's Scissors: protected readout rotations for open LLMs. Diagnose weak output directions, repair answers, preserve successful generations, and analyze scaling deviations with Rust and Triton on CUDA/HIP.
GitHub · Rust · 33 stars · 8 forks · +1 stars in 24 hours · Trending score 0.61
-
11
chessformer-lens/chessformer_lens
A toolkit+visualizer that is designed for mech interp researchers working with chess models that use transformers tokenized by square.
GitHub · Python · 10 stars · 0 forks · +1 stars in 24 hours · Trending score 0.55
-
12
OpenBMB/ParamMute
[NeurIPS 2025] ParamMute (a.k.a. PIP-KAG): suppressing knowledge-critical FFNs for faithful retrieval-augmented generation (RAG knowledge conflicts)
GitHub · Python · 64 stars · 5 forks · +1 stars in 24 hours · Trending score 0.55
-
13
oumi-ai/weights-lab
Agent skill for weight-space programming: edit LLM weights
GitHub · Python · 8 stars · 0 forks · +1 stars in 24 hours · Trending score 0.47
-
14
johnmarktaylor91/torchlens
Inspect, visualize and intervene on any operation in any neural network. Capture every activation and gradient, the full computational graph and 800+ metadata fields without changing model code. Supports PyTorch, JAX, TensorFlow, MLX, PaddlePaddle and tinygrad. Validated on 6400+ models. Built for researchers and their AI agents.
GitHub · Python · 714 stars · 35 forks · 0 stars in 24 hours · Trending score 0.44
-
15
EleutherAI/bergson
Mapping out the "memory" of neural nets with data attribution
GitHub · Python · 110 stars · 35 forks · 0 stars in 24 hours · Trending score 0.42
-
16
moudrkat/brainscope
OpenAI-compatible server with a live view into any HF model's residual stream. pip install brainscope
GitHub · Python · 54 stars · 11 forks · 0 stars in 24 hours · Trending score 0.29
-
17
moudrkat/steeropathy
Agents that talk through model internals — activations & J-space — instead of text. No words pass between them. The lab on top of brainscope + hidden-directions.
GitHub · Python · 29 stars · 3 forks · 0 stars in 24 hours · Trending score 0.28
-
18
ndif-team/nnterp
Unified access to Large Language Model modules using NNsight
GitHub · HTML · 123 stars · 12 forks · 0 stars in 24 hours · Trending score 0.27
-
19
AuremHQ/weight-space-programming
wsp-kit by Aurem — an engineering tool for small models. Try to fix small open-weight models by editing weights directly, no retraining. An eval gate decides.
GitHub · Python · 18 stars · 2 forks · 0 stars in 24 hours · Trending score 0.27
-
20
EigenformAI/modulo
Modulo lets agents running local models try modifying their own activations to change their inner state. Changes decay after a certain time and can be undone at any time.
GitHub · Python · 41 stars · 0 forks · 0 stars in 24 hours · Trending score 0.20
-
21
ctrimm/weight-space-programming
Can an LLM write a neural network that IS a benefits formula, exactly? Weight-space programming experiments on SNAP, EITC, SSI, and Medicaid arithmetic.
GitHub · Python · 6 stars · 2 forks · 0 stars in 24 hours · Trending score 0.15
-
22
kevinpl07/weight-patching
One neuron, no training: what editing an LLM's weights is actually good for
GitHub · Python · 9 stars · 2 forks · 0 stars in 24 hours · Trending score 0.14
-
23
BrandeisPatrick/openbench
Reverse-engineer the RL reward an LLM was trained on — from black-box agentic coding behavior alone. Tested on Opus 4.8, Fable 5, GPT-5.5.
GitHub · Python · 9 stars · 0 forks · 0 stars in 24 hours · Trending score 0.09
-
24
zjunlp/Mechanist
AI Systems as Scientific Instruments for Understanding the Mechanisms of Intelligence
GitHub · Python · 82 stars · 8 forks · 0 stars in 24 hours · Trending score 0.08