mechanistic-interpretability repositories

Trending repositories tagged mechanistic-interpretability, ranked by trending score. Updated daily.

  1. 1
    lyramakesmusic/neuron-explorer

    Steering-first neuron explorer for Qwen3.5-4B: push every MLP neuron, keep every rollout, browse the results

    GitHub · Python · 2,030 stars · 253 forks · +29 stars in 24 hours · Trending score 2.65

  2. 2
    chrishayuk/larql

    Run, serve, query and edit AI models — a Rust model runtime built on VINDEX3, an executable, inspectable model format.

    GitHub · Rust · 1,193 stars · 216 forks · +10 stars in 24 hours · Trending score 1.85

  3. 3
    ShutongLiang/Neural-State-OS

    Calibrated internal-state modulation, readout, and neural observability for local dense language models.

    GitHub · Python · 215 stars · 0 forks · +43 stars in 24 hours · Trending score 1.79

  4. 4
    stdlogits/neuron-lens

    A catalog of every MLP neuron and attention head of Qwen3-8B and Gemma 4 12B, read from the weights through two Jacobian lenses (R and S), from activations, and from their exact token drivers

    GitHub · Python · 57 stars · 5 forks · +4 stars in 24 hours · Trending score 1.39

  5. 5
    google-deepmind/detectorscope

    Build AI text detectors whose decisions are mechanistically grounded and human-auditable (EMNLP 2026)

    GitHub · Python · 496 stars · 44 forks · +2 stars in 24 hours · Trending score 1.38

  6. 6
    stdlogits/neuron-reflection

    Swap, fuse, remove or split a concept in Qwen3-8B's weights, without training, by mirroring the neurons and attention heads that carry it

    GitHub · Python · 16 stars · 2 forks · +11 stars in 24 hours · Trending score 1.30

  7. 7
    whp199/theodolite

    An atlas of all 1,290,240 neurons in Gemma-4-31B-it, a census of what they do when pushed, and the instrument to read them with.

    GitHub · Python · 39 stars · 5 forks · +2 stars in 24 hours · Trending score 1.27

  8. 8
    canivel/dynolab

    Dyno Lab — an AI safety, alignment and interpretability research workbench for Apple Silicon. Explore activations, probes, SAEs and interventions with local inference, a Python SDK, APIs and MCP.

    GitHub · Python · 220 stars · 162 forks · 0 stars in 24 hours · Trending score 1.24

  9. 9
    stdlogits/neuron-lens.com

    neuron-lens.com: what each neuron and attention head inside Qwen3-8B and Gemma 4 12B responds to, and what it makes the model say

    GitHub · Svelte · 26 stars · 1 forks · 0 stars in 24 hours · Trending score 0.67

  10. 10
    Miyamoto-Renya/LaplacesScissors

    Laplace's Scissors: protected readout rotations for open LLMs. Diagnose weak output directions, repair answers, preserve successful generations, and analyze scaling deviations with Rust and Triton on CUDA/HIP.

    GitHub · Rust · 33 stars · 8 forks · +1 stars in 24 hours · Trending score 0.61

  11. 11
    chessformer-lens/chessformer_lens

    A toolkit+visualizer that is designed for mech interp researchers working with chess models that use transformers tokenized by square.

    GitHub · Python · 10 stars · 0 forks · +1 stars in 24 hours · Trending score 0.55

  12. 12
    OpenBMB/ParamMute

    [NeurIPS 2025] ParamMute (a.k.a. PIP-KAG): suppressing knowledge-critical FFNs for faithful retrieval-augmented generation (RAG knowledge conflicts)

    GitHub · Python · 64 stars · 5 forks · +1 stars in 24 hours · Trending score 0.55

  13. 13
    oumi-ai/weights-lab

    Agent skill for weight-space programming: edit LLM weights

    GitHub · Python · 8 stars · 0 forks · +1 stars in 24 hours · Trending score 0.47

  14. 14
    johnmarktaylor91/torchlens

    Inspect, visualize and intervene on any operation in any neural network. Capture every activation and gradient, the full computational graph and 800+ metadata fields without changing model code. Supports PyTorch, JAX, TensorFlow, MLX, PaddlePaddle and tinygrad. Validated on 6400+ models. Built for researchers and their AI agents.

    GitHub · Python · 714 stars · 35 forks · 0 stars in 24 hours · Trending score 0.44

  15. 15
    EleutherAI/bergson

    Mapping out the "memory" of neural nets with data attribution

    GitHub · Python · 110 stars · 35 forks · 0 stars in 24 hours · Trending score 0.42

  16. 16
    moudrkat/brainscope

    OpenAI-compatible server with a live view into any HF model's residual stream. pip install brainscope

    GitHub · Python · 54 stars · 11 forks · 0 stars in 24 hours · Trending score 0.29

  17. 17
    moudrkat/steeropathy

    Agents that talk through model internals — activations & J-space — instead of text. No words pass between them. The lab on top of brainscope + hidden-directions.

    GitHub · Python · 29 stars · 3 forks · 0 stars in 24 hours · Trending score 0.28

  18. 18
    ndif-team/nnterp

    Unified access to Large Language Model modules using NNsight

    GitHub · HTML · 123 stars · 12 forks · 0 stars in 24 hours · Trending score 0.27

  19. 19
    AuremHQ/weight-space-programming

    wsp-kit by Aurem — an engineering tool for small models. Try to fix small open-weight models by editing weights directly, no retraining. An eval gate decides.

    GitHub · Python · 18 stars · 2 forks · 0 stars in 24 hours · Trending score 0.27

  20. 20
    EigenformAI/modulo

    Modulo lets agents running local models try modifying their own activations to change their inner state. Changes decay after a certain time and can be undone at any time.

    GitHub · Python · 41 stars · 0 forks · 0 stars in 24 hours · Trending score 0.20

  21. 21
    ctrimm/weight-space-programming

    Can an LLM write a neural network that IS a benefits formula, exactly? Weight-space programming experiments on SNAP, EITC, SSI, and Medicaid arithmetic.

    GitHub · Python · 6 stars · 2 forks · 0 stars in 24 hours · Trending score 0.15

  22. 22
    kevinpl07/weight-patching

    One neuron, no training: what editing an LLM's weights is actually good for

    GitHub · Python · 9 stars · 2 forks · 0 stars in 24 hours · Trending score 0.14

  23. 23
    BrandeisPatrick/openbench

    Reverse-engineer the RL reward an LLM was trained on — from black-box agentic coding behavior alone. Tested on Opus 4.8, Fable 5, GPT-5.5.

    GitHub · Python · 9 stars · 0 forks · 0 stars in 24 hours · Trending score 0.09

  24. 24
    zjunlp/Mechanist

    AI Systems as Scientific Instruments for Understanding the Mechanisms of Intelligence

    GitHub · Python · 82 stars · 8 forks · 0 stars in 24 hours · Trending score 0.08