Spinning Up in Mechanistic Interpretability

From vision circuits to the global workspace — a hands-on, controls-first curriculum for reading the inside of neural networks, and getting to where the 2025–2026 papers begin.

---

In 2020, a group at OpenAI opened up an image classifier and found curve detectors — individual neurons that fire for curves, wired together into legible circuits. They argued that neural networks are not inscrutable: they are built from features connected by weights, and you can reverse-engineer them. In 2021 that program moved to transformers, and over the next five years produced a chain of ideas — the residual stream, superposition, induction heads, sparse autoencoders, attribution graphs — that leads, in 2025–2026, to claims that language models contain an introspectable subspace, a global workspace, and a society of thought.

Those recent papers are exciting and easy to misread. They are also mostly run on models no outsider can open. This curriculum walks the chain from the beginning, on open-weight models you can run yourself, and it teaches one discipline the source papers often skip: every technique is paired with the control that could kill it. By the last rung you can not only read the 2025–2026 papers — you can reproduce their methods on open weights and check whether the claims survive a null.

Modeled on OpenAI's Spinning Up in Deep RL and the spinning-up-in-* family. Companion to the Consciousness-Indicator Scorecard — the last rung is the first cells of that benchmark.

How this fits with other resources

There is good mech-interp material already. This one exists for three reasons none of the others combine.

¹ ARENA §1.3.3 identifies the problem precisely — that autointerp on randomized SAE latents scores better than expected, because top-k over a large dataset finds patterns in anything — but leaves it as an optional bonus bullet with no starter code or solution. It is the best statement of the issue in any existing course, and it is an aside. See rung 5.

The gap we fill: existing courses teach the techniques; the source threads are the recent papers but run them on closed models with few controls. Nobody teaches the chain as a controls-first, open-weights on-ramp whose explicit endpoint is the ability to adjudicate the 2025–2026 claims yourself. That endpoint is the Scorecard program; this repo is how you learn to read its cells.

Who is this for?

• Anyone who wants to understand the recent interpretability papers (global workspace, introspection, societies of thought) rather than take them on faith.
• Students and researchers entering mechanistic interpretability who want to build, not just read.
• People who have seen a striking interpretability result and want to learn the reflex of asking "what is the control?"

Prerequisites: Python, PyTorch or JAX basics, and comfort with linear algebra and softmax. No prior interpretability experience. No GPU required — every runnable rung was verified end-to-end on a laptop CPU in under 25 seconds each.

Start here

pip install -r requirements.txt && ./verify_all.sh

---

The curriculum

Eight rungs. Every rung has the same shape: read the paper → run the technique on an open model → run the control that could falsify it. Each rungN_*/ directory is a self-contained project with its own README (the reading, the build, the null).

Part 0 — Build the model first (prerequisite)

Part I — Where the paradigm came from (vision)

Part II — Transformers, from the ground up

Part III — Reading and tracing computation

Part IV — The recent claims, adjudicated (capstone)

How each rung is built

Every rungN_*/README.md follows the same structure:

1. The idea — the concept in a paragraph, and why it is a rung on this ladder.
2. Read — the primary paper (and the one prerequisite reading), with links.
3. Build — the software project: a specific technique to implement on a named open model.
4. The control — the null, baseline, or ablation that could falsify what you just built, and what passing/failing it means. This is the point of the rung.
5. Toward the recent papers — the sentence of the 2025–26 papers this rung unlocks.

Rungs 1–5 and 7 ship a tested, self-contained starter.py you can run today — each finds a real result on an open model and then runs its own null, using only torch / transformers / torchvision (canonical tools like TransformerLens, SAELens, lucent and circuit-tracer are named as scale-ups, not required). Rung 3 needs only a laptop CPU; rungs 1, 2, 4, 5, 7 download a small open model (GPT-2 or InceptionV1) once. Rungs 6 and 8 draw directly on our working research code (jacobian-lens, jlens-lab, lenses on the Hub).

What each runnable starter finds, and the null that guards it:

Resources

• Build-it-from-scratch (rung 0): Karpathy, nanoGPT + Zero to Hero · Raschka, Build an LLM from Scratch and Build a Reasoning Model from Scratch · Mike X Cohen, 50 ML projects to understand LLMs (investigate transformer internals as data)
• Post-training, hands-on: TRL v1.0 (SFT + reward modelling + DPO + GRPO in one stack) and the smol-course. The three layers fit together: Lambert's course is the theory of post-training, TRL is the implementation, and this repo is the measurement — what those stages do to a model's representations, with a control attached.
• Post-training (the training side of rung 8): Nathan Lambert, RLHF Book + course — free, 11 lectures, and Conversation 2 is a case study in DPO for OLMo 3, the model rung 8 measures. Lectures and theory, no exercises; complementary to this repo's runnable projects.
• Foundational threads: Distill Circuits · Transformer Circuits Thread
• Tooling: TransformerLens · SAELens · Gemma Scope · circuit-tracer · Neuronpedia
• Our companion work: jacobian-lens · jlens-lab · the Consciousness-Indicator Scorecard
• Further: Nanda, 200 Concrete Open Problems in Mechanistic Interpretability

License

Apache-2.0. Course text CC BY 4.0. An Orthogonal Research and Education Lab (OREL) project.

Related

Part of one program — a controls-first, open-weights attempt to make the 2025–26 interpretability claims checkable:

Datasets: induction-emergence-pythia · inceptionv1-tuning-atlas · trilens-instrument-agreement · olmo3-jacobian-lenses