Circuit Reuse

Code for How Much Do Circuits Tell Us? Measuring the Consistency and Specificity of Language Model Circuits.

For each example in a task dataset, we extract a circuit, defined as the top-K% of components by attribution score. We then define the task's shared circuit, $S_P$, as the set of components that appear in at least P% of these per-example circuits. We evaluate $S_P$ on two criteria.

Consistency is a measure of how well one circuit describes the whole task. Reuse@P quantifies it as the fraction of each example's circuit that is contained in $S_P$, averaged over examples. To check that $S_P$ matters causally, we zero-ablate it and measure the accuracy drop relative to ablating a random set of components with the same size and the same head/MLP composition.

Specificity is a measure of how much the circuit belongs to its task rather than to the model in general. We ablate task A's shared circuit and measure the accuracy drop on task A, then ablate every other task's shared circuit and measure the drop on task A again. A specific circuit hurts its own task more than the others' circuits do.

We run this at two granularities, attention heads and MLP blocks (head_mlp) or individual MLP neurons (neuron), with three attribution methods (EAP, EAP-IG, RelP), on six tasks and five models: Gemma 2 2B and its instruction-tuned variant, Llama 3.2 3B and its Instruct variant, and Qwen3 4B.

Setup

python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt -e .
huggingface-cli login   # gemma-2 and llama-3.2 are gated

Layout

Tasks

Each run generates or reads --num_examples examples, shuffles them with --seed, and holds out --val-fraction of them for validation. google/gemma-2-2b-it receives a few-shot prefix on addition.

Pipeline

1. Extraction

python main_experiment.py \
  --model_name google/gemma-2-2b --task ioi --num_examples 1000 --digits 3 \
  --method eap_ig --granularity head_mlp --ig-steps 5 \
  --top_k_list 1,5,10,20,30 --reuse-thresholds 50,75,85,90,95,96,97,98,99,100 \
  --perm-trials 5000 --dtype bf16 --amp --device cuda \
  --run-name granularity_parity_eap_ig_head_mlp \
  --output-dir results/granularity_parity/granularity_parity_extraction \
  --cache-dir cache_granularity_parity/eap_ig_head_mlp

The output is <output-dir>/<run-name>/<model>__main__<task>__<method>__.../metrics.json, with the following structure:

baseline_{train,val}_accuracy
by_k[K].thresholds[P]: shared_components, shared_circuit_size, reuse_percent,
                       train/val: {ablation_accuracy, control_accuracy, permutation}

Per-example attribution scores are cached as JSONL files in --cache-dir, one line per example with every component's score. --analysis recomputes the metrics from the cache without loading a model; --force-extract ignores the cache and re-runs attribution.

2. Cross-task ablation

python cross_task_experiment.py \
  --results-dir results/granularity_parity/granularity_parity_extraction/granularity_parity_eap_ig_head_mlp \
  --model_name google/gemma-2-2b --method eap_ig --granularity head_mlp \
  --tasks addition,arc_challenge,arc_easy,boolean,ioi,mcqa \
  --K 1,5,10,20,30 --threshold 50,75,85,90,95,96,97,98,99,100 \
  --num-examples 100 --digits 3 --device cuda \
  --output-dir results/granularity_parity/granularity_parity_cross_task/eap_ig_head_mlp

The model is loaded once and every (K, P) matrix is evaluated. Each cross_task_<model>_<method>_<granularity>_K<K>_p<P>.json file holds cells[donor][target] with the baseline and ablated accuracies and the drop in percentage points, plus the baselines and circuit sizes. Matrices that already exist are skipped, so the command can be rerun after an interruption. --refresh-tasks t1,t2 recomputes only the cells where the donor or the target is in the list and copies the other cells from the existing file.

3. Aggregation and figures

python -m analysis.granularity_parity --results-root results/granularity_parity \
    --output-dir results/granularity_parity_analysis --plots

This reads every metrics.json and cross-task JSON under --results-root and writes the tidy CSVs (extraction_tidy.csv, cross_task_tidy.csv, overlap_pairs.csv, bootstrap_summary.csv, circuit_composition.csv) and summary figures. The scripts below read those CSVs:

The older scripts in analysis/ (plot_k_sweep.py, multiplot_*.py, ...) predate the tidy CSVs and read metrics.json files directly.

main_experiment.py arguments

Attribution methods

Top-K selection ranks components by the stored score. Where the score is signed, the circuit therefore consists of the most positive components.

References & acknowledgements

This project builds on:

• TransformerLens — hook-based mechanistic-interpretability library; all forward passes and component hooks are TL primitives.
• eap-ig — our eap and eap_ig edge-graph paths (circuit_reuse/graph.py) are derived from this implementation of Edge Attribution Patching (Syed et al. 2023, arXiv:2310.10348).
• RelP (Jafari et al. 2025) — the relp method and the LRP rules in circuit_reuse/lrp_patch.py are ported from the authors' TransformerLens fork at FarnoushRJ/RelP (see reference_code/RelP/).
• ADAG / Arora et al. 2026 — "Language Model Circuits Are Sparse in the Neuron Basis." --method relp --granularity neuron reproduces their MLP-neuron-basis circuit scoring. The reference_code/circuits/ directory vendors the ADAG library (Transluce) for comparison.
• LRP propagation rules: LN-rule (Ali et al. 2022), AH-rule (Ali et al. 2022), Half-rule (Arras et al. 2019; Jafari et al. 2024).