NLA — do humans and an NLA understand the same text the same way?

Code, data and results behind a small study of a Natural Language Autoencoder (NLA): a model pair that verbalizes one activation of a language model into an explanation z (the AV) and maps the explanation back to a reconstructed activation (the AR). Reference paper: https://transformer-circuits.pub/2026/nla/index.html. Inference only, no training.

The main experiment takes the community NLA for Qwen/Qwen3.6-27B (layer 42, ceselder/qwen3.6-27b-nla-rl, adapter step 300), 256 text samples from FineFineWeb, and asks what happens to the reconstruction when the explanation is transformed: paraphrased, shuffled, translated, flipped (every phrase negated), replaced by an unrelated explanation, or edited only around the final input token. Two numbers per transformed explanation: the activation FVE drop (how much worse the reconstruction of the activation gets) and the output KL increase (how much the target model's next-token distribution moves when the reconstruction is patched back in).

Where to look

Names used in the write-up and in the code

The write-up's two alignment curves, P(N=0 | H=1) and P(N=1 | H=0), threshold the FVE drop and are computed in experiments/002_writeup.py. The analysis stage also writes alignment.csv, which thresholds the distance between the two reconstructions instead (‖R(z) − R(z′)‖² / V_h, the definition in the metrics note); both readings give the same conclusion. A vocabulary-swap transformation was also run (vocab); it is in the results and the notebook but not in the write-up.

How it was run

One H200. The pipeline is experiments/002_nla_metrics.py, staged and resumable (--stage sanity|extract|verbalize|edit|reconstruct|output|nli|analyze), one 27B-class model resident at a time:

1. extract — sample contexts from FineFineWeb (four domains, each cut at a random whole word 50–256 tokens in) and read the layer-42 activation at the last token, plus the target's own next-token distribution.
2. verbalize — the AV writes one explanation per activation (T = 1), plus resamples.
3. edit — the hand-written transformations are loaded from hand_edits.jsonl; the code builds the shuffle, unrelated, "cat" and unrelated-with-token variants.
4. reconstruct / output — the AR reconstructs every variant; each reconstruction is patched into the target at the last token for the output KL.
5. nli — a DeBERTa NLI judge scores every transformed explanation against the original and against the input text.
6. analyze — tables, curves and plots into the run directory.

The hand-written transformations were authored by coding agents in parallel, one part of ~24 explanations each, against the rules in experiments/guides/HAND_EDITS.md, and validated by experiments/002_hand_edits.py check (negation per phrase, vocabulary unchanged, quotes kept verbatim, and so on).

Reproduce

cp .env.example .env                       # then adjust paths and caches
uv sync --all-groups                       # venv (Python 3.11, CUDA torch, transformers, peft, fla)
./bin/run pytest                           # unit tests on tiny random models (CPU)
./bin/run python experiments/002_nla_metrics.py --stage sanity                       # the author's 64 shipped activations -> FVE
bin/launch_bg.sh experiments/002_nla_metrics.py --stage extract,verbalize --n 256    # detached; log in $LOG_DIR
# the edit stage writes hand_edits_template.jsonl and stops; author hand_edits.jsonl
# (or copy configs/hand_edits_exp002_r3.jsonl into the run directory), then:
bin/launch_bg.sh experiments/002_nla_metrics.py --stage edit,reconstruct,output,nli,analyze --tag r3
./bin/run python experiments/002_walkthrough.py --run exp_002_r3     # the companion page
./bin/run python experiments/002_writeup.py --run exp_002_r3         # the write-up page

Model weights (~140 GB) are downloaded from the Hugging Face Hub on demand. Everything runs through ./bin/run, which loads .env.

Layout

instructions/           # the metric definitions and the briefs this work follows
experiments/            # NNN_*.py drivers, PLANS.md (design) and NOTEBOOKS.md (round log), guides/, results/
configs/                # the hand-written transformations of each round
src/nla/                # library: HF model layer, editor, NLI, metrics, analysis
tests/                  # pytest (CPU, tiny random models)
bin/                    # run (env wrapper), launch_bg.sh (detached local run)
CLAUDE.md               # working rules for coding agents
REPOSTART.md            # repo conventions (uv, env vars)
gcp/                    # Cloud TPU scripts of the earlier experiment (unused since EXP002)

An earlier experiment (EXP001) ran the same metrics on the 7B pair kitft/nla-qwen2.5-7b-L20-{av,ar} on a Cloud TPU with per-claim contradictions and deletions; its results are in experiments/results/exp_001{,_gold}/ and its page is experiments/results/exp_001_walkthrough.html.