Spyglass

A prompt-local linear readout for intermediate facts inside a decoder-only model, compared against released trained lenses, and two residual edits that test what a late state controls.

The idea. Given a single prompt whose answer depends on a fact it never states — "the language spoken in the country where the Amazon River ends" depends on Brazil — Spyglass fits a linear map from an intermediate layer's residuals to the final layer's, using only that prompt's own unlabeled prefix states, with the scored position held out. Decoding the held-out position through the fitted map surfaces the unstated intermediate at far better rank than the released lenses do, over the first three quarters of the network.

Layout

spyglass/          the package
scripts/           one experiment per file
data/surfaces/     frozen corpora and experiment registrations
data/results/      committed receipts
tests/             unit tests

• spyglass/ — transport (fit / apply / adjoint / inverse), readout (token scoring under the shared rank rule), items (the locked 40-item corpus and its wrong-intermediate controls), model (plain Hugging Face capture), baselines (released J-Lens / R-Lens), clusters (spherical k-means over the output embedding), causal (residual edits and patching).

Experiments

Models, revisions, and baseline-lens pins live in data/models.json, so no script takes them as arguments. Surfaces and output paths default to the committed ones; the only required flags are paths to things outside the repo (--jlens-code, and --eval-dir for the public suite).

Two receipts have no script here because they are inputs rather than outputs: readout_profile.json is the summary the head-to-head figures are drawn from, derived from compare.json and official_compare.json; logit_blockade.json records what banning a token at the logit level does, which is the baseline subtract_row.py is contrasted against.

Which claim rests on which receipt

The shared rank rule

Every readout experiment scores the same way: take the state at the scored position, transport it (identity for the raw carrier), decode it against the output vocabulary, and count the rows scoring strictly higher than the best spelling variant of the target word. Lower is better; pass@k counts best-over-layers ranks below k.

Reproducing

The readout, audit, sweep, patching, suite, and subtraction runs are Qwen3.6-27B; the steering run is Qwen3.8-27B. Loading the released lenses needs the camilablank/workspace-lenses repository's jlens package, passed as --jlens-code.

uv sync
uv run python -m pytest tests/ -q

uv run python scripts/causal_patch.py                       # no arguments
uv run python scripts/subtract_row.py
uv run python scripts/compare_lenses.py --jlens-code PATH

Provenance notes

• data/surfaces/locked40.json carries the item list and the selection rule unchanged from the frozen original, trimmed to the fields these experiments read. The original also registered a coded/open output protocol and suppression arms belonging to a separate line of work; those are not here, and the file says so.
• steering.py and subtract_row.py are cleaned reimplementations of the procedure that produced steering.json and subtraction.json. The logic, the doses, the arms, and the frozen cases are the same; the surrounding code is not the code that produced those files.
• steering.py recomputes its vocabulary chart from the checkpoint with a fixed seed rather than loading the precomputed one the receipt used, so class indices differ between runs. Nothing selects a class by index: the target class is the one containing the anchor token " Yes", and both the target and the random-control class have their token lists audited into the receipt so the selection can be checked. In the committed run the target class is yes / Yes / 确实 / oui / 是的 and the occupancy-matched control class is constants.