Tri-lens: do three interpretability instruments agree about the same activation?

Three 2025–26 instruments each claim to read what a model internally holds. Nobody had run several on the same activations of the same model and asked whether they agree. This does, with a mismatch null, and then makes it causal with steering as ground truth.

Model: google/gemma-3-27b-it, layer 41 — the only layer where a J-lens, an NLA checkpoint, Gemma Scope 2 SAE weights, and Neuronpedia feature labels all exist.

Phase 1 — agreement vs a cross-prompt mismatch null

The J-lens is a linear readout (averaged forward Jacobian through the unembedding); the NLA is a trained, RL-optimised verbaliser. They share no machinery and were built by different efforts — yet they recover the same content, 42× above the null. That is convergent validity for "verbalizable content" as a property of the representation, not of either instrument.

The SAE rows are inconclusive, not negative. Neuronpedia publishes 2,048 labels for a 16,384-feature SAE (12.5% coverage), so unlabelled features contribute empty strings. Those numbers measure label coverage.

Phase 2 — SAE steering as ground truth (causal)

Inject a known SAE feature and both instruments report it; inject nothing, a random direction, or the negation and neither does. This is what agreement alone cannot establish.

Also here

• RECON_NULL_FINDING.md — a retraction. We first reported that NLA round-trip fidelity "does not survive a mismatch null." It was two bugs of ours: sampling token positions below the AV's min_position=50 training regime, and applying a √d scaling correction in the wrong direction. Verified against the authors' published worked example, the pipeline scores fve_nrm 0.839 vs their 0.821. A null-only design cannot distinguish "the method fails" from "my reproduction is broken."
• MOE_SPARSITY_THRESHOLD.md — revised twice. An apparent 4–6× sparsity penalty on the J-lens turned out to be mostly model family: dense models alone span a 3.3× dispersion range, and a within-family control (Qwen3-30B-A3B 6.2% sparse, 1.90, vs dense Qwen3-32B, 2.11–3.45) reverses the conclusion.
• MOE_FEASIBILITY.md, PHASE1_RESULT.md, PHASE2_RESULT.md, and full run logs.

Limitations

Lexical-overlap scoring handicaps the J-lens (multilingual/emoji tokens vs English prose); one model, one layer; Phase 2 uses 6 features × 4 prompts × 2 positions; some Neuronpedia labels are grab-bags. Every figure above is a contrast against a control, which is the part that carries the evidence.

Morgan Hough, Orthogonal Research and Education Lab (OREL). Part of Controls & Trajectories · curriculum.