Structure is not mechanism

High-gain gated-FFN rows across text and genomic foundation models

Code, artifacts and preregistrations for the paper of this title. The manuscript itself is not in this repository; it ships with the publication.

The question. A handful of "super weights" in text LLMs can be catastrophically important. Do analogous structures recur in genomic foundation models — and, more sharply, does a high-gain row's structural prominence tell you anything about its causal importance?

The answer, across a preregistered 22-model causal census and focused mechanistic case studies, is that three quantities routinely conflated in this literature are distinct:

STRUCTURAL GEOMETRY  ≠  FUNCTIONAL CRITICALITY  ≠  CAUSAL RESPONSE COMPLEXITY

---

Start here

What the paper establishes

Installation

git clone https://github.com/Georgakopoulos-Soares-lab/superweights.git
cd superweights
pip install -r requirements.txt
export PYTHONPATH=.          # required: scripts import the shared library as `src.*`

All commands below assume PYTHONPATH=. and the repository root as the working directory.

Three environments are needed, because the model families pin incompatible transformers:

Running DNABERT-2 under the main environment fails with BertConfig has no attribute pad_token_id. That is the wrong-environment signature, not a bug.

Data

Other large public inputs are documented rather than vendored — see data/README.md.

Reproducing the paper

One row per object in the paper. Every artifact path below was verified to exist in this repository; where an input is gitignored and therefore absent from a clone, the row says so and names the command that regenerates it.

A note on filenames. Moving the within-layer section ahead of the DNABERT-2 section shifted the figure numbers after it by one, but the script filenames were not renamed — renaming them risks breaking the literal-path sys.path inserts this repository depends on (see tests/test_entrypoints.py). So fig3_dnabert.py renders Figure 4 and fig4_generator.py renders Figure 5. The offset is intentional and is flagged on each affected row.

Main figures

Supplementary figures

Supplementary tables

S2, S3 and S6–S8 are generated; S1, S4 and S5 are reformatted verbatim from the audit and are not rebuilt. One command produces all five generated tables.

python scripts/census_analysis/build_supplementary_tables.py

Results in the text, not in a figure

The authoritative, machine-checked mapping is docs/EXPERIMENT_MAP.md (21 experiments) and, panel by panel, experiments/figures/FIGURE_PROVENANCE.md. This table surfaces them; it does not replace them. python scripts/build_experiment_map.py re-verifies that every script and artifact named there still exists.

Running the pipelines

Prefer the pipelines. Each skips stages whose output already exists, so an interrupted run resumes and a finished one costs nothing; FORCE=1 recomputes.

bash pipelines/census.sh                 # the preregistered 22-model causal census (restartable)
bash pipelines/census.sh dnabert2 ntv3   # ...or a subset
bash pipelines/detector_provenance.sh    # selection-rule resolution; runs its positive control first
bash pipelines/fig5_within_layer.sh      # the within-layer sweeps and second critical row
                                         # (filename retains an earlier figure number)

Individual stages:

# structural panel and causal census
python experiments/E11_scale_ladder/run_model.py --model <slug>
python experiments/E13_full_cohort_causal_census/run_singleton_census.py --model <slug>

# mechanism case studies
python experiments/E9_mechanistic_tomography/run_fit_observers.py
python experiments/E12_generator_degradation_control/run_bos_mediation_main.py

# within-layer sweeps
python scripts/within_layer_sweep/run_within_model_slope_text.py     # SmolLM2-1.7B, ~7 min, 1 GPU
python scripts/within_layer_sweep/run_smollm2_row_geometry.py        # data-free, no GPU

# supplementary tables S2, S3, S6-S8 (rebuilt from committed artifacts)
python scripts/census_analysis/build_supplementary_tables.py

# two flags that are REQUIRED and silently wrong if omitted
python scripts/detection/run_detection.py --model ntv3 --pad_to_multiple 256   # stride-256 U-Net
python scripts/evaluation/run_gue_multiseed.py --model ntv3 --max_length 400   # nucleotide tokenizer

Verify the repository's own invariants:

python scripts/build_experiment_map.py    # every claim's code and artifacts exist
python src/prereg_lock.py verify --all    # 12 preregistrations, content-locked
pytest tests/                             # unit tests + every CLI entry point starts

Repository map

experiments/ contains the preregistered harnesses (E1–E13, each with the preregistration it was run under in docs/prereg/); scripts/ contains analyses developed after preregistration. The split is deliberate: it lets a reader see at a glance which analyses were locked in advance and which came later.

Provenance and known limitations

Stated plainly, because a reader will find them anyway.

1. Some raw artifacts are not committed. Several were produced on a cluster and never transferred. For every affected item the committed figure source-data snapshot under experiments/figures/source_data/ carries the plotted values, so each reported number remains inspectable.
2. NTv3's original weight revision was not recorded and is not recoverable. Resolved Hub commits exist for 21 of 22 models.
3. One n=10 subgroup is fragile. The text-decoder correlation moves from ρ=0.770 to 0.673 under an OLMo coordinate substitution, with a 95% lower bound of +0.032. It should not be read as independently robust; the 22-model result is insensitive to the substitution.
4. Experiment lines that the paper does not report are not in the working tree. Their preregistrations are retained in docs/prereg/, each with an explicit disposition, rather than deleted — see docs/prereg/README.md. E5_dimensionality/ keeps one file, a library three audit builders import.
5. The preregistrations record what was planned, including paths that predate later directory renames. They are content-locked and were deliberately never rewritten to match.

Citation

See CITATION.cff. Released under the MIT License (LICENSE).

Prior work this builds on: Yu M, Wang D, Shan Q, Reed CJ, Wan A. The Super Weight in Large Language Models. arXiv:2411.07191 (2024).