Offsets and Spectra: Predicting Which Correction a Frozen Representation Gap Needs

Code, stored results and preregistration records for the paper. The study asks which classical post-hoc correction (centring, whitening, rotation) a gap between two frozen representation spaces needs, and tests two inexpensive diagnostics: mean dominance (offset size) and spectral mismatch (covariance disagreement).

This repository accompanies the paper Offsets and Spectra: Predicting Which Correction a Frozen Representation Gap Needs. It contains the analysis code, the stored result files behind the tables, figures and numbers in the paper, and the preregistration records.

Layout

• experiments/review/ - analysis scripts (Python).
• tasks/ - small data-loading and embedding helpers that the scripts import.
• results/review/ - stored results (*.json), generated LaTeX table rows (*.tex), and the preregistration records (prereg*.json, which store SHA-256 hashes of the frozen scripts and files).
• requirements.txt, LICENSE (MIT, for the authors' code).

Which script produces which result

Scripts locate the repository root from their own path, so run them from the root of this repository, for example python experiments/review/variance_components.py. Two scripts read only the stored result files and need no download or GPU: variance_components.py and diag_selector.py; we verified that both regenerate their stored results exactly. The other scripts need the embeddings described below.

Environment used for the reported results

Linux 6.8, Python 3.12.3, PyTorch 2.14 (CUDA 13), transformers 5.17, scikit-learn 1.9, SciPy 1.17, NumPy 1.26; 2 x NVIDIA RTX 6000 Ada (49 GB), AMD EPYC 9124 (32 threads), 125 GB RAM. Regularised CCA at d = 1536 takes about 0.25 s on the CPU; embedding the 2,009 FLORES English sentences with one encoder takes roughly 4-20 s on one GPU; the 20-source by 17-target analysis (340 gaps) runs in about ten minutes.

Reproducing from raw data

python -m venv .venv && .venv/bin/python -m pip install -r requirements.txt
python experiments/review/benchmark.py xling_e5 cuda:0      # embeds with public Hugging Face encoders, caches to results/emb_cache
python experiments/review/ladder.py
python experiments/review/heldout.py cuda:0
python experiments/review/scale_embed.py cuda:0 && python experiments/review/scale_families.py

The first run downloads public datasets and encoder weights (FLORES-200, MS-COCO captions, MASSIVE, XLM-T sentiment, SLURP, LibriSpeech and the encoders cited in the paper). Raw embeddings, upstream data and model weights are not included; they are regenerated by the scripts. laya_embed.py needs the third-party laya package. Hugging Face revisions used for the batch-3 checkpoints are listed in the paper's appendix.

Notes

• The preregistration records are local JSON files with self-reported timestamps and hash chains; they show internal consistency, not external timing (Section 7 of the paper).
• results/emb_cache/librispeech__text__xlm-roberta-large__test.pt (not included) was found to be in a different row order from the LibriSpeech audio cache; librispeech_rerun_dataset_order.py rebuilds the transcript embeddings in dataset order (see the appendix of the paper).
• results/review/reviewer_checks_librispeech_corrected.json is the first LibriSpeech rerun, which paired the audio with a transcript cache in a different row order (near-chance retrieval); it is kept for transparency and is superseded by librispeech_dataset_order_rerun.json.
• No raw examples, annotations, or personal data are redistributed. Some public source datasets (for example the tweets of the XLM-T sentiment benchmark and the speech corpora LibriSpeech and SLURP) may contain personal or sensitive content and must be obtained from their original providers under their own terms.

License

MIT; see LICENSE.