expert-text-transfer

Companion repository for the paper:

Okauchi, R. and Chun, P.-J. (2026). Fine-grained geological descriptions enable cross-region prediction of penetration resistance across national borehole archives. Submitted to Nature Communications.

This repository is the canonical reproduction package for every numerical result, table and figure in the paper. The paper makes a single claim with a cautionary contrast: the expert's free-text description of what was observed — recorded in nearly every borehole, field and survey log yet discarded by predictive models — carries information that transfers to out-of-region prediction of a physical quantity (standard penetration resistance, SPT-N), established on two independent national borehole archives (Japan KuniJiban, 2.66 M records; UK BGS) and reproduced in a maximally different domain (US storm reports), whereas learned spatial coordinates memorize location and do not transfer.

It is the sister repository of kanto-calibrated-spt-prior (the single-region precursor paper); it contains only the code, tests and result-provenance artefacts needed to reproduce this paper's results, independent of the research monorepo in which they were developed.

Quick reference: paper headline numbers ↔ provenance files

Every number below is quoted in the manuscript; each traces to a checked-in JSON produced by a script in this repository (see REPRODUCIBILITY.md for the full number → command → file map, and prereg/ for the pre-registration the confirmatory analyses were run against).

Primary analysis — borehole-block permutation null (pre-registered)

Earlier rounds (still reported; superseded as the headline)

What changed since the earlier draft

Two corrections to the statistical machinery; both are recorded in prereg/2026-08-14_nc_text_prereg_amendment_1.md and the numbers above are all post-correction.

1. The borehole-block permutation null is now a bijection. Its predecessor drew a donor borehole by free permutation and then clipped the donor's text block to the recipient's length, repeating the donor's deepest layer whenever the recipient was longer — 42.7 % of null-arm rows duplicated, 22,547 source rows never used. That is not a permutation of the embedding matrix, so the observed statistic was not exchangeable with the null draws, and every effect it touched was inflated by 0.6–1.4 pp. national/evaluation/grouped_null.py now performs a length-matched whole- block derangement with a rank-wise residual; every draw is checked and every artefact records all_draws_bijective. The defective routine is kept, unused, as legacy_clipped_block_permutation so the bias can be measured — tests/national/test_grouped_null.py asserts that the new routine returns a permutation and that the legacy one does not.
2. Fold p-values are combined by Bonferroni-corrected minimum and Cauchy. Leave-one-region-out folds share most of their training rows, so they are dependent and Stouffer (which assumes independence) is anti-conservative. Both reported combinations are valid under arbitrary dependence. Intervals are borehole-block BCa over paired per-borehole losses using the full leave-one-out jackknife, with the null accumulated over all draws.

Layout

scripts/    entry points (python -m scripts.<name> from the repo root)
national/   library code (models, evaluation harnesses, calibration, data tools)
tests/      unit tests for the harness code (pytest tests/ -q)
results/    result-provenance JSONs for every manuscript number
prereg/     the pre-registration, its amendment, and the verdicts (verbatim)
manifest.json  sha256 of every file at assembly time

Key entry points:

• scripts/nc_grouped_null.py — the pre-registered primary analysis: borehole-block permutation null with the (1+r)/(1+n) correction, seed pairing, Bonferroni/Cauchy fold combination and borehole-block BCa intervals, on the primitive in national/evaluation/grouped_null.py. --strip-mode lithology_only_depersonalised is the depersonalisation control; --regions/--shard/--combine shard it by held-out region.
• scripts/nc_provenance_folds.py — leave-project-, leave-contractor-, leave-client-, leave-year- and leave-schema-out evaluation.
• scripts/nc_fewshot_curve.py — borehole-budgeted few-shot curve over a holdout shared by every budget, arm and seed.
• scripts/nc_descriptor_families.py — descriptor-family mechanism, on the parser in scripts/text_leakage_controls.py (vocabulary strips, dictionary, structured lithology parser, character n-grams).
• scripts/attach_identity_to_parquet.py, scripts/extract_kunijiban_metadata.py, scripts/audit_text_join.py — the borehole identity spine, the national archive-header pass across all six DTD generations, and the join audit.
• scripts/nc_null_controls.py — the earlier row-level shuffled-embedding nulls that the block null supersedes; retained because the paper reports the row-vs-block contrast.
• scripts/uk_transfer_test.py, scripts/japan_transfer_test.py, scripts/storm_transfer_test.py — the three-domain leave-region-out transfer harnesses (leak-proof per-fold PCA, multi-seed); scripts/nc_geo_ablation.py, scripts/nc_depth_band.py — geological-fold and depth-band robustness.
• scripts/train_kanto_smoke.py --leave-region <r> [--zero-fourier] [--no-residual-geo] — the from-scratch national DKL coordinate-ablation cells, including the genuinely coordinate-free arm; scripts/train_lmc_national.py — multi-task LMC training.
• scripts/run_mondrian_recal_grouped.py — per-regime split-conformal recalibration under row-, borehole- and site-grouped calibration splits, with interval width and Winkler score alongside coverage; scripts/run_mondrian_recal_lmc.py — the LMC counterpart. national/evaluation/calibration.py implements the Mondrian quantiles described in Methods.
• scripts/build_paper2_figs.py, scripts/build_forest_plot.py, scripts/build_mondrian_recal_appendix_table.py — figures and SI tables.

Data availability

Raw KuniJiban borehole XML is not redistributable by us; it is publicly downloadable from https://www.kunijiban.pref.hokkaido.lg.jp / the national KuniJiban portal (see the paper's Data Availability). UK BGS AGS borehole records are available under BGS terms from the BGS AGS download service. NOAA storm reports are US public domain. Derived, redistributable artefacts (embeddings, calibration tables, predictions, the trained national model) are archived on Zenodo (DOI in the paper's Data Availability statement). The results/ artefacts in this repository let you verify the paper's statistical results — every effect size, interval, p-value, fold count and coverage figure — without downloading any raw data. The trained-model tables (the model hierarchy, the per-region leave-region-out detail, the test-time-adaptation sweep and the 23-cell Mondrian recalibration tables) are read from per-run artefacts in the Zenodo bundle; results/ carries the collated written record of those, and REPRODUCIBILITY.md §14 states exactly which numbers fall outside this repository and where they live.

Environment

pip install -e .          # or: uv sync
pytest tests/ -q          # 294 tests, CPU-only, ~30 s

Five of those tests skip without the raw corpora (three leave-region-out cases need the administrative-polygon lookup and the Kanto corpus, two leakage cases need the real text corpus); the other 289 run on synthetic fixtures and must pass for the build to succeed.

Transfer/leakage/null experiments are CPU-only (hours; the full-population grouped-null shards are 1–3 h each); the national DKL coordinate-ablation cells require a GPU (~12 h/cell at n_inducing=6000).

License

MIT (code). See LICENSE. Cite via CITATION.cff.