GOLDMARK — Manuscript Reference Pipeline - https://artificialintelligencepathology.org/

This repository is in support of the manuscript GOLDMARK: Governed Outcome-Linked Diagnostic Model Assessment Reference Kit and is composed of the end-to-end benchmarking pipeline:

1. Target construction (TCGA via GDC download + variant labeling / OncoKB annotation)
2. Tiling (20x/40x coordinate generation)
3. Feature extraction (canonical pathology foundation models + custom encoders) with QC metadata
4. Training (gated-attention MIL reference head) on predefined patient-level splits
5. Inference + external inference and attention export (best checkpoint + fixed late epoch)

The design goal is reproducibility and clarity: cross-validation alone is not sufficient—the pipeline is built around reciprocal external testing (e.g., TCGA→external and external→TCGA) under identical preprocessing and evaluation criteria.

Manuscript Figures (Fig 1-3)

• Fig 1 (workflow): images/fig1_workflow.pdf
• Fig 2 (cross-validation + reciprocal external): images/fig2_cv_reciprocal_external.pdf
• Fig 3 (top eight task grid): images/fig3_top8_task_grid.pdf

Start here (real GDC download + end-to-end pipeline)

Downloads and labels real TCGA SVS slides via GDC and runs:

download → tiling → features → training → inference

This is a real-data run (WSI deps + OpenSlide required). Expect multiple TB of downloads and non-trivial runtime.

Preflight checklist (avoid the common failure modes)

• Most encoders are hosted on Hugging Face; approval + a valid HF token are required to run this pipeline. Without access you will hit 401 Unauthorized during feature extraction.
• Ensure configs/secrets.env exists and is sourced for non-interactive runs (SLURM/nohup). Example: set -a; source configs/secrets.env; set +a
• OncoKB is required for TCGA mutation labeling in this pipeline. Set ONCOKB_TOKEN in configs/secrets.env (or provide precomputed OncoKB labels). The pipeline will refuse to run without it.
• If you use h-optimus-0, your HUGGINGFACE_HUB_TOKEN (or HF_TOKEN) must have explicit access to bioptimus/H-optimus-0. A 401 Unauthorized means the token is missing or lacks access.
• gdc-client requires newer glibc on some clusters. If it fails, GOLDMARK falls back to the GDC API (slower). For full cohorts, prefer a compatible gdc-client or a newer node.
• Some clusters do not provide python. Use python3 or set PYTHON_BIN in the SLURM script.
• The LUAD KRAS end-to-end run defaults to the full cohort (--per-class 0). Set --per-class to a small value for a quick sanity check.

git clone https://github.com/chadvanderbilt/GOLDMARK.git
cd GOLDMARK

# Put tokens in one place (never commit the filled file).
# GOLDMARK commands will auto-load `configs/secrets.env` if present.
cp configs/secrets.env.example configs/secrets.env
# edit configs/secrets.env (GDC token file path, ONCOKB_TOKEN, HF_TOKEN, ...)

# Option A (recommended on HPC): conda
source /home/vanderbc/.bashrc
bash scripts/setup_conda_goldmark_env.sh
conda activate goldmark

# Option B (sudo/venv): installs system deps + creates venv
bash scripts/setup_goldmark_sudo.sh

# Installs gdc-client into bin/ (ignored by git)
python scripts/install_gdc_client.py --dest bin/gdc-client
# Note: some HPC nodes ship older glibc; if gdc-client fails to run, the pipeline
# will fall back to downloading via the GDC API (sufficient for small runs).

# Downloads 2 tumor + 2 normal slides (smallest by file size) and runs the full pipeline on CPU.
python scripts/gdc_smoke_test_tcga.py --project-id TCGA-COAD --per-class 2 --device cpu --force

Outputs land in runs/gdc_smoke_test/ (including runs/gdc_smoke_test/inference/inference/inference_results.csv).

Reciprocal TCGA→external end-to-end runs (mutation labels + external inference)

This repo ships two end-to-end scripts that exercise the full mutation-label → MIL training → attention export pathway and validate reciprocal external inference (TCGA→external):

1. scripts/tcga_to_external_smoke_test.py (fast, minimal; good for “does anything run?”)
2. scripts/tcga_cv_to_external_full_run.py (full run by default; 5-fold CV, attention exports, external inference)

Recommended: TCGA-LUAD KRAS (5×70/30 splits) → external LUAD inference

What it does:

• Downloads TCGA-LUAD diagnostic slides via GDC (filters to -00-DX unless --allow-non-dx)
• Only downloads slides with targets: the script labels cases using MAFs, then builds a subset manifest and downloads only those labeled slides.
• Labels each patient as KRAS positive/negative from the GDC Masked Somatic Mutation MAF
• Builds 5 independent 70/30 splits with per-split val assignments (train/val/test stored as columns)
• Trains for a full cohort by default and writes a cv_summary.csv
• Runs held-out test inference per split and exports attention vectors
• Runs external inference on the external LUAD cohort using the best split checkpoint (and links it from all split dirs)

python scripts/tcga_cv_to_external_full_run.py \
  --run-name TCGA-LUAD \
  --per-class 0 \
  --device cuda \
  --epochs 25 \
  --patience 50 \
  --force

Generic SLURM + nohup launchers (configurable project/gene/encoder)

For GitHub users, we provide generic launchers that accept options via environment variables. These are not hard-coded to a specific project or gene.

Make sure the goldmark conda environment is active (or set CONDA_ENV=goldmark before running the scripts).

# Example: minimal resume (20x+40x, no external)
export PROJECT_ID=TCGA-LUAD
export GENE=EGFR
export ENCODER=h-optimus-0
export RUN_NAME=TCGA-LUAD
export RUN_MODE=resume
export PER_CLASS=0
export TARGET_MPP=0.5
export EXTRA_TARGET_MPP=0.25
sbatch examples/slurm/submit_tcga_cv_to_external.sh

export PROJECT_ID=TCGA-LUAD
export GENE=EGFR
export ENCODER=h-optimus-0
export RUN_NAME=TCGA-LUAD
export RUN_MODE=resume
export PER_CLASS=0
export TARGET_MPP=0.5
export EXTRA_TARGET_MPP=
sbatch examples/slurm/submit_tcga_cv_to_external.sh

export PROJECT_ID=TCGA-LUAD
export GENE=EGFR
export ENCODER=h-optimus-0
export RUN_NAME=TCGA-LUAD
export RUN_MODE=resume
export PER_CLASS=0
export TARGET_MPP=0.25
export EXTRA_TARGET_MPP=
sbatch examples/slurm/submit_tcga_cv_to_external.sh

export PROJECT_ID=TCGA-LUAD
export GENE=EGFR
export ENCODER=h-optimus-0
export RUN_NAME=TCGA-LUAD
export RUN_MODE=resume
export PER_CLASS=0
export EXTERNAL_PER_CLASS=0
export TARGET_MPP=0.5
export EXTRA_TARGET_MPP=0.25
export EXTERNAL_MANIFEST=/path/to/external_manifest.csv
export EXTERNAL_ROOT=/path/to/foundation_model_training_images/EXTERNAL

sbatch examples/slurm/submit_tcga_luad_EGFR_cv_to_external.sh

External inference only runs if EXTERNAL_MANIFEST (and optionally EXTERNAL_ROOT) are set.

Minimum required columns in EXTERNAL_MANIFEST:

• Label column: one of <GENE> (e.g., EGFR), label_index, label, or target
• Slide ID column: one of slide_id, DMP_ASSAY_ID, sample_id, case_id, or slide

Optional columns:

• feature_path (absolute/relative path to features per slide; overrides EXTERNAL_ROOT)
• OncoTree_Code (used to resolve features under EXTERNAL_ROOT)
• scanned_slides_exist (rows with 0/false are skipped)

# Example: minimal resume (20x+40x, no external)
export PROJECT_ID=TCGA-LUAD
export GENE=EGFR
export ENCODER=h-optimus-0
export RUN_NAME=TCGA-LUAD
export RUN_MODE=resume
export PER_CLASS=0
export TARGET_MPP=0.5
export EXTRA_TARGET_MPP=0.25
bash scripts/nohup_tcga_cv_to_external.sh

export PROJECT_ID=TCGA-LUAD
export GENE=EGFR
export ENCODER=h-optimus-0
export RUN_NAME=TCGA-LUAD
export RUN_MODE=resume
export PER_CLASS=0
export TARGET_MPP=0.5
export EXTRA_TARGET_MPP=
bash scripts/nohup_tcga_cv_to_external.sh

export PROJECT_ID=TCGA-LUAD
export GENE=EGFR
export ENCODER=h-optimus-0
export RUN_NAME=TCGA-LUAD
export RUN_MODE=resume
export PER_CLASS=0
export TARGET_MPP=0.25
export EXTRA_TARGET_MPP=
bash scripts/nohup_tcga_cv_to_external.sh

export PROJECT_ID=TCGA-LUAD
export GENE=EGFR
export ENCODER=h-optimus-0
export RUN_NAME=TCGA-LUAD
export RUN_MODE=resume
export PER_CLASS=0
export EXTERNAL_PER_CLASS=0
export TARGET_MPP=0.5
export EXTRA_TARGET_MPP=0.25
export EXTERNAL_MANIFEST=/path/to/external_manifest.csv
export EXTERNAL_ROOT=/path/to/foundation_model_training_images/EXTERNAL
bash scripts/nohup_tcga_cv_to_external.sh

tail -f runs/<run-name>/logs/nohup.out

# Example: minimal force (20x+40x, no external)
export PROJECT_ID=TCGA-LUAD
export GENE=EGFR
export ENCODER=h-optimus-0
export RUN_NAME=TCGA-LUAD
export RUN_MODE=force
export PER_CLASS=0
export TARGET_MPP=0.5
export EXTRA_TARGET_MPP=0.25
bash examples/run_tcga_luad_egfr_end_to_end.sh

export PROJECT_ID=TCGA-LUAD
export GENE=EGFR
export ENCODER=h-optimus-0
export RUN_NAME=TCGA-LUAD
export RUN_MODE=force
export PER_CLASS=0
export TARGET_MPP=0.5
export EXTRA_TARGET_MPP=
bash examples/run_tcga_luad_egfr_end_to_end.sh

export PROJECT_ID=TCGA-LUAD
export GENE=EGFR
export ENCODER=h-optimus-0
export RUN_NAME=TCGA-LUAD
export RUN_MODE=force
export PER_CLASS=0
export TARGET_MPP=0.25
export EXTRA_TARGET_MPP=
bash examples/run_tcga_luad_egfr_end_to_end.sh

export PROJECT_ID=TCGA-LUAD
export GENE=EGFR
export ENCODER=h-optimus-0
export RUN_NAME=TCGA-LUAD
export RUN_MODE=force
export PER_CLASS=0
export EXTERNAL_PER_CLASS=0
export TARGET_MPP=0.5
export EXTRA_TARGET_MPP=0.25
export EXTERNAL_MANIFEST=/path/to/external_manifest.csv
export EXTERNAL_ROOT=/path/to/foundation_model_training_images/EXTERNAL
bash examples/run_tcga_luad_egfr_end_to_end.sh

Options (env vars)

• PROJECT_ID (default: TCGA-LUAD)
• GENE (default: KRAS)
• ENCODER (default: h-optimus-0)
• DEVICE (default: cuda)
• TARGET_MPP (default: 0.5)
• EXTRA_TARGET_MPP (default: 0.25; comma-separated MPP buckets — each slide is assigned to the nearest bucket)
• RUN_NAME (default: ${PROJECT_ID})
• RUN_MODE in {force|resume|rebuild} (default: force)
  • force: reset derived outputs only (tiling/features/training/inference/etc.) and preserve gdc_downloads/; re-downloads happen only if checksum/size validation fails.
  • resume: keep all existing outputs; only run missing stages (downloads/tiling/features/training/inference).
  • rebuild: keep downloads + derived labels, but re-run tiling, features, training, inference, external_inference, plots.
• PER_CLASS, EXTERNAL_PER_CLASS, LIMIT_TILES, EPOCHS, PATIENCE, VAL_PER_CLASS (default: 0 = use full test split for validation)

By default, RUN_NAME is set to the project id so all outputs live under runs/<project-id>/.

Output layout (important files and where to find them)

All outputs land under runs/<project-id>/ (the default RUN_NAME is the project id).

A) Project inputs (download + derived labels)

• runs/<project-id>/gdc_downloads/<PROJECT>_svs_manifest.tsv — full GDC SVS manifest for the project
• runs/<project-id>/gdc_downloads/svs/**/**.svs — downloaded TCGA SVS files (GDC UUID folders)
• runs/<project-id>/gdc_downloads/maf/**/**.maf.gz — downloaded GDC mutation calls used for labels
• runs/<project-id>/checkpoints/<TARGET>/manifests/<PROJECT>_<TARGET>_svs_manifest_subset.tsv — selected SVS rows
• runs/<project-id>/checkpoints/<TARGET>/manifests/<PROJECT>_<TARGET>_slides.csv — selected slide list + labels (schema below)
• runs/<project-id>/checkpoints/<TARGET>/manifests/external_manifest_<TARGET>_<encoder>.csv — external subset manifest (schema below)

B) Tiling

• runs/<project-id>/tiling/tiles_20x/manifests/<slide_id>_tiles.csv — per-slide tile coordinate manifest (schema below)
• runs/<project-id>/tiling/tiles_20x/cases/<case_id>_tiles.csv — per-case tile coordinates aggregated across slides
• runs/<project-id>/tiling/tiles_20x/cases/cases_index.csv — case_id → case manifest path index
• runs/<project-id>/tiling/tiles — alias pointing at tiles_20x for compatibility
• runs/<project-id>/tiling/tile_coords_20x.csv — aggregated tile coords (x,y,slide,sample_id,target)
• runs/<project-id>/tiling/tiling_manifest.csv — slide inventory (file_name,sample_id,target,slide_path)

When EXTRA_TARGET_MPP is set (e.g., 0.25), slides are bucketed by native MPP:

• runs/<project-id>/tiling/tiles_20x/... for slides closer to 0.5
• runs/<project-id>/tiling/tiles_40x/... for slides closer to 0.25
• runs/<project-id>/tiling/tile_coords_20x.csv and tile_coords_40x.csv each contain only the slides assigned to that bucket

Tiles keep the same physical size across buckets by scaling pixel size (e.g., 224px at 0.5 mpp ⇔ 448px at 0.25 mpp).

C) Features + QC

• runs/<project-id>/features/<encoder>/features_<slide_id>.pt — per-slide feature tensor (N tiles × D; tile size set by the slide’s MPP bucket)
• runs/<project-id>/features/<encoder>/features_<slide_id>.json — QC metadata + checksums (schema below)
• If feature extraction fails QC, tensors are renamed:
  • features_<slide_id>.FAILED_tile_count_mismatch.pt
  • features_<slide_id>.FAILED_degenerate_embeddings.pt

Feature tensors are saved as PyTorch .pt files with rows aligned to the slide’s tile manifest order, so each row can be joined back to tile coordinates using the corresponding tile manifest CSV.

Performance note: the feature extractor auto-tunes GPU batch size based on available VRAM and uses CPU data loader workers to keep the GPU fed. You can override both via --batch-size and --num-workers. As a rule of thumb, start with --num-workers 4 and increase to 8–16 on beefy GPU nodes if CPU tile decoding becomes the bottleneck.

When EXTRA_TARGET_MPP is set, features are still written once per slide (no suffix). Slides assigned to 40x are extracted with the larger pixel tile size, but the output path remains:

• runs/<project-id>/features/<encoder>/features_<slide_id>.pt

D) Training (5-fold CV)

• AGGREGATOR is the upper-case aggregation method (default: GMA).
• runs/<project-id>/training/checkpoints/<GENE>/versioned_split_manifest/<GENE>_all_splits_latest.csv — shared split manifest used for all encoders (schema below)
• runs/<project-id>/training/checkpoints/<GENE>/<ENCODER>/<AGGREGATOR>/classification_report/cv_summary.csv — per-split best epoch + metrics (schema below)
• runs/<project-id>/training/checkpoints/<GENE>/<ENCODER>/<AGGREGATOR>/split_1_set/checkpoint/checkpoint_best.pt — best checkpoint for that split (and similarly for split_2_set..split_5_set)

E) Per-split held-out test inference (attention export + ROC/PR plots)

Written under each split so split context is self-contained:

• runs/<project-id>/training/checkpoints/<GENE>/<ENCODER>/<AGGREGATOR>/split_1_set/inference/test/inference_results.csv
• runs/<project-id>/training/checkpoints/<GENE>/<ENCODER>/<AGGREGATOR>/split_1_set/inference/test/attention/<slide_id>_attention.csv
• runs/<project-id>/training/checkpoints/<GENE>/<ENCODER>/<AGGREGATOR>/split_1_set/inference/test/plots/roc_pr_curves.png

Note: the training stage also writes probability exports under the same directory (e.g. probabilities_test_set.csv).

F) External inference (external LUAD)

External inference runs for each split using the best AUC epoch from CV plus any configured epochs (e.g., the final epoch), and evaluates every slide in the external cohort. Results are written under each split:

• runs/<project-id>/training/checkpoints/<GENE>/<ENCODER>/<AGGREGATOR>/split_1_set/external_inference/external/ckpt_best_<epoch>/inference_results.csv
• runs/<project-id>/training/checkpoints/<GENE>/<ENCODER>/<AGGREGATOR>/split_1_set/external_inference/external/ckpt_best_<epoch>/attention/<slide_id>_attention.csv
• runs/<project-id>/training/checkpoints/<GENE>/<ENCODER>/<AGGREGATOR>/split_1_set/external_inference/external/ckpt_best_<epoch>/plots/roc_pr_curves.png

Additional epochs (e.g., final epoch) appear under:

• .../external_inference/external/ckpt_<epoch>/...

File schemas (column names)

TCGA slide selection manifest (training/checkpoints/<GENE>/manifests/<PROJECT>_<GENE>_slides.csv)

• slide_id (e.g. TCGA-05-4244-01Z-00-DX1)
• slide_path (downloaded .svs path)
• label_index (0/1 mutation label for the chosen gene)
• patient_id (TCGA case/patient barcode prefix)

Versioned split manifest (training/checkpoints/<GENE>/versioned_split_manifest/<GENE>_all_splits_latest.csv)

• slide_path, slide_id, label_index
• target (legacy column retained for compatibility)
• split_1_set … split_5_set with values in {train,val,test}

Tile manifest (tiling/tiles_20x/manifests/<slide_id>_tiles.csv)

• slide_id
• tile_id (stable tile identifier)
• x, y (level-0 pixel coordinates)
• level (OpenSlide level used for extraction / overlays)
• width, height (tile size in pixels at level)
• tissue_fraction (fraction of tile kept after tissue mask filtering)
• tile_path (optional; blank when only coordinates are generated)

Tile manifest with attention (.../attention/<slide_id>_tiles_with_attention.csv)

• Same columns as the tile manifest, plus attention for that slide.

Feature metadata JSON (features/<encoder>/features_<slide_id>.json)

• slide_id, encoder
• tile_manifest
• num_tiles, num_features, feature_dim
• embedding_stats (degenerate/variance checks)
• feature_sha256, feature_bytes
• status in {ok,failed} and failure_reason when failed Metadata is written once per slide (no suffix), regardless of its MPP bucket.

CV summary (training/checkpoints/<GENE>/<ENCODER>/<AGGREGATOR>/classification_report/cv_summary.csv)

• split (e.g. split_1_set)
• best_epoch
• val_* and test_* metrics, including *_roc_auc, *_accuracy, *_precision, *_recall, *_f1, *_balanced_error_rate

Inference results (.../inference_results.csv)

• slide_id
• probability (predicted P(class=1))
• prediction (thresholded at 0.5 by default)
• target (ground-truth label, from label_index)

Attention exports (.../attention/<slide_id>_attention.csv)

• slide_id
• tile_index (0-based)
• tile_id, x, y, level (when tile manifest metadata is available)
• attention (raw attention weight)
• probability (slide-level probability repeated for convenience)

External inference manifest (training/checkpoints/<GENE>/manifests/external_manifest_<GENE>_<encoder>.csv)

Minimum required columns for external inference with InferenceRunner:

• slide_id (string id used to resolve features / label rows)
• label_index (0/1 ground-truth label; used as target_column)

Strongly recommended columns (enables overlays + clearer provenance):

• slide_path (absolute .svs path for overlays; optional if you do not generate overlays)
• feature_path (absolute .pt path; optional if you pass --feature-dir and follow features_<slide_id>.pt naming)

Optional columns:

• split (only needed if you want to run inference on a subset via InferenceConfig.split_column/split_value)

Docs:

• docs/targets.md
• docs/pipeline.md
• SLURM (generic): examples/slurm/submit_tcga_cv_to_external.sh
• SLURM (EGFR example): examples/slurm/submit_tcga_luad_EGFR_cv_to_external.sh

Canonical pathology foundation model encoders

Each encoder below reflects the implementation in goldmark/features/canonical_sources.py, which follows the recommended preprocessing from the original model repositories. Access typically requires a Hugging Face token.

Repo: prov-gigapath/prov-gigapath
URL: https://huggingface.co/prov-gigapath/prov-gigapath

Repo: MCCPBR/EAGLE
URL: https://huggingface.co/MCCPBR/EAGLE

git clone --no-checkout https://huggingface.co/MCCPBR/EAGLE && cd EAGLE

from PIL import Image
import numpy as np
import eagle
import torch
import torchvision.transforms as transforms

# Load model
model = eagle.EAGLE()

# Set up transform
transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225))
])

# Image
img = np.random.randint(0, 256, size=224*224*3).reshape(224,224,3).astype(np.uint8)
img = Image.fromarray(img)
img = transform(img).unsqueeze(0)

# Inference
with torch.no_grad():
    h, att, p = model(img)

Repo: MahmoodLab/UNI
URL: https://huggingface.co/MahmoodLab/UNI

Repo: paige-ai/Virchow
URL: https://huggingface.co/paige-ai/Virchow

Repo: paige-ai/Virchow2
URL: https://huggingface.co/paige-ai/Virchow2

Repo: bioptimus/H-optimus-0
URL: https://huggingface.co/bioptimus/H-optimus-0

Repository layout

Installation

Preferred (conda):

source /home/vanderbc/.bashrc
bash scripts/setup_conda_goldmark_env.sh

If you have sudo and prefer a venv:

bash scripts/setup_goldmark_sudo.sh

Quickstart (typical flow)

Set a repo root (recommended):

export REPO_ROOT="$PWD"

1) Targets (TCGA example)

Generate a GDC manifest (SVS example):

python -m goldmark gdc-manifest svs \
  --project-id TCGA-COAD \
  --out tcga_coad_svs_manifest.tsv

Download data from GDC:

targets/tcga/gdc_download.sh --manifest tcga_coad_svs_manifest.tsv --out data/gdc_download
# If needed for controlled-access data, set GDC_TOKEN_FILE in configs/secrets.env
# (or pass --token /path/to/gdc_token.txt).

Annotate MAF with OncoKB (token via env var):

# export ONCOKB_TOKEN="..."  # or set in configs/secrets.env
python targets/variants/annotate_maf_oncokb_by_hgvsg.py \
  --maf-glob "data/gdc_download/**/**.maf.gz" \
  --oncotree-code <ONCOTREE_CODE> \
  --output data/oncokb/oncokb_annotations.csv

--oncotree-code ensures OncoKB levels are computed for the correct tumor type (e.g., LUAD for TCGA-LUAD).

Summarize a single gene’s patient-level mutation status:

python targets/variants/summarize_gene_status.py \
  --annotations data/oncokb/oncokb_annotations.csv \
  --gene <GENE> \
  --output data/targets/<GENE>_patient_labels.csv

The output includes per-patient p. changes and OncoKB level summaries for spot-checking:

• p_changes (joined with | for multiple mutations; no_gene_changes if none)
• oncokb_levels (joined with |)
• oncokb_positive / has_gene_mutation / label_reason

label_index is positive if any mutation is OncoKB level 1/2/3 for the target gene.

Build a slide-level manifest by joining SVS paths to patient labels:

python targets/tcga/build_slide_manifest_from_svs_and_mutations.py \
  --svs-root data/gdc_download \
  --labels data/targets/<GENE>_patient_labels.csv \
  --output data/manifests/TCGA_<GENE>_slide_manifest.csv

Generate the manuscript-style split manifest:

PYTHONPATH=. python scripts/generate_versioned_split_manifest.py \
  --manifest data/manifests/TCGA_<GENE>_slide_manifest.csv \
  --target <GENE> \
  --label-column label_index \
  --target-dir data/checkpoints/<GENE>

See manifest header examples:

• TCGA split manifest schema: examples/manifests/tcga_split_manifest_header.csv
• External manifest schema (header only): examples/manifests/external_LUAD_manifest_header.csv

2) Tiling (20x/40x coordinates)

The canonical tiler is goldmark/tiling/extractor.py. For convenience, this repo also exposes a small CLI:

PYTHONPATH=. python -m goldmark tiling data/manifests/TCGA_<GENE>_slide_manifest.csv \
  --output runs \
  --run-name tcga_<GENE>_demo \
  --tile-size 224 \
  --stride 224 \
  --target-mpp 0.5

Tile manifests are written under runs/<run-name>/tiling/tiles/manifests/ as <slide_id>_tiles.csv.

3) Feature extraction + QC metadata

Feature extraction writes:

• features_<slide_id>.pt (per-slide tensor)
• features_<slide_id>.json (QC metadata + checksums)

If tile/feature counts mismatch or embeddings are degenerate (low variance), the tensor is renamed: features_<slide_id>.FAILED_<reason>.pt and training/inference will treat the slide as missing.

Example:

PYTHONPATH=. python -m goldmark features data/manifests/TCGA_<GENE>_slide_manifest.csv \
  --tile-manifests runs/tcga_<GENE>_demo/tiling/tiles \
  --output runs \
  --run-name tcga_<GENE>_demo \
  --encoder prov-gigapath \
  --device cuda \
  --num-workers 4

Optional integrity audit (tile counts vs feature lengths):

python scripts/check_tile_feature_counts.py \
  --tile-manifest-dir runs/tcga_<GENE>_demo/tiling/tiles/manifests \
  --feature-dir runs/tcga_<GENE>_demo/features/prov-gigapath

4) Training (reference gated-attention MIL head)

Manuscript-scale training is orchestrated by:

• scripts/run_training_scan_target.py (scan encoders + submit missing runs)
• scripts/train_task_v2.py (train one target across 5 splits)

5) Inference + external inference + attention export

The manuscript pipeline exports attention vectors for (i) best checkpoint and (ii) a fixed late epoch to show how checkpoint selection affects results and to support consistent downstream overlays.

See:

• scripts/run_inference_from_plan.py
• scripts/gma_inference_pipeline.py
• SLURM template: examples/slurm/submit_attn_external_BLCA_ERBB2.sh

Reference figures/tables

• Workflow: paper/figures/workflow_schematic.pdf
• Tile/feature QC: paper/figures/qc_tile_feature_integrity.pdf
• Table 1 (counts): paper/tables/table1_gene_mutation_counts.tex

Manuscript tasks (Table 1)

Launch all manuscript tasks (optional)

This repo includes a convenience launcher that loops over the manuscript task list and calls scripts/run_training_scan_target.py for each tumor/target.

export MIL_DATA_ROOT="/path/to/foundation_model_training_images"

# dry-run (prints commands)
python scripts/launch_manuscript_tasks.py

# execute (runs training scans)
python scripts/launch_manuscript_tasks.py --execute

Note: this wrapper runs commands sequentially. On HPC clusters you will typically wrap each printed command in sbatch (or your scheduler of choice).

Run a full project workflows

python scripts/api_demo_from_preextracted_features.py \
  --base-url https://artificialintelligencepathology.org \
  --slug rf207d22e2c0c~TCGA-BLCA_svs \
  --gene FGFR3 \
  --encoder uni \
  --work-dir runs/api_demo_blca_fgfr3_uni \
  --sample-per-class 40 \
  --epochs 5 \
  --device cuda

BASE_URL="https://artificialintelligencepathology.org"
RUN_SLUG="rf207d22e2c0c~TCGA-BLCA_svs"
CATALOG_URL="${BASE_URL}/api/runs/${RUN_SLUG}/downloads?download_source=postgres&results_mode=publication&epoch_view=best"

curl -fsS "${CATALOG_URL}" | jq -r --arg base "${BASE_URL}" '
  .categories[]
  | select(.key=="features")
  | .items[]
  | select(.download_url != null)
  | [
      .label,
      ((.path // .relative_path // (.label + ".download")) | split("/") | .[-1]),
      (if (.download_url | startswith("http")) then .download_url else ($base + .download_url) end)
    ]
  | @tsv
'

BASE_URL="https://artificialintelligencepathology.org"
RUN_SLUG="rf207d22e2c0c~TCGA-BLCA_svs"
CATEGORY="manifest"
CATALOG_URL="${BASE_URL}/api/runs/${RUN_SLUG}/downloads?download_source=postgres&results_mode=publication&epoch_view=best"
LOG="goldmark_download_$(date -u +%Y%m%dT%H%M%SZ).log"

FIRST_ROW=$(curl -fsS "${CATALOG_URL}" | jq -r --arg base "${BASE_URL}" --arg category "${CATEGORY}" '
  .categories[]
  | select(.key==$category)
  | .items[]
  | select(.download_url != null)
  | [
      ((.path // .relative_path // (.label + ".download")) | split("/") | .[-1]),
      (if (.download_url | startswith("http")) then .download_url else ($base + .download_url) end)
    ]
  | @tsv
' | head -n 1)

FILENAME=$(printf '%s\n' "${FIRST_ROW}" | cut -f1)
DOWNLOAD_URL=$(printf '%s\n' "${FIRST_ROW}" | cut -f2-)

{
  printf 'started_utc=%s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)"
  printf 'filename=%s\nurl=%s\n' "${FILENAME}" "${DOWNLOAD_URL}"
  curl --http1.1 -fL --continue-at - -o "${FILENAME}" \
    --retry 1000 --retry-all-errors --retry-delay 30 --connect-timeout 30 \
    --speed-time 300 --speed-limit 1024 \
    --write-out '\nhttp_code=%{http_code}\nremote_ip=%{remote_ip}\nsize_download=%{size_download}\ntime_total=%{time_total}\nurl_effective=%{url_effective}\n' \
    "${DOWNLOAD_URL}"
  printf 'finished_utc=%s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)"
} 2>&1 | tee -a "${LOG}"

slide_id,slide_path,label_index,split
CASE_001,/absolute/path/CASE_001.svs,1,test
CASE_002,/absolute/path/CASE_002.svs,0,test

PYTHONPATH=. python -m goldmark tiling user_manifest.csv \
  --output runs \
  --run-name user_slides_fgfr3 \
  --tile-size 224 \
  --stride 224 \
  --target-mpp 0.5

PYTHONPATH=. python -m goldmark features user_manifest.csv \
  --tile-manifests runs/user_slides_fgfr3/tiling/tiles \
  --output runs \
  --run-name user_slides_fgfr3 \
  --encoder uni \
  --device cuda \
  --num-workers 4

BASE_URL="https://artificialintelligencepathology.org"
RUN_SLUG="rf207d22e2c0c~TCGA-BLCA_svs"
GENE="FGFR3"
ENCODER="uni"
export GENE ENCODER

DOWNLOADS_JSON="$(mktemp)"
export DOWNLOADS_JSON
curl -sS "${BASE_URL}/api/runs/${RUN_SLUG}/downloads?download_source=postgres&results_mode=publication&epoch_view=best" > "${DOWNLOADS_JSON}"

BEST_JSON="$(python - <<'PY'
import json,sys,re,os
obj=json.load(open(os.environ["DOWNLOADS_JSON"]))
gene=os.environ["GENE"].upper()
enc=os.environ["ENCODER"].lower()
def norm(x): return re.sub(r'[^a-z0-9]+','',str(x).lower())
rows=[]
for cat in obj.get("categories",[]):
    if cat.get("key")!="slide_models":
        continue
    for it in cat.get("items",[]):
        m=it.get("metadata") or {}
        if str(m.get("gene","")).upper()!=gene:
            continue
        if norm(m.get("encoder",""))!=norm(enc):
            continue
        score=m.get("external_auc")
        if score is None:
            score=m.get("cv_auc")
        rows.append((float(score) if score is not None else -1.0, it))
if not rows:
    raise SystemExit("No matching slide_models found.")
rows.sort(key=lambda t:t[0], reverse=True)
print(json.dumps(rows[0][1]))
PY)"
export BEST_JSON

CKPT_URL="$(python -c 'import json,os,urllib.parse; u=json.loads(os.environ["BEST_JSON"])["download_url"]; b=os.environ["BASE_URL"].rstrip("/") + "/"; print(u if u.startswith(("http://","https://")) else urllib.parse.urljoin(b, u.lstrip("/")))')"
mkdir -p models
curl --http1.1 -fL --continue-at - -o "models/${GENE}_${ENCODER}_best_external_auc.pt" "${CKPT_URL}"
rm -f "${DOWNLOADS_JSON}"

GENE="FGFR3"
ENCODER="uni"

PYTHONPATH=. python -m goldmark inference user_manifest.csv \
  --feature-dir "runs/user_slides_fgfr3/features/${ENCODER}" \
  --checkpoint "models/${GENE}_${ENCODER}_best_external_auc.pt" \
  --output runs \
  --run-name "user_slides_fgfr3_${ENCODER}_inference" \
  --target label_index \
  --split-column split \
  --split-value test \
  --export-attention \
  --no-overlays

BASE_URL="https://artificialintelligencepathology.org"
RUN_SLUG="rf207d22e2c0c~TCGA-BLCA_svs"
GENE="FGFR3"
ENCODER="uni"
export GENE ENCODER

DOWNLOADS_JSON="$(mktemp)"
export DOWNLOADS_JSON
curl -sS "${BASE_URL}/api/runs/${RUN_SLUG}/downloads?download_source=postgres&results_mode=publication&epoch_view=best" > "${DOWNLOADS_JSON}"

python - <<'PY' > /tmp/checkpoints.tsv
import json,sys,re,os
obj=json.load(open(os.environ["DOWNLOADS_JSON"]))
gene=os.environ["GENE"].upper()
enc=os.environ["ENCODER"].lower()
def norm(x): return re.sub(r'[^a-z0-9]+','',str(x).lower())
for cat in obj.get("categories",[]):
    if cat.get("key")!="slide_models":
        continue
    for it in cat.get("items",[]):
        m=it.get("metadata") or {}
        if str(m.get("gene","")).upper()!=gene:
            continue
        if norm(m.get("encoder",""))!=norm(enc):
            continue
        split=str(m.get("split","na"))
        ext=m.get("external_auc")
        cv=m.get("cv_auc")
        url=it['download_url']
        base=os.environ.get("BASE_URL", "https://artificialintelligencepathology.org").rstrip("/") + "/"
        if not str(url).startswith(("http://", "https://")):
            import urllib.parse
            url=urllib.parse.urljoin(base, str(url).lstrip("/"))
        print(f"{split}\t{ext}\t{cv}\t{url}")
PY
rm -f "${DOWNLOADS_JSON}"

while IFS=$'\t' read -r SPLIT EXT_AUC CV_AUC URL; do
  OUT_CKPT="models/${GENE}_${ENCODER}_split${SPLIT}.pt"
  curl --http1.1 -fL --continue-at - -o "${OUT_CKPT}" "${URL}"
  PYTHONPATH=. python -m goldmark inference user_manifest.csv \
    --feature-dir runs/user_slides_fgfr3/features/uni \
    --checkpoint "${OUT_CKPT}" \
    --output runs \
    --run-name "user_slides_fgfr3_uni_split${SPLIT}" \
    --target label_index \
    --split-column split \
    --split-value test \
    --no-overlays
done < /tmp/checkpoints.tsv

Notes

• This repo intentionally does not ship raw WSIs or protected clinical data.
• OncoKB and GDC access are governed by their respective terms; keep tokens out of git history.
• The codebase is built around downloadable artifacts and reproducible workflows exposed via the GOLDMARK API: https://artificialintelligencepathology.org/