PathForge

PathForge is a modular benchmarking framework for multiple instance learning (MIL) in computational pathology. It supports WSI feature extraction, H5 artifact generation, tile overview reports, MIL benchmarking, pipeline optimization, support for classification, regression, survival and retrieval tasks, and support for model inference and visualization.

PathForge is the successor to and replacement for PathBench-MIL. PathBench-MIL is expected to be deprecated; new development and new projects should use PathForge.

Documentation

Start with the documentation home, then use the page that matches your task:

• End-to-end classification tutorial and the complete tutorial index
• Introduction for PathForge's purpose, supported tasks, and pipeline-level approach
• Installation and quickstart
• Data preparation
• Configuration reference
• MIL benchmark and optimization options
• Backend integrations
• API reference and API usage examples
• HDF5 artifact layout
• Task outputs, metrics, and visualizations
• Testing and troubleshooting

The end-to-end tutorial covers slide and annotation preparation, feature extraction, MIL training, evaluation, and packaged-model inference. The configuration reference defines the complete YAML schema; the MIL options page identifies benchmark grids, installed backend catalogs, and config-defined Optuna search spaces.

Ready-to-edit benchmark, optimization, feature-extraction, and distributed SLURM templates are provided in default_config/.

What PathForge Does

PathForge is organized around a config-driven workflow:

1. Read a YAML config.
2. Create an experiment folder with copied annotations and metadata.
3. Build WSI datasets from annotation rows.
4. Generate tiling and feature-extraction combinations.
5. Write per-slide .h5 artifacts with coordinates, tiling specs, optional tile overview images, and extracted features.
6. Train or evaluate MIL models through registry-selected trainers and losses.
7. Run benchmark grids or Optuna optimization using the same registry boundary.

Primary functionality:

• Feature extraction: tile WSIs, segment tissue, extract tile features, and persist row-aligned H5 artifacts.
• Single-slide feature extraction: process one WSI from a larger configured dataset, useful for SLURM array jobs.
• Tile reports: render PDF reports from stored tiles_overview payloads.
• Benchmarking: evaluate MIL model/loss combinations from config grids.
• Slide retrieval: rank reference slides against query slides using bag-level features and configurable representation/search strategies.
• Optimization: run Optuna studies over model and training choices.
• Inference: config-driven inference over a CSV of slides (pathforge-infer), plus packaged-model prediction and heatmap generation from a single feature artifact (pathforge-infer-model).
• Backends: native PathForge models plus optional TorchMIL and MIL-Lab models through backend adapters.
• Metrics/loss adapters: optional TorchMetrics and TorchSurv integrations.
• Explainability: optional heatmap adapter for per-instance MIL scores.

Installation

Recommended install — LazySlide feature extraction is included by default; this command adds TorchMIL, TorchMetrics, and TorchSurv:

uv sync --extra mil-backends

For GPU (CUDA 12.8) builds, add --extra cu128:

uv sync --extra mil-backends --extra cu128

Development install (adds pytest):

uv sync --extra mil-backends --extra dev

Individual extras:

mil-backends installs:

• torchmil
• torchmetrics
• torchsurv

These packages are optional. Native PathForge workflows must remain import-safe and runnable without them.

MIL-Lab is also supported as an optional MIL backend, but it is not distributed through the mil-backends extra. Install it from the MIL-Lab repository following its upstream instructions; PathForge detects the installed builder at runtime.

Command Line Entry Points

PathForge installs a single umbrella command, pathforge, plus flat console scripts for the most common workflows. Every workflow command accepts --config (a YAML file) and an optional --log-level {DEBUG,INFO,WARNING,ERROR}.

Unified pathforge command

pathforge features run    --config features.yaml
pathforge features slide  --config features.yaml --dataset TrainingSet --input /path/to/slide.svs
pathforge benchmark run   --config benchmark.yaml
pathforge evaluate run    --config benchmark.yaml
pathforge visualize run   --config benchmark.yaml
pathforge visualize summary --input /project/benchmark_results.csv
pathforge optimize run    --config optimize.yaml
pathforge optimize worker --config optimize.yaml --trials 10
pathforge optimize finalize --config optimize.yaml
pathforge execution plan  --config benchmark.yaml --output /project/work/plan
pathforge execution run   --plan /project/work/plan/plan.json --stage features --backend local
pathforge execution status --plan /project/work/plan/plan.json
pathforge execution aggregate --plan /project/work/plan/plan.json
pathforge report tiles    --config features.yaml
pathforge retrieval representations --config retrieval.yaml
pathforge retrieval mean-rgb        --config retrieval.yaml --dataset ReferenceSet --slide-id SLIDE_001
pathforge retrieval sish-vqvae      --config retrieval.yaml
pathforge infer run       --config inference.yaml --input-csv slides.csv

Distributed feature extraction creates one work shard per slide by default. To submit fewer, longer-running local, Dask, or SLURM tasks, configure how many slides each shard processes sequentially:

execution:
  slides_per_shard: 4
  max_shards: 50

For example, 10 slides with slides_per_shard: 4 produce three feature shards containing 4, 4, and 2 slides. The default is 1. Optional max_shards caps the total number of feature shards by grouping more slides sequentially when needed. execution.slurm.max_concurrent is separate: it limits how many SLURM array elements run simultaneously, not how many shards the plan contains.

Run pathforge --help (or pathforge <group> --help) to list every command.

Flat console scripts

Shortcuts are installed for the common workflows:

pathforge-features   --config features.yaml
pathforge-benchmark  --config benchmark.yaml
pathforge-evaluate   --config benchmark.yaml
pathforge-optimize   --config optimize.yaml
pathforge-visualize  --config benchmark.yaml
pathforge-mean-rgb   --config retrieval.yaml --dataset ReferenceSet --slide-id SLIDE_001
pathforge-slide-retrieval-representations --config retrieval.yaml
pathforge-infer      --config inference.yaml --input-csv slides.csv

Packaged-model inference and heatmaps

pathforge-infer-model runs a packaged checkpoint on a single feature artifact and can optionally attach a heatmap:

pathforge-infer-model \
  --model_path checkpoints/best_package.pt \
  --input artifacts/SLIDE_001.h5 \
  --output predictions/SLIDE_001.json

Data Layout

Annotation CSV

The experiment copies experiment.annotation_file into the experiment root as annotations.csv. WSI datasets expect at least these columns:

dataset,slide,patient,category
TrainingSet,SLIDE_001,PATIENT_001,case
TrainingSet,SLIDE_002,PATIENT_002,control

Optional column:

• fallback_mpp: positive floating-point microns-per-pixel fallback used when a WSI backend cannot read valid base MPP metadata.
• wsi_path: explicit absolute or relative slide path. When present and valid, PathForge uses it instead of resolving {slide} inside datasets[].slides_dir.

Rules:

• dataset must match one entry in datasets[].name.
• slide is resolved as either an exact direct file {slides_dir}/{slide}.<supported_suffix> or an exact DICOM folder {slides_dir}/{slide}/*.dcm.
• Supported WSI suffixes include .svs, .ndpi, .tiff, .tif, and .mrxs.
• patient and category are preserved in WSI metadata and downstream grouping.

Dataset Directories

Each dataset entry points to slide inputs and artifact outputs:

datasets:
  - name: TrainingSet
    slides_dir: /data/slides/train
    artifacts_dir: /data/pathforge_artifacts/train
    tissue_annotations_dir: null
    used_for: training

artifacts_dir is created if needed. Each slide writes one H5 file:

artifacts_dir/{slide_id}.h5

TCGA-Tools Datasets

PathForge can call the tcga-tools package to check whether requested datasets exist in TCGA or TCIA, download metadata first, select the configured task column, and download image data only when it is missing.

Install this optional integration before using remote dataset declarations:

uv sync --extra tcga

tcga-tools is intentionally optional because the repository supplies it as a local uv source and it is not published on PyPI. Standard pip and Read the Docs installations therefore do not attempt to resolve it.

datasets:
  - source: gdc
    dataset_names: ["TCGA-LUSC", "TCGA-LUAD"]
    annotation_column: diagnoses.0.vital_status
    metadata_table: clinical_csv
    annotations: ["clinical"]
    datatype: ["wsi"]
    used_for: ["training", "testing"]

This allows users to:

• specify TCGA or TCIA datasets directly in the PathForge config
• let PathForge validate those dataset names through tcga-tools
• generate a PathForge annotation CSV automatically under datasets/
• split one downloaded dataset across multiple roles when used_for contains more than one role

To find which columns exist for a dataset, use tcga-tools to do a metadata-only download first and inspect the generated CSV files such as files_metadata.csv, clinical.csv, molecular_index.csv, or diagnosis.csv. The chosen column name then becomes annotation_column in the PathForge config.

Configuration Reference

The canonical, maintained schema is the configuration reference. See the MIL options overview for benchmark axes and Optuna search-space syntax. The examples below provide a compact orientation.

Minimal feature extraction config:

experiment:
  project_name: example_features
  annotation_file: /data/annotations.csv
  project_root: /data/pathforge_projects
  mode: feature_extraction
  task: null
  report: true
  mixed_precision: true
  num_workers: 8

slide_processing:
  backend: lazyslide
  save_tiles: false
  segmentation_method: otsu
  qc_filters: []

datasets:
  - name: TrainingSet
    slides_dir: /data/slides/train
    artifacts_dir: /data/artifacts/train
    tissue_annotations_dir: null
    used_for: training

benchmark_parameters:
  tile_px: [256]
  tile_mpp: [0.5]
  feature_extraction: [resnet18]
  mil: []

weights_dir: ./pretrained_weights

Top-level sections:

• experiment: project lifecycle, task, mode, reporting, and workers.
• slide_processing: WSI backend and tissue/tiling behavior.
• datasets: slide directories, artifact directories, and dataset roles.
• benchmark_parameters: candidate pipeline values. Each task selects its own active grid keys; current MIL benchmarks vary feature extractor, tile size, resolution, MIL model, and loss.
• mil: training loop, backend selection, model kwargs, and MIL hyperparameters.
• metrics: metric backend selection.
• explainability: heatmap backend selection.
• optimization: Optuna study settings.

Supported experiment.mode values:

• feature_extraction
• benchmark
• optimization

Supported experiment.task values:

• classification
• regression
• survival
• survival_discrete
• slide_retrieval

experiment.task may be omitted only for feature_extraction mode.

Feature Extraction

Feature extraction creates WSI H5 artifacts. It uses:

• benchmark_parameters.tile_px
• benchmark_parameters.tile_mpp
• benchmark_parameters.feature_extraction
• slide_processing.backend
• slide_processing.segmentation_method
• experiment.report

Run all configured datasets and combinations:

pathforge-features --config features.yaml --log-level INFO

The policy builds combinations over:

feature_extraction x tile_px x tile_mpp

For each slide and combination, PathForge:

1. Validates base MPP.
2. Reuses existing valid coordinates and tiling specs when possible.
3. Segments or loads tissue polygons.
4. Extracts tile coordinates.
5. Writes coords and tiling_spec to H5.
6. Optionally writes tiles_overview when experiment.report: true.
7. Extracts tile features.
8. Writes feature matrices row-aligned with coordinates.

Coordinates are stored as int32 arrays shaped (N, 5):

[x_level0, y_level0, read_w, read_h, level]

Feature matrices are stored as floating arrays shaped (N, D), where rows align exactly with coords.

Single-Slide Feature Extraction

Use this for cluster jobs where each task processes one slide:

pathforge features slide \
  --config features.yaml \
  --dataset TrainingSet \
  --input /data/slides/train/SLIDE_001.svs \
  --log-level INFO

Requirements:

• --dataset must match one configured dataset name.
• --input must exist.
• The source annotation CSV must contain exactly one row matching the selected dataset and slide stem.

The CLI rewrites the project annotations for that invocation to a single row and then runs all configured feature extraction combinations for the selected slide. When SLURM_JOB_ID is present, the project name is suffixed with the job id to avoid collisions between array jobs.

Tile Overview Reports

If experiment.report: true, feature extraction writes tiles_overview image bytes into the slide H5 files. Generate PDF reports after extraction with:

pathforge report tiles --config features.yaml --log-level INFO

The report CLI derives bag ids from all configured tile_px and tile_mpp combinations. A bag id has this format:

{tile_px}px_{tile_mpp:g}mpp

Example:

256px_0.5mpp

The report CLI skips dataset/bag combinations where no overview exists yet and returns a non-zero exit code only when unexpected report generation failures occur.

H5 Artifact Contract

PathForge writes one H5 artifact per slide. The layout is backend-agnostic and row-aligned:

• coordinates: (N, 5) int32
• tiling spec: JSON-compatible metadata
• features: (N, D) floating matrix
• tile overview: compressed image bytes as a one-dimensional uint8 payload

Invariants:

• Coordinates and features share row order.
• Tiling specs include tile_px, tile_mpp, stride_px, and coord_space="level0".
• Feature extraction can skip recomputation when valid rows already exist.
• Reports are optional and should not affect feature matrix row alignment.

MIL Backends

PathForge supports three MIL backend modes:

• native: use PathForge model classes registered directly in MODELS.
• torchmil: use one generic TorchMIL adapter registered under the PathForge model key torchmil.
• mil-lab: use one generic MIL-Lab adapter registered under the PathForge model key mil-lab.

TorchMIL, MIL-Lab, TorchMetrics, and TorchSurv are optional integrations. They are not required to import PathForge or to run native workflows. mil-backends installs TorchMIL and the metric packages; MIL-Lab must be installed separately from its upstream repository. Package-specific imports are confined to:

• src/pathforge/adapters/...
• src/pathforge/utils/optional/...

Trainer, policy, config, and domain code select implementations through configuration and registries. They do not call torchmil, MIL-Lab, torchmetrics, or torchsurv directly.

Native Backend

Use native when you want existing PathForge models and no optional MIL backend dependency.

experiment:
  project_name: native_benchmark
  annotation_file: /data/annotations.csv
  mode: benchmark
  task: classification

mil:
  backend: native
  batch_size: 1
  epochs: 20

metrics:
  classification_backend: native

benchmark_parameters:
  feature_extraction: [resnet18]
  mil: [PerceiverMIL]
  loss: [CrossEntropyLoss]

Native datasets return canonical bag dictionaries:

sample = dataset[index]

where sample["X"] is a finite floating tensor shaped [N, D] for one slide bag, and sample["Y"] is the task label.

TorchMIL Backend

Use torchmil when you want TorchMIL models while keeping PathForge's trainer, policy, dataset, and registry contracts.

experiment:
  project_name: torchmil_benchmark
  annotation_file: /data/annotations.csv
  mode: benchmark
  task: classification

mil:
  torchmil_model_kwargs:
    in_shape: [1024]
    out_shape: 2
  use_torchmil_collate: true
  batch_size: 4
  epochs: 20

metrics:
  classification_backend: torchmetrics

benchmark_parameters:
  feature_extraction: [resnet18]
  mil: [ABMIL, CLAM]
  loss: [CrossEntropyLoss]

Important rules:

• benchmark_parameters.mil contains concrete available model names from PathForge, TorchMIL, or MIL-Lab.
• mil.torchmil_model_kwargs are forwarded to the TorchMIL constructor.
• mil.use_torchmil_collate: true enables padded dict batches compatible with TorchMIL semantics.
• The generic TorchMILBackendModel is the only PathForge model adapter for TorchMIL models.

If a TorchMIL model is selected but TorchMIL is unavailable, config validation reports that the model is not registered in the active environment.

MIL model 'ABMIL' not found in registry.

Canonical MIL Bag Batch

The TorchMIL integration introduces a canonical batch schema shared by adapters:

batch = {
    "X": features,       # float tensor [B, N, D]
    "Y": labels,         # labels [B] or survival target dict
    "mask": mask,        # optional bool tensor [B, N], true = real instance
    "coords": coords,    # optional tensor [B, N, 2]
    "adj": adj,          # optional tensor [B, N, N]
    "y_inst": y_inst,    # optional instance labels [B, N]
}

Shape and value contracts:

• X is floating point, finite, and shaped [N, D] for a single bag or [B, N, D] for a batch.
• mask is boolean or integer binary and shaped [B, N].
• coords is shaped [B, N, 2] and stores x/y instance coordinates.
• adj is shaped [B, N, N]; avoid this for large WSI bags unless the selected model requires graph structure.
• Padded instances are zero-filled and marked false in mask.

Datasets and collate adapters use the canonical bag dictionary throughout.

Graph models receive X from the feature bag and require adj. Configure automatic dense k-nearest-neighbor adjacency construction with:

mil:
  graph:
    enabled: true
    neighbor_space: spatial  # spatial coordinates, or feature embeddings
    k: 8
    symmetric: true
    self_loops: true

Known models that declare adj as required, including PatchGCN, trigger construction automatically even when enabled is false. Spatial graphs use tile (x, y) coordinates from the HDF5 artifact; feature graphs use rows of X. The dense adjacency has shape [B, N, N].

Benchmarking

Benchmark mode evaluates combinations from benchmark_parameters. It writes one ranked global benchmark_results.csv containing every combination, its pipeline choices, objective value, status, and checkpoint.

Run:

pathforge-benchmark --config benchmark.yaml

Minimal native benchmark:

experiment:
  project_name: native_benchmark
  annotation_file: /data/annotations.csv
  mode: benchmark
  task: classification

mil:
  backend: native
  lr: 0.0001
  weight_decay: 0.00001
  batch_size: 1
  epochs: 20

metrics:
  classification_backend: native

benchmark_parameters:
  feature_extraction: [resnet18]
  mil: [PerceiverMIL]
  loss: [CrossEntropyLoss]

For a TorchMIL benchmark, every run resolves:

1. The concrete benchmark_parameters.mil name, for example ABMIL
2. Its catalogued backend and the generic TorchMILBackendModel
3. mil.torchmil_model_kwargs, forwarded to the selected constructor
4. LightningTrainer, which accepts canonical dict batches

This keeps TorchMIL as one backend plugin. Benchmarking policies still interact with PathForge registries and trainer/model interfaces; they do not import or call TorchMIL directly.

Native, TorchMIL, and MIL-Lab names may share one model grid when all required packages are installed. Use separate config files when their shared backend constructor kwargs are incompatible.

Optimization

Optimization mode runs Optuna studies while preserving the same registry boundary as benchmarking. It writes the raw Optuna table plus a normalized, ranked global optimization_results.csv with the same core result columns as benchmarking.

Either global CSV can be visualized later without retraining:

pathforge visualize summary \
  --input /project/benchmark_results.csv \
  --output /project/benchmark_summary_visualizations

Run:

pathforge-optimize --config optimize.yaml

Example:

experiment:
  project_name: torchmil_optimization
  annotation_file: /data/annotations.csv
  mode: optimization
  task: classification

mil:
  torchmil_model_kwargs:
    in_shape: [1024]
    out_shape: 2
  batch_size: 4

optimization:
  study_name: torchmil_abmil_search
  objective_metric: val_loss
  objective_mode: min
  sampler: TPESampler
  pruner: HyperbandPruner
  trials: 50
  search_space:
    lr: {kind: float, low: 1.0e-5, high: 1.0e-3, log: true}
    epochs: {kind: int, low: 10, high: 50, step: 5}
    dropout_p: {kind: float, low: 0.0, high: 0.5}

benchmark_parameters:
  feature_extraction: [resnet18]
  mil: [ABMIL]
  loss: [CrossEntropyLoss]

Define ranges explicitly under optimization.search_space in the YAML config. Each entry uses kind: float, kind: int, or kind: categorical; numeric entries require low and high, while categorical entries require choices. The policy applies supported MIL training keys (optimizer, scheduler, batch_size, epochs, lr, weight_decay, dropout_p, bag_size, z_dim, encoder_layers, and k) and active mil, loss, and feature_extraction choices. Multi-value benchmark_parameters lists also become categorical Optuna dimensions automatically.

Concrete model names in benchmark_parameters.mil are selectable pipeline dimensions. mil.torchmil_model_kwargs remains one shared fixed mapping; the current policy does not apply dotted search-space keys or arbitrary constructor kwargs. Compare models in one config only when those kwargs are compatible, and use separate configs for different constructor layouts. Objective metrics can be native, TorchMetrics-backed, or TorchSurv-backed, selected by config.

Slide Retrieval

Slide retrieval ranks reference slides against query slides using bag-level features. It reuses existing H5 artifacts — no training is required.

Run:

pathforge-benchmark --config retrieval.yaml

Minimal config:

experiment:
  project_name: tcga_retrieval
  annotation_file: /data/annotations.csv
  mode: benchmark
  task: slide_retrieval
  aggregation_level: slide

datasets:
  - name: ReferenceSet
    slides_dir: /data/slides/reference
    artifacts_dir: /data/artifacts/reference
    used_for: reference
  - name: QuerySet
    slides_dir: /data/slides/query
    artifacts_dir: /data/artifacts/query
    used_for: query

benchmark_parameters:
  tile_px: [256]
  tile_mpp: [0.5]
  feature_extraction: [uni]
  retrieval_representation: [yottixel-features]
  search_strategy: [yottixel]

slide_retrieval:
  exclusion_level: patient

Dataset used_for roles for slide retrieval:

• reference — slides added to the search database only.
• query — slides used as queries only.
• query_reference — slides in both database and query set (leave-one-out style).

slide_retrieval.exclusion_level controls self-retrieval exclusion: none, slide, case, or patient (default). Use patient to exclude slides from the same patient when querying a shared pool.

Pre-compute representations ahead of the search step for large datasets:

pathforge-slide-retrieval-representations --config retrieval.yaml

Outputs are written to:

project_root/{project_name}/slide_retrieval/{tiling_id}/{feature}/{representation}/{search}/run_{hash}/
├── manifest.json       — run configuration and summary counts
└── query_results.xlsx  — ranked hits per query slide

Metrics

Classification metric backend:

metrics:
  classification_backend: torchmetrics

The default implementation key is torchmetrics. It is optional and resolved through the classification metrics registry. If selected but unavailable, validation raises:

Classification metrics backend requires 'torchmetrics'. Install torchmetrics or choose another classification metrics backend.

Native workflows can opt out:

metrics:
  classification_backend: native

Continuous survival backend:

metrics:
  survival_continuous_backend: torchsurv

If torchsurv is selected but unavailable, validation raises:

Continuous survival backend requires 'torchsurv'. Install torchsurv or choose another survival backend.

Survival Tasks

Continuous survival support is explicit:

experiment:
  task: survival

mil:
  batch_size: 1

benchmark_parameters:
  mil: [PerceiverMIL]
  loss: [CoxPHLoss]

metrics:
  survival_continuous_backend: torchsurv

PathForge expects continuous survival outputs to normalize to risk or log-hazard tensors shaped [B] or [B, 1]. Targets should follow the existing survival loss contract:

target = {
    "time": time,    # float tensor [B]
    "event": event,  # binary tensor [B], one = observed event, zero = censored
}

Discrete survival outputs must be shaped [B, T], where T is the number of time bins. Unsupported model/task combinations should be blocked during config or model construction rather than failing inside a training step.

Explainability And Heatmaps

The TorchMIL heatmap explainer is optional:

explainability:
  heatmap_backend: torchmil

It consumes per-instance scores plus coordinates:

payload = {
    "coords": coords,                  # tensor [N, 2]
    "instance_scores": scores,         # tensor [N]
    "mask": optional_mask,             # optional tensor [N]
}

The output is a HeatMap object containing coordinates and normalized finite scores in [0, 1]. Prediction heatmaps should be stored in a dedicated H5 prediction namespace rather than overloading existing tile overview datasets.

Inference

The inference CLI provides a stable surface for packaged-model prediction workflows. Pass the *_package.pt file written beside a successful training checkpoint, not the raw Lightning .ckpt file:

pathforge-infer-model \
  --model_path checkpoint_package.pt \
  --input /data/artifacts/SLIDE_001.h5 \
  --output predictions.json

The current implementation writes a JSON prediction payload. It can also attach an inference heatmap to a slide H5 artifact when per-instance scores are available from a backend model.

TorchMIL heatmap inference example:

pathforge-infer-model \
  --model_path /models/abmil_package.pt \
  --input /data/artifacts/SLIDE_001.h5 \
  --output /data/predictions/SLIDE_001.json \
  --heatmap-backend torchmil \
  --bag-id 256px_0.5mpp \
  --scores /data/predictions/SLIDE_001_attention.npy \
  --heatmap-name abmil_attention \
  --heatmap-output /data/predictions/SLIDE_001_heatmap.json

Inputs:

• --input: slide H5 artifact. When --coords is omitted, PathForge reads bags/{bag_id}/coords and uses the first two columns as level-0 x/y coordinates.
• --scores: .npy, .npz, or .json vector shaped [N] containing per-instance attention, attribution, or instance score values.
• --coords: optional .npy, .npz, or .json matrix shaped [N, 2]. Use this when scores do not align with H5 bag coordinates.
• --mask: optional .npy, .npz, or .json boolean/binary vector shaped [N]; false entries are removed before persistence.
• --heatmap-backend: use torchmil to resolve the torchmil_heatmap explainer through the EXPLAINERS registry.
• --heatmap-name: H5 namespace for this prediction heatmap.
• --heatmap-output: optional JSON sidecar for downstream tools that do not read H5.

Output H5 namespace:

bags/{bag_id}/predictions/heatmaps/{heatmap_name}/coords
bags/{bag_id}/predictions/heatmaps/{heatmap_name}/scores
bags/{bag_id}/predictions/heatmaps/{heatmap_name}/metadata

Persisted heatmap contracts:

• coords: floating array shaped (N, 2).
• scores: float32 array shaped (N,), finite and normalized to [0, 1].
• metadata: JSON with backend, explainer key, model path, score path, optional coordinate path, optional mask path, score range, and coordinate space.

Inference resolves the heatmap implementation through EXPLAINERS, while TorchMIL-specific behavior remains in pathforge.adapters.torchmil.heatmap_explainer.

Registries And Extensibility

PathForge uses registries as the plugin backbone:

• MODELS
• LOSSES
• TRAINERS
• TASKS
• EXPLAINERS
• FEATURE_EXTRACTORS
• SLIDE_PROCESSORS
• CLASSIFICATION_METRICS
• SURVIVAL_METRICS
• SURVIVAL_LOSSES

Register new implementations by importing a module that calls the relevant registry decorator or explicit registration function. Keep concrete package logic in adapter/infrastructure modules and expose it through PathForge interfaces.

Example native model registration:

from pathforge.core.models.mil_base import MILModelBase
from pathforge.utils.registries import MODELS


@MODELS.register("MyMIL")
class MyMIL(MILModelBase):
    ...

Optional backends should be registered conditionally through dynamic registry population so missing packages do not break imports.

Integration Boundaries

The integration is intentionally interface-first:

• Domain/core contracts remain stable: MILModelBase, TrainerBase, ExplainerBase, and the bag schema.
• Optional package guards live under pathforge.utils.optional.
• TorchMIL model/collate/output/heatmap code lives under pathforge.adapters.torchmil.
• MIL-Lab model construction and output normalization live under pathforge.adapters.mil_lab.
• TorchMetrics and TorchSurv code lives under pathforge.adapters.metrics.
• Dynamic registry population conditionally registers optional implementations only when packages are installed.
• Trainer code accepts canonical batches but does not import torchmil, MIL-Lab, torchmetrics, or torchsurv.
• Policies continue to use MODELS, LOSSES, TRAINERS, and other registries.

Architecture tests enforce that direct optional-package imports stay confined to adapter and optional-guard modules.

Troubleshooting

MIL backend 'torchmil' selected, but 'torchmil' is not installed.

: Install .[mil-backends], install torchmil, or set mil.backend: native.

MIL backend 'mil-lab' selected, but 'MIL-Lab' is not installed.

: Install MIL-Lab following its upstream instructions, or select a native or TorchMIL model.

Classification metrics backend requires 'torchmetrics'.

: Install torchmetrics or set metrics.classification_backend: native.

Continuous survival backend requires 'torchsurv'.

: Install torchsurv or choose another survival backend.

Feature extractor '<name>' is not registered.

: Ensure dynamic registries are populated before config validation. LazySlide and timm extractors are included in the default installation.

cfg.experiment.project_root must be an absolute path.

: Use an absolute path such as /data/pathforge_projects. If omitted, PathForge writes under the repository-level experiments/ directory.

No slides are found for a dataset.

: Check that annotation dataset values match datasets[].name, that slides_dir exists, and that slide filenames use {slide_id}.svs or another supported WSI suffix.

Testing

Run focused tests for the backend integration and documentation:

uv run pytest -q \
  tests/unit/test_torchmil_optional.py \
  tests/unit/test_bag_schema_collate.py \
  tests/unit/test_torchmil_task_output.py \
  tests/unit/test_lightning_batch_unpack.py \
  tests/unit/test_torchmil_architecture.py \
  tests/unit/test_torchmil_docs.py \
  tests/unit/test_config_validation.py

Run the standard repository checks before merging:

uv run ruff check . --fix
uv run ruff format .
uv run ruff check .
uv run pytest -q

For CI, use at least two profiles:

• Base profile without torchmil, MIL-Lab, torchmetrics, or torchsurv: verifies that imports, native configs, and missing-backend errors behave correctly.
• Optional-backend profile with .[mil-backends]: verifies TorchMIL construction, TorchMIL collation, TorchMetrics classification metrics, TorchSurv survival losses/metrics, and heatmap explanation.
• MIL-Lab profile with an upstream MIL-Lab checkout installed: verifies model construction and normalized output handling through the MIL-Lab adapter.

Acknowledgements and third-party backends

PathForge builds on and integrates several open-source projects. If you use a specific backend in published work, please also follow that project's citation guidance:

• LazySlide provides tissue segmentation, tiling, and feature-extraction operations used by pathforge.core.slide_processing.lazyslide.
• WSIData provides whole-slide image loading and the interoperable slide data model used by the LazySlide backend.
• timm provides image encoder models available to the feature-extraction backend.
• torchmil provides optional MIL models, collation behavior, task-output normalization, and heatmap support integrated through pathforge.adapters.torchmil.
• MIL-Lab provides optional MIL model implementations integrated through pathforge.adapters.mil_lab.
• TorchMetrics provides the optional classification metrics exposed by pathforge.adapters.metrics.
• TorchSurv provides optional survival losses and continuous-survival metrics exposed by pathforge.adapters.losses and pathforge.adapters.metrics.survival.
• PyTorch Lightning provides the training runtime used by pathforge.training.lightning.
• Optuna provides hyperparameter search and pruning for pathforge.policy.optimization.

We thank the authors and contributors of these projects. PathForge's adapters do not replace the need to cite the underlying methods and software used in an experiment.

Citation

If you use PathForge, cite the PathBench-MIL framework paper:

@misc{brussee2025pathbenchmilcomprehensiveautomlbenchmarking,
      title={PathBench-MIL: A Comprehensive AutoML and Benchmarking Framework for Multiple Instance Learning in Histopathology},
      author={Siemen Brussee and Pieter A. Valkema and Jurre A. J. Weijer and Thom Doeleman and Anne M. R. Schrader and Jesper Kers},
      year={2025},
      eprint={2512.17517},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2512.17517},
}