foveate

Research question: can saccadic/foveated sampling improve whole-slide pathology accuracy per unit compute compared with uniform patch sampling?

Phase 1 implements lazy WSI access, tissue masking, uniform grid sampling, reproducible patch budgets, JSON manifests, and a PyTorch/MONAI-compatible dataset. Phase 2 adds a uniform slide-classification baseline with configurable encoders, mean pooling, metrics, checkpointing, and compute accounting. Learned saccades are not implemented. Local development is CPU-only; GPU training will run remotely.

Structure

• src/foveate/data/: readers, coordinates, masks, sampler, manifests, dataset
• src/foveate/{models,training,evaluation}/: classifier, training, metrics, accounting
• src/foveate/saccade/: reserved for the future sampling policy
• scripts/: environment check, slide inspection, metadata, sampling, smoke training
• tests/: synthetic/mock unittest suite; no pathology dataset required
• docs/phase1_uniform_baseline.md: coordinate contract and research design
• data/raw/, data/processed/: local data locations; raw data remains immutable
• CODEX_PROGRESS.md, CODEX_RUN_SUMMARY.md: recovery log and implementation report

Commands (PowerShell, repository root)

Activate the existing Python 3.11 environment:

conda activate foveate
& 'C:\Users\megaa\miniconda3\envs\foveate\python.exe' scripts/check_environment.py

Phase 4D plotting uses Matplotlib 3.11.1, already installed locally and pinned in requirements-plotting.txt. When preparing the plotting dependency in another existing project environment, use its Python interpreter to install that file. The local interpreter command is:

& 'C:\Users\megaa\miniconda3\envs\foveate\python.exe' -m pip install -r requirements-plotting.txt

This supplements the existing Conda environment; it is not a complete environment lock. On PACE, use the Python interpreter of the configured remote environment.

Inspect an existing, OpenSlide-supported WSI:

& 'C:\Users\megaa\miniconda3\envs\foveate\python.exe' scripts/inspect_slide.py 'data/raw/slide.svs'

Create an unfiltered uniform baseline with an explicit per-slide budget:

& 'C:\Users\megaa\miniconda3\envs\foveate\python.exe' scripts/sample_uniform.py 'data/raw/slide.svs' --output outputs/uniform.json --level 0 --patch-size 256 256 --stride 256 256 --max-patches 128 --seed 42

Add --min-tissue-fraction 0.5 to filter candidates before budgeting, or --measure-tissue to record fractions without filtering. Omit --max-patches for all eligible patches. Both scripts support --help. Tissue masking requires a pyramid level fitting --mask-max-size (default 2048); oversized overviews fail explicitly. Existing manifests require --overwrite to replace.

Run all CPU tests, and enable library imports for your own scripts:

$env:PYTHONPATH = 'src'
& 'C:\Users\megaa\miniconda3\envs\foveate\python.exe' -m unittest discover -v

See the Phase 1 design for dataset usage, coordinate conventions, reproducibility and fair compute comparisons. Real WSI validation remains necessary before research experiments. See the Phase 2 classifier design for the metadata, model, metrics, budgets, and future policy interface. Phase 3 prepares the frozen Virchow2-compatible encoder contract, physical scale provenance, and embedding cache; see the Phase 3 foundation-encoder design. Phase 4A prepares the complete CAMELYON16 cohort inventory, integrity audit, resumable download workflow, and explicit training-only splits; see the Phase 4A cohort design. Phase 4B defines physical-scale uniform manifests for the primary 0.5 um/pixel experiment; see the Phase 4B manifest design. Phase 4C connects exact physical observations to pinned Virchow2, a resumable observation cache, and one-slide-per-task GPU orchestration; see the Phase 4C extraction workflow. Use scripts/extract_virchow_embeddings.py for Phase 4B manifests; the older extract_embeddings.py and example configs target earlier smoke workflows. Phase 4D freezes the training-only research split and evaluates cached uniform representations with a linear classifier, separate sampling/training seeds, provenance-checked resume, and explicit compute attribution; see the Phase 4D uniform baseline. Its production CLI is scripts/run_uniform_baseline.py; local structural validation uses scripts/validate_phase4d_local.py and the existing tiny caches.

For bounded, one-WSI-at-a-time extraction on Kaggle or Colab, see free-cloud execution and the thin launchers under notebooks/cloud/. They reuse Phase 4B/4C and require NVIDIA acceptance before CUDA streaming; cloud validation remains pending.

Resources

Data

CAMELYON16 is Foveate's current primary dataset. CAMELYON17 is not part of the primary experiment; its page is included as a related challenge/data resource.

• CAMELYON Open Data Registry
• CAMELYON17 / CAMELYON challenge data page

Models

• Virchow2: the frozen pathology foundation encoder used in the current Foveate pipeline.

MONAI / NVIDIA pathology resources

These provide medical-imaging and pathology implementation context; inclusion does not imply that their code was copied directly into Foveate.

• Project MONAI tutorials
• NVIDIA MONAI pathology tumor-detection resource

Related Work and References

• Lan et al. (2020). SaccadeNet: A Fast and Accurate Object Detector. Its learned selection of informative object keypoints provides context for saccadic/selective visual processing in general object detection, outside pathology.
• Cruz-Roa et al. (2018). High-throughput adaptive sampling for whole-slide histopathology image analysis (HASHI) via convolutional neural networks: Application to invasive breast cancer detection. Adaptive WSI sampling for cancer detection is relevant to Foveate's comparison of sampling strategies under a compute budget.
• Thandiackal et al. (2022). Differentiable Zooming for Multiple Instance Learning on Whole-Slide Images (ZoomMIL). This WSI multiple-instance-learning method uses multi-level zooming for efficient classification, making it conceptually relevant to selective visual sampling.
• Dimitriou et al. (2024). Magnifying Networks for Histopathological Images with Billions of Pixels. Attention-driven coarse-to-fine region analysis offers a related approach to reducing the image information processed for slide classification.

Acknowledgments / Built With

Foveate builds on these software projects, listed separately from scientific references:

• PyTorch
• MONAI
• OpenSlide
• timm
• Hugging Face Hub