H. Pylori Classification Benchmarks

This repository contains benchmarks for H. Pylori detection and classification using various pathology foundation models.

Installation

Prerequistie

System install

sudo apt-get update && sudo apt-get install -y libgl1-mesa-glx libglib2.0-0

Log in to Huggingface

from huggingface_hub import login
login(token="<YOUR_HUGGINGFACE_WRITE_TOKEN>")

Clone the Repository

Since this repository points to other repositories (submodules), perform a recursive clone:

git clone --recursive git@github.com:laneegale/hp_detection.git
cd hp_detection

Environment Setup

This project uses uv

uv sync

Docker

The repository now includes a Dockerfile for the main patch-image workflows:

• get_feats.py
• downstream_logreg.py
• fine-tune.py
• draw_attn.py

This image is intended for the top-level benchmark scripts above. It does not attempt to package the separate CHIEF environment. The image includes the extra dependencies that trident currently imports at module load time so the repository can run without modifying code in that folder.

Build

docker build -t hp-detection .

Run with GPU

If the host machine has the NVIDIA Container Toolkit installed:

docker run --rm -it \
	--gpus all \
	-e HF_TOKEN=<your_huggingface_token> \
	-v /path/to/data:/data \
	-v /path/to/output:/output \
	hp-detection \
	python get_feats.py virchow2 /data /output/features --device cuda

Run on CPU

docker run --rm -it \
	-e HF_TOKEN=<your_huggingface_token> \
	-v /path/to/data:/data \
	-v /path/to/output:/output \
	hp-detection \
	python get_feats.py virchow2 /data /output/features --device cpu

Additional examples

docker run --rm -it \
	--gpus all \
	-e HF_TOKEN=<your_huggingface_token> \
	-v /path/to/data:/data \
	-v /path/to/output:/output \
	hp-detection \
	python fine-tune.py /data/train /output/run1 --model virchow2 --device cuda

docker run --rm -it \
	-v /path/to/features:/features \
	-v /path/to/output:/output \
	hp-detection \
	python downstream_logreg.py virchow2 /features /output

docker run --rm -it \
	-v /path/to/images:/images \
	-v /path/to/output:/output \
	hp-detection \
	python draw_attn.py --folder /images --output /output/attn --model virchow2

Notes:

• Most foundation models download weights from Hugging Face on first use, so the container needs network access unless weights are already cached.
• get_feats.py now accepts --device, --batch-size, and --num-workers, which makes it easier to tune for different machines.
• If a colleague only needs an interactive shell inside the container, run docker run --rm -it hp-detection bash.

---

Feature Extraction

Extract features from pathology images using various foundation models.

!!! I have used qa.py to remove images that are mostly background, which removed around ~4000 images, leaving only ~7000 images for testing.

Standard Models

Usage:

python get_feats.py <models> <data_dir> <save_path>

Arguments:

• <models>: Comma-separated list of models to use (e.g., virchow2,hoptimus0,musk).
• <data_dir>: Path to the directory containing input images (can be nested).
• <save_path>: Path where the extracted .h5 features will be saved.

Supported Models:

• virchow2
• ctranspath
• hoptimus0
• hoptimus1
• uni_v2
• musk
• conch_v15
• hibou-l (Transformers: histai/hibou-L)

CHIEF Model

Note: The CHIEF model requires a specific environment setup. Please refer to the CHIEF Repository for installation instructions.

Usage: Once the CHIEF environment is active:

python get_CHIEF_feats.py <data_dir> <save_path>

---

Classification

Train and evaluate a logistic regression classifier on the extracted features. This implementation is adapted from the UNI repository.

Usage:

python downstream_logreg.py <models> <h5_dir> <save_path>

Arguments:

• <models>: Comma-separated list of models to classify (e.g., virchow2,hoptimus0,musk).
• <h5_dir>: Directory containing the .h5 features extracted in the previous step.
• <save_path>: Location to save the results (saved as a pickle file).

Partial Fine-Tuning For Attention Maps

Fine-tune only the last transformer blocks of a patch encoder on a binary image dataset laid out as:

DATA_DIR/
	positive/
	negative/

Usage:

python fine-tune.py DATA_DIR OUTPUT_DIR --model virchow2 --unfreeze-last-n-blocks 2 --epochs 5

This saves:

• OUTPUT_DIR/<model>_backbone_best.pt: fine-tuned backbone weights for attention-map rendering.
• OUTPUT_DIR/<model>_finetune.pt: full training checkpoint including classifier head and metrics.

To render attention masks with the fine-tuned backbone:

python draw_attn.py --folder DATA_DIR/positive --output results/attn --model virchow2 --weights-path OUTPUT_DIR/virchow2_backbone_best.pt

---

Visualization (Attention Masks)

Generate attention masks to visualize model focus on specific images or folders.

Usage:

python draw_attn.py [-i IMAGE] [-f FOLDER] -o OUTPUT -m MODEL

Arguments:

*Note: You must provide either an image (-i) or a folder (-f).

Supported Models for Visualization:

• conch_v15
• virchow2

Example

python draw_attn.py --image data/slide_1.tif --output results/attn_masks/ --model virchow2