VELCRO

This repository contains the code for the work "Visual Medical Entity Linking with Contrastive Region Alignment". VELCRO is a deep learning model designed for user-guided medical visual entity linking (VEL).

Getting Started

To get started, make sure Python3 is installed. Then, run the following code:

# Clone the repository
git clone https://github.com/carbonkat/VELCRO/tree/main

# Create a virtual environment
python -m venv medical_venv

# Install dependencies
pip install -r v1/src/requirements.txt

Navigation

Model, data, and experiment configurations are located in the v1/src/configs folder. Files pertaining to dataset creation and storage are located in v1/src/data, files pertaining to model architecture and instantiation are located in v1/src/components, and files pertaining to general Hydra configuration instantiation are located in v1/src/utils. For most use cases, configuration should be constrained to the configs folder.

This repository relies heavily on PyTorch Lightning, a deep learning framework that abstracts boilerplate experimentation code. Please see v1/src/model.py, v1/src/trainer.py, and v1/src/train.py for training loop hooks.

To run experiments using SLURM, see v1/src/experiment_script.sh and v1/src/load_checkpoint.sh for example usage.