In addition to the installation, this repository requires setting up three additional paths:

1. nnssl_raw -- The path holding datasets of raw pretrain_data.json files.
2. nnssl_preprocessed -- A path where preprocessed data will be stored.
3. nnssl_results -- A path where results will be stored.

More details can be found here.

Medical datasets generally center around studies of subjects. These subjects can be imaged in different sessions with different scanners or through different imaging protocols. This is reflected in the common BIDS data structure, which differentiates into:

• subjects - The individual subjects in the dataset
• sessions - The individual sessions of the subjects
• scans - The individual scans of the sessions

In our case, we are also interested in aggregating multiple datasets, hence we include

• dataset - The individual datasets that was included

All this information may be valuable for pre-training, e.g. one may want to develop a contrastive pre-training method that uses scans of the same subject during one session as positive pair and others as negative. Or one may want to develop a longitudinal pre-training that e.g. tries to predict the next scan of the next session. To allow using such information, we need to maintain this information in the pretrain_data.json file.

{  // Examplary Structure
  "collection_index": 745,    // Collection Index -- 
  "collection_name": "Dataset745__OpenNeuro",     // Collection Index -- 
  "datasets": {     // Dict of all datasets included
    "ds000001": {
      "dataset_index": "ds000001",
      "dataset_info": null,   // dict holding meta info of the dataset
      "subjects": {    // dict of all subjects
        "sub-01": {
          "subject_id": "sub-01",
          "subject_info": {
            "age": 26,
            "sex": "female",
          },
          "sessions": {     // Dict of all session and session information
            "ses-DEFAULT": {
                "session_id": "ses-DEFAULT",
                "session_info": null,
                "images": [   // list of images -- Each image is it's own dictionary
                    {
                        "name": "sub-01_T1w.nii.gz",   // Image Name
                        "modality": "T1w",  // Modality of the Image
                        "image_info": {},  // Additional meta-data of the image (e.g. Scanner, etc.)
                        "image_path": "<Path_to_image>",
                        "associated_masks": {  // Associated masks of the image (if available)
                            "anatomy_mask": "<Path_to_associated_anatomy_mask>",
                            "anonymization_mask": "<Path_to_associated_anon_mask>",
                        },
                    },
                    ... // Additional images if taken in the session.
                ],
            }
          }
        }
      }
    }
  }
}

To generate this file, we recommend writing a python script that creates a Collection dataclass (located in src/nnssl/data/raw_dataset.py) and uses the .to_dict() method of the collection which will yield a valid pretrain_data.json file.

To allow this file to be valid for differing machines, the file-paths support relative paths. Relative paths are indicated through the pre-fix $. Moreover, when saving absolute paths the paths are checked, if the image path beginnings can be replaced by the paths in the Environment Variables: ["nnssl_raw", "nnssl_preprocessed"], replacing them with $nnssl_raw or $nnssl_preprocessed respectively.

@misc{harsy2026nnfoundation3dfoundationmodels,
      title={nnFoundation: 3D Foundation Models for Radiology}, 
      author={Constantin Ulrich Harsy and Tassilo Wald and Karol Gotkowski and Yannick Kirchhoff and Marcel Knopp and Maximilian Rokuss and Elisa Stegmeier and Philipp Schader and Dasha Trofimova and Raphael Stock and Kim-Celine Kahl and Stephen Schaumann and Selen Erkan and David Zimmerer and Stefan Denner and Moritz Langenberg and Sebastian Ziegler and Katharina Eckstein and Maximilian Fischer and Jonathan Suprijadi and Bálint Kovács and Benjamin Hamm and Anand Deshpande and Dimitrios Bounias and Nico Disch and Shuhan Xiao and Jessica Kächele and Jan Sellner and Rajesh Baidya and Jeremias Traub and Lars Krämer and Maximilian Zenk and Tim Rädsch and Stefan Dvoretskii and Robin Peretzke and Jonathan Deissler and Alexandra Ertl and Partha Ghosh and Kris Dreher and Stefan Dinkelacker and Annika Reinke and Evangelia Christodoulou and Numan Saeed and Yoland Savriama and Santiago Estrada and David Kügler and Laura Alexandra Daza Barragan and Cristina Isabel Gonzalez Osorio and Jan Peeken and Michael Baumgartner and Marvin Teichmann and Guillaume Chabin and Matthias Kirchler and Valentin Koch and for the ALFA study and Markus Hohenhaus and Dimitri Koslov and Nina Decker and Mohammad Yaqub and Arnd Heuser and Martin Reuter and Julia A. Schnabel and Tobias Heimann and Florin Ghesu and Paul Brachmann and Claus P. Heußel and Alexander Radbruch and Gianluca Brugnara and Aditya Rastogi and Martha Foltyn-Dumitru and Heinz-Peter Schlemmer and Ignaz Reicht and Julius C. Holzschuh and Michael Bach and Bram Stieltjes and Kai Schlamp and Lena Maier-Hein and Marco Nolden and Ralf Floca and Paul F. Jäger and Philipp Vollmuth and Fabian Isensee and Klaus H. Maier-Hein},
      year={2026},
      eprint={2609.26924},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2609.26924}, 
}

@InProceedings{Wald_2025_ICCV,
    author    = {Wald, Tassilo and Ulrich, Constantin and Suprijadi, Jonathan and Ziegler, Sebastian and Nohel, Michal and Peretzke, Robin and Kohler, Gregor and Maier-Hein, Klaus},
    title     = {An OpenMind for 3D Medical Vision Self-supervised Learning},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    month     = {October},
    year      = {2025},
    pages     = {23839-23879}
}