Yunkun-Zhang/Data-Centric-FM-Healthcare

A survey on data-centric foundation models in healthcare.

28 commits

updated Feb 14, 2026

See the code

README

Data-Centric Foundation Models in Computational Healthcare

:fire::fire::fire: A survey on data-centric foundation models in computational healthcare

Project Page | Paper [arXiv]

Last updated: 2026/02/05

:pencil: If you find this repo helps, please kindly cite our survey, thanks!

@article{zhang2024data,
  title={Data-Centric Foundation Models in Computational Healthcare: A Survey},
  author={Zhang, Yunkun and Gao, Jin and Tan, Zheling and Zhou, Lingfeng and Ding, Kexin and Zhou, Mu and Zhang, Shaoting and Wang, Dequan},
  journal={arXiv},
  year={2024},
  eprint={2401.02458},
  archivePrefix={arXiv},
  primaryClass={cs.LG},
  doi={10.48550/arXiv.2401.02458},
  url={https://arxiv.org/abs/2401.02458}
}

In this repository, we provide an up-to-date list of healthcare-related foundation models and datasets, which are also mentioned in our survey paper.

:book: Contents

Healthcare and medical foundation models
Datasets for foundation model
- Text
- Imaging
- Genomics
- Drug
- Multi-modal

Healthcare and Medical Foundation Models

A star (*) after the pre-training data shows that the authors constructed the data with more than three sources.

Language Models

Model	Subfield	Paper	Code	Base	Pre-Training Data
Baichuan-M2	Medicine	Baichuan-M2: Scaling Medical Capability with Large Verifier System	Github	Qwen2.5	*
Baichuan-M1	Medicine	Baichuan-M1: Pushing the Medical Capability of Large Language Models	-	Transformer	20T tokens*
EHRMamba	Clinic	EHRMamba: Towards Generalizable and Scalable Foundation Models for Electronic Health Records	Github	Mamba	MIMIC-IV
MMedLM 2	Medicine	Towards building multilingual language model for medicine	Github	InternLM 2	MMedC*
BiMediX	Medicine	BiMediX: Bilingual Medical Mixture of Experts LLM	Github	Mixtral	BiMed1.3M*
Me LLaMA	Medicine	Me LLaMA: Foundation Large Language Models for Medical Applications	Github	LLaMA 2	*
BioMistral	Biomedicine	BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domains	-	Mistral	PubMed Central
PULSE	Medicine	-	Github	InternLM	*
Meditron	Medicine	MEDITRON-70B: Scaling Medical Pretraining for Large Language Models	Github	LLaMA 2	GAP-Replay*
Taiyi	Biomedicine	Taiyi: A Bilingual Fine-Tuned Large Language Model for Diverse Biomedical Tasks	Github	Qwen-7B / GLM4-9B	BigBio + CBLUE
BioMedGPT	Biomedicine	BioMedGPT: An Open Multimodal Large Language Model for BioMedicine	Github	LLaMA 2	S2ORC
Clinical LLaMA-LoRA	Clinic	Parameter-Efficient Fine-Tuning of LLaMA for the Clinical Domain	-	LLaMA	MIMIC-IV
Med-PaLM 2	Clinic	Toward expert-level medical question answering with large language models	Google	PaLM 2	MedQA
PMC-LLaMA	Medicine	PMC-LLaMA: toward building open-source language models for medicine	Github	LLaMA	MedC
MedAlpaca	Medicine	MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Data	Github	LLaMA	Medical Meadow
BenTsao (HuaTuo)	Biomedicine	HuaTuo: Tuning LLaMA Model with Chinese Medical Knowledge	Github	LLaMA	CMeKG
ChatDoctor	Medicine	ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledge	Github	LLaMA	HealthCareMagic*
Clinical-T5	Clinic	Clinical-T5: Large Language Models Built Using Mimic Clinical Text	PhysioNet	T5	MIMIC-III + MIMIC-IV
Med-PaLM	Clinic	Large Language Models Encode Clinical Knowledge	Google	PaLM	MedQA
BioGPT	Biomedicine	BioGPT: Generative Pre-Trained Transformer for Biomedical Text Generation and Mining	Github	GPT-2	PubMed
BioLinkBERT	Biomedicine	LinkBERT: Pretraining Language Models with Document Links	Github	BERT	PubMed (citation links)
PubMedBERT	Biomedicine	Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing	Microsoft	BERT	PubMed
BioBERT	Biomedicine	BioBERT: A Pre-Trained Biomedical Language Representation Model for Biomedical Text Mining	Github	BERT	PubMed + PMC
BlueBERT	Biomedicine	An Empirical Study of Multi-Task Learning on BERT for Biomedical Text Mining	Github	BERT	PubMed + MIMIC-III
Clinical BERT	Clinic	Publicly Available Clinical BERT Embeddings	Github	BERT	MIMIC-III
SciBERT	Biomedicine	SciBERT: A Pretrained Language Model for Scientific Text	Github	BERT	Semantic Scholar

Vision Models

Model	Subfield	Paper	Code	Base	Pre-Training Data
FastGlioma	Pathology	Foundation models for fast, label-free detection of glioma infiltration	-	-	Label-free optical microscopy (4M images)*
MedLSAM	Radiology	MedLSAM: Localize and Segment Anything Model for 3D CT Images	Github	SAM	*
BiomedParse	Biomedicine	A Foundation Model for Joint Segmentation, Detection and Recognition of Biomedical Objects across Nine Modalities	Github	SEEM	BiomedParseData*
Universal Model	Radiology	Universal and Extensible Language-Vision Models for Organ Segmentation and Tumor Detection from Abdominal Computed Tomography	Github	-	*
CHIEF	Pathology	A pathology foundation model for cancer diagnosis and prognosis prediction	Github	CTransPath	*
USFM	Sonography	USFM: A Universal Ultrasound Foundation Model Generalized to Tasks and Organs towards Label Efficient Image Analysis	-	MIM	3M-US*
BrainSegFounder	Radiology	BrainSegFounder: towards 3D foundation models for neuroimage segmentation	Github	SwinUNETR	UK Biobank + BraTS + ATLAS
MedSAM	Medicine	Segment Anything in Medical Images	Github	SAM	*
Prov-GigaPath	Pathology	A Whole-Slide Foundation Model for Digital Pathology from Real-World Data	Github	-	Prov-Path*
BEPH	Pathology	A Foundation Model for Generalizable Cancer Diagnosis and Survival Prediction from Histopathological Images	Github	BEiTv2	*
Pai et al.	Radiology	Foundation Model for Cancer Imaging Biomarkers	Github	SimCLR	*
VIS-MAE	Radiology	VIS-MAE: An Efficient Self-supervised Learning Approach on Medical Image Segmentation and Classification	-	MAE	*
SegmentAnyBone	Radiology	SegmentAnyBone: A Universal Model that Segments Any Bone at Any Location on MRI	Github	SAM	*
RudolfV	Pathology	RudolfV: A Foundation Model by Pathologists for Pathologists	-	DINOv2	*
PathoDuet	Pathology	PathoDuet: Foundation Models for Pathological Slide Analysis of H&E and IHC Stains	Github	MoCo v3	TCGA + HyReCo + BCI
UNI	Pathology	Towards a general-purpose foundation model for computational pathology	-	DINOv2	Mass-100K
REMEDIS	Radiology	Robust and Data-Efficient Generalization of Self-Supervised Machine Learning for Diagnostic Imaging	Github	SimCLR	MIMIC-IV + CheXpert
Virchow	Pathology	A foundation model for clinical-grade computational pathology and rare cancers detection	-	DINOv2	*
RETFound	Retinopathy	A Foundation Model for Generalizable Disease Detection from Retinal Images	Github	MAE	*
CTransPath	Pathology	Transformer-Based Unsupervised Contrastive Learning for Histopathological Image Classification	Github	-	TCGA + PAIP
HIPT	Pathology	Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning	Github	DINO	TCGA

Vision-Language Models

Model	Subfield	Paper	Code	Base	Pre-Training Data
TITAN	Pathology	A multimodal whole-slide foundation model for pathology	-	CONCH	335k WSIs + reports + synthetic captions*
DentVLM	Dentistry	DentVLM: A Multimodal Vision-Language Model for Comprehensive Dental Diagnosis and Enhanced Clinical Practice	-	Qwen2-VL	2.4M oral images + 88.6K dental VQA*
Lingshu	Medicine	Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning	Project	Qwen2.5-VL	*
MUSK	Pathology	A Vision–Language Foundation Model for Precision Oncology	Github	BEiT	*
MONET	Dermatology	Transparent Medical Image AI via an Image–Text Foundation Model Grounded in Medical Literature	Github	CLIP	PubMed + textbooks
Uni-Med	Medicine	Uni-Med: A Unified Medical Generalist Foundation Model For Multi-Task Learning Via Connector-MoE	-	CLIP + LLaMA 2	*
MaCo	Radiology	Enhancing Representation in Radiography-Reports Foundation Model: A Granular Alignment Algorithm Using Masked Contrastive Learning	Github	CLIP + MAE	MIMIC-CXR
RadFound	Radiology	Expert-Level Vision-Language Foundation Model for Real-World Radiology and Comprehensive Evaluation	-	-	RadVLCorpus*
BiomedGPT	Biomedicine	A Generalist Vision–Language Foundation Model for Diverse Biomedical Tasks	Github	-	*
PRISM	Pathology	PRISM: A Multi-Modal Generative Foundation Model for Slide-Level Histopathology	-	CoCa	*
Med-Gemini	Medicine	Capabilities of Gemini Models in Medicine	-	Gemini	*
EchoCLIP	Sonography	Vision-Language Foundation Model for Echocardiogram Interpretation	Github	CLIP	*
Med-PaLM M	Biomedicine	Towards Generalist Biomedical AI	-	PaLM	MultiMedBench*
ChemDFM	Chemistry	Developing ChemDFM as a large language foundation model for chemistry	-	LLaMA	PubMed + USPTO
CheXagent	Radiology	A Vision-Language Foundation Model to Enhance Efficiency of Chest X-ray Interpretation	Github	BLIP-2	CheXinstruct*
SAT	Radiology	Large-vocabulary segmentation for medical images with text prompts	Github	-	SAT-DS*
PathChat	Pathology	Vision–language AI assistance in human pathology	-	LLaVA	PathChatInstruct*
Qilin-Med-VL	Radiology	Qilin-Med-VL: Towards Chinese Large Vision-Language Model for General Healthcare	Github	LLaVA	Chi-Med-VL*
CXR-CLIP	Radiology	CXR-CLIP: Toward Large Scale Chest X-ray Language-Image Pre-training	Github	CLIP	MIMIC-CXR + CheXpert + ChestX-ray14
PathLDM	Pathology	PathLDM: Text conditioned Latent Diffusion Model for Histopathology	Github	Latent Diffusion	TCGA-BRCA + GPT-3.5
RadFM	Radiology	Towards generalist foundation model for radiology by leveraging web-scale 2D&3D medical data	Github	-	MedMD*
KAD	Radiology	Knowledge-Enhanced Visual-Language Pre-Training on Chest Radiology Images	Github	CLIP	MIMIC-CXR + UMLS
Med-Flamingo	Medicine	Med-Flamingo: A Multimodal Medical Few-Shot Learner	Github	Flamingo	MTB + PMC-OA
CONCH	Pathology	A Visual-Language Foundation Model for Computational Pathology	Github	CoCa	PubMed + PMC
QuiltNet	Pathology	Quilt-1M: One Million Image-Text Pairs for Histopathology	Github	CLIP	Quilt-1M*
PathAsst	Pathology	PathAsst: A Generative Foundation AI Assistant towards Artificial General Intelligence of Pathology	Github	CLIP	PathCap + PathInstruct*
PLIP	Pathology	A Visual-Language Foundation Model for Pathology Image Analysis Using Medical Twitter	Huggingface	CLIP	OpenPath*
MI-Zero	Pathology	Visual Language Pretrained Multiple Instance Zero-Shot Transfer for Histopathology Images	Github	CLIP	ARCH
LLaVA-Med	Biomedicine	LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day	Github	LLaVA	PMC-15M + GPT-4
MedVInT	Biomedicine	PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answering	Github	-	PMC-VQA*
PMC-CLIP	Biomedicine	PMC-CLIP: Contrastive Language-Image Pre-Training Using Biomedical Documents	Github	CLIP	PMC-OA*
BiomedCLIP	Biomedicine	A Multimodal Biomedical Foundation Model Trained from Fifteen Million Image–Text Pairs	Huggingface	CLIP	PMC-15M*
MedKLIP	Radiology	MedKLIP: Medical Knowledge Enhanced Language-Image Pre-Training for X-ray Diagnosis	Github	CLIP	MIMIC-CXR
MedCLIP	Medicine	MedCLIP: Contrastive Learning from Unpaired Medical Images and Text	Github	CLIP	CheXpert + MIMIC-CXR
CheXzero	Radiology	Expert-Level Detection of Pathologies from Unannotated Chest X-ray Images via Self-Supervised Learning	Github	CLIP	MIMIC-CXR
PubMedCLIP	Radiology	PubMedCLIP: How Much Does CLIP Benefit Visual Question Answering in the Medical Domain?	Github	CLIP	ROCO

Protein and Molecule Models

Model	Subfield	Paper	Code	Base	Pre-Training Data
nach0	Molecules	nach0: Multimodal Natural and Chemical Languages Foundation Model	Github	T5	*
MoleculeSTM	Drug	Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editing	Github	CLIP	PubChem
AlphaMissense	Proteomics	Accurate Proteome-Wide Missense Variant Effect Prediction with AlphaMissense	Github	AlphaFold	PDB + UniRef
GET	Genomics	A foundation model of transcription across human cell types	Huggingface	Transformer	*
GIT-Mol	Molecules	GIT-Mol: A Multi-Modal Large Language Model for Molecular Science with Graph, Image, and Text	Github	T5 + BLIP-2	PubChem
ESM-2	Proteomics	Evolutionary-Scale Prediction of Atomic-Level Protein Structure with a Language Model	Github	Transformer	UniRef
AlphaFold 2	Proteomics	Highly Accurate Protein Structure Prediction with AlphaFold	Github	-	PDB + Uniclust30

Other Models

Model	Subfield	Paper	Code	Base	Pre-Training Data
OmniNA	Nucleotide sequence	OmniNA: A Foundation Model for Nucleotide Sequences	-	LLaMA	NCBI
LaBraM	EEG	Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCI	-	Transformer	*
Neuro-GPT	EEG	Neuro-GPT: Towards A Foundation Model For EEG	-	-	TUH EEG

Datasets for Foundation Model

Text

Dataset (Paper)	Description	Link
MedBench (DOI)	A Chinese medical LLM benchmark with 300,901 Chinese questions covering 43 clinical specialties, combined with an automatic evaluation system	Official site
MMedBench (DOI)	A multilingual medical QA benchmark, where questions are categorized into 21 topics	Github
MMedC (DOI)	A multilingual medical corpus containing over 25.5B tokens	Github
BiMed1.3M (DOI)	An English and Arabic bilingual dataset of 1.3M samples of medical QA and chat	Github
GAP-Replay (arXiv)	48.1B tokens from 4 medical corpora including guidelines, abstracts, papers, and replay	Github
Huatuo-26M (DOI)	26M Chinese medical QA pairs	Github
Medical Meadow (arXiv)	16M medical QA pairs collected from 9 sources	Github
MultiMedQA (Nature)	6 existing and 1 online-collected medical QA dataset	Nature
BigBio (NeurIPS)	126+ biomedical NLP datasets covering 13 task categories and 10+ languages	Github
MedMCQA (MLR)	194K multiple-choice questions covering 2.4K healthcare topics	Official site
MedQA-USMLE (MDPI)	61,097 multiple choice questions based on USMLE in three languages	Github
CBLUE (DOI)	A Chinese biomedical language understanding evaluation benchmark with 18 datasets	Official site
BLURB (DOI)	13 biomedical NLP datasets in 6 tasks	Official site
PubMedQA (DOI)	1K expert-annotated, 61.2K unlabeled, and 211.3K artificially generated biomedical QA instances	Official site
BLUE (DOI)	5 language tasks with 10 biomedical and clinical text datasets	Github
webMedQA (BMC)	63,284 real-world Chinese medical questions with over 300K answers	Github
MedMentions (DOI)	4,392 papers annotated by experts with mentions of UMLS entities	Github
MIMIC-III (Nature)	Critical care data for over 40,000 patients	Official site
ClinicalTrials.gov	An online database of clinical research studies, including clinical trials and observational studies	Official site

Imaging

Dataset (Paper)	Description	Link
3M-US	2,187,915 ultrasound images of 12 common organs	-
AbdomenAtlas (arXiv)	20,460 3D CT volumes from 112 hospitals, with 673K masks of anatomical structures	Github
BiomedParseData (Nature)	1.1M images, 3.4M image-mask-label triples, and 6.8M image-mask-description triples	Github
Mass-100K (DOI)	100M tissue patches from 100,426 diagnostic H&E WSIs across 20 major tissue types	-
RETFound (Nature)	Unannotated retinal images, containing 904,170 CFPs and 736,442 OCT scans	Nature
AbdomenAtlas-8K (NeurIPS)	8,448 CT volumes with per-voxel annotated eight abdominal organs	Github
Med-MNIST v2 (Nature)	12 2D and 6 3D datasets for biomedical image classification	Official site
EchoNet-Dynamic (DOI)	10,030 expert-annotated echocardiogram videos	Official site
CheXpert (DOI)	224,316 chest radiographs of 65,240 patients	Official site
Kather Colon Dataset (PMC)	100K histological images of human colorectal cancer and healthy tissue	Zenodo
DeepLesion (PMC)	32K CT scans with annotations and semantic labels from radiological reports	NIH
ChestXray-NIHCC (DOI)	100K radiographs with labels from more than 30,000 patients	NIH
ISIC	An archive containing 23K skin lesion images with labels & Imaging	Official site

Genomics

Dataset (Paper)	Description	Link
1000 Genomes Project (Nature)	A comprehensive catalog of human genetic variations	Official site
ENCODE (Nature)	A platform of genomics data and encyclopedia with integrative-level and ground-level annotations	NIH
dbSNP (NIH)	A collection of human single nucleotide variations, microsatellites, and small-scale insertions and deletions	NIH

Drug

Dataset (Paper)	Description	Link
DrugChat (arXiv)	143,517 question-answer pairs covering 10,834 drug compounds, collected from PubChem and ChEMBL	-
PubChem (NIH)	A collection of 900+ sources of chemical information data	NIH
DrugBank (NIH)	A web-enabled structured database of molecular information about drugs	Official site
ChEMBL (NIH)	20M bioactivity measurements for 2.4M distinct compounds and 15K protein targets	Official site

Dataset (Paper)	Description	Link
MultiMedBench (NEJM AI)	A multi-modal benchmark comprising 12 data sources and 14 tasks	-
RadGenome-Chest CT (arXiv)	A dataset of 3D chest CT, including 197 organ-level segmentation masks, 665K multi-granularity grounded reports, and 1.3M grounded VQA pairs	-
OmniMedVQA (DOI)	131,813 question-answering items with 120,530 images from 12 modalities and 26 human anatomical regions, collected from 75 medical datasets	-
SAT-DS (DOI)	11,462 scans with 142,254 segmentation annotations spanning 8 human body regions from 31 medical image segmentation datasets, together with domain knowledge from e-Anatomy and UMLS	Github
PathChatInstruct (DOI)	250K+ diverse disease-agnostic visual-language instructions with image and text	-
Chi-Med-VL (arXiv)	580,014 image-text pairs and 469,441 question-answer pairs for general healthcare in Chinese	Github
MedMD (DOI)	15.5M 2D scans and 180k 3D radiology scans with textual descriptions	Github
OpenPath (Nature)	208,414 pathology images paired with natural language descriptions	Huggingface
Quilt-1M (NeurIPS)	1M image-text pairs for histopathology	Github
Med-MMHL (arXiv)	Human- and LLM-generated misinformation detection dataset	Github
Mol-Instructions (OpenReview)	148K molecule-oriented, 505K protein-oriented, and biomolecular text instructions	Huggingface
PathInstruct (DOI)	180K samples of LLM-generated instruction-following data	Github
PMC-VQA (arXiv)	227K VQA pairs of 149K images of various modalities or diseases	Github
PMC-OA (DOI)	1.6M fine-grained biomedical image-text pairs	Github
PathCap (DOI)	142K pathology image-caption pairs from various sources	Github
SwissProtCLAP (DOI)	441K text-protein sequence pairs	Github
MIMIC-IV (Nature)	Clinical information for hospital stays of over 60,000 patients	Official site
MIMIC-CXR (Nature)	227,835 chest imaging studies with free-text reports for 65,379 patients	PhysioNet
TCGA	A landmark cancer genomics program, molecularly characterized over 20,000 primary cancer and matched normal samples spanning 33 cancer types	Official site

Contributors

Yunkun-Zhang

26 commits

shiyegao

2 commits

Similar repos

openmedlab/Data-Centric-FM-Healthcare

YutingHe-list/Awesome-Foundation-Models-for-Advancing-Healthcare

We present a comprehensive and deep review of the HFM in challenges, opportunities, and future…

258

Jianing-Qiu/Awesome-Healthcare-Foundation-Models

525

xmindflow/Awesome-Foundation-Models-in-Medical-Imaging

A curated list of foundation models for vision and language tasks in medical imaging

305

AI-in-Health/MedLLMsPracticalGuide

[Nature Reviews Bioengineering🔥] Application of Large Language Models in Medicine. A curated…

2,042

FreedomIntelligence/Awesome-AI4Med

A curated list of medical LLMs, multimodal systems, datasets, benchmarks, and more. 🏥

2,888

mingze-yuan/Awesome-LLM-Healthcare

The paper list of the review on LLMs in medicine - "Large Language Models Illuminate a Progressive…

272

yczhou001/Awesome-Medical-LLM-Agent

Reasoning as the Engine: The Evolution from Medical LLMs to Versatile Medical Agents