ClinFusion-Eval-Data is the unified evaluation corpus used to benchmark the ClinFusion model series (ClinFusion-8B, ClinFusion-32B). It packages 211,810 evaluation records spanning 22 public medical benchmarks into a single, consistently-formatted suite, together with 509 GiB of the underlying 2D images and native 3D CT volumes they refer to.

The goal is reproducibility: instead of re-downloading and re-normalising two dozen heterogeneous benchmarks, you get one media archive, one path convention, and one record schema covering multiple-choice, open-ended, and report-generation evaluation.

---

🌟 What's Inside

• Three evaluation tracks: general 2D multimodal & text-only medical QA, native 3D volumetric CT reasoning, and an instruction-following stress set.
• Unified record schema: every record carries a source, an eval_method, and media paths that resolve directly into the extracted mm_data/ tree.
• Rich difficulty metadata: the general track is tagged with clinical task type, organs_involved, sample_quality, and hardness, so results can be sliced by capability instead of reported as one aggregate number.
• Native 3D support: 3D questions reference NIfTI (.nii.gz) volumes, including multi-phase longitudinal series for temporal diagnosis.

---

📁 Files

The _lite file is a subset of the full 3D track, so record counts are not additive. The four jsonl files can be accessed in ClinFusion Github; this repository hosts the media archive they point to.

Why the media is sharded

The Hub caps single files at 50 GB, so mm_data.tar (509.25 GiB) ships as 102 byte-exact shards of 5 GiB (the last one is 4.25 GiB). Concatenating them reproduces the original archive bit-for-bit.

---

🚀 Quick Start

1. Download and reassemble the media

huggingface-cli download Alibaba-DAMO-Academy/ClinFusion-Eval-Data \
    --repo-type dataset --local-dir ClinFusion-Eval-Data

cd ClinFusion-Eval-Data
cat mm_data.tar.part-* | tar xf -      # -> ./mm_data/

Reassembly needs ~1 TB of free space while both the shards and the extracted tree are present; delete the shards afterwards to halve that. Verify the concatenated archive before extracting if you prefer:

cat mm_data.tar.part-* | wc -c         # expect 546797690880

2. Load the annotations

The jsonl files ship with the ClinFusion GitHub repository; clone it alongside the extracted media.

import json

records = [json.loads(line) for line in open("general_eval_data_with_tags_v1.jsonl")]
sample = records[0]

print(sample["source"])        # chexpert_plus
print(sample["eval_method"])   # report_generation
print(sample["images_path"])   # ['mm_data/CheXpert_Plus/valid/patient64620/study1/view1_frontal.png']

Media paths are relative to the directory holding mm_data/, so run evaluation from the extraction root and they resolve as-is.

For end-to-end inference and scoring scripts, see the ClinFusion GitHub repository.

---

🧬 Record Schema

2D / text track (general_eval_data_with_tags_v1.jsonl)

3D track (3d_eval_data_*.jsonl) replaces images_path with ct_path (NIfTI volumes), carries question plus optional options, and uses meta_info.task for the 3D capability label.

Instruction-following track (if_eval_data_no_staging.jsonl) carries question, options, and images_path, with eval_method selecting the required answer format.

---

📊 Composition

2D / text sources (19)

3D sources: 3d_rad (33,910), ct_rate_vqa (15,249), amos-mm (3,187).

Clinical task distribution (2D / text track)

3D capability distribution: Existence_Detection (23,472), Static_Temporal_Diagnosis (2,873), Longitudinal_Temporal_Diagnosis (2,873), Anomaly_Detection (2,666), Image_Observation (1,024), Medical_Computation (1,002).

Evaluation formats: mcq dominates the 2D track (152,850), with open (4,815), mcq_context (500) and report_generation (496). The instruction-following track spreads 803 items across mcq, mcq_context, open, report_generation, prognosis, treatment and seer.

Media formats: media references in the general track resolve to PNG (90,081), JPG/JPEG (56,051), TIF (2,573) and BMP (431) files; all 3D items reference NIfTI .nii.gz volumes.

Difficulty and quality: hardness splits into mild (100,468), easy (57,547) and hard (646); sample_quality flags 1,171 items as bad and 24 as mild, which you may exclude for a cleaner comparison.

---

⚖️ Licensing and Intended Use

This release is distributed under Apache-2.0, but each constituent benchmark remains under its own original license and terms of use. Before redistributing or using any subset commercially, consult the upstream license for the relevant source (CT-RATE, AMOS-MM, OmniMedVQA, PMC-VQA, CheXpert Plus, MedXpertQA, and the others listed above).

The data is provided for research evaluation only. It is not a medical device, contains de-identified third-party clinical data, and must not be used for diagnosis or treatment decisions.

---

📄 Citation

If you find ClinFusion useful in your research, please consider citing:

@article{yuan2026ClinFusion,
  title={ClinFusion: A Vision-Centric Multimodal LLM System for Holistic Medical Understanding},
  author={Yuan, Hangjie and Qian, Yichen and Tang, Zhiwei and Xu, Xianzhe and Wu, Lirong and Yang, Sicheng and Wang, Jinwang and Wang, Pengju and Zeng, Zhitao and Han, Yizeng and Xing, Yan and Luo, Shengxuan and Feng, Tao and Xie, Qing and Yao, Weigen and Yang, Yi and Liu, Zuozhu and Tang, Jiasheng and Wang, Shaocheng and Wang, Jitao and Dong, Jiahong and Chen, Weihua and Xu, Feng and Wang, Fan},
  journal={arXiv preprint arXiv:2607.24743},
  year={2026}
}

Please also cite the individual benchmarks you report results on.

---

📞 Contact & Acknowledgements

Built with ❤️ by Alibaba DAMO Academy. This suite would not exist without the teams behind the public benchmarks it aggregates — our thanks to all of them for releasing their data to the community.