TemMed-Bench: Evaluating Temporal Medical Image Reasoning in Vision-Language Models

🌐 Homepage | 🐱 Github | 📖 Paper

Intro

TemMed-Bench features three primary highlights.

• Temporal reasoning focus: Each sample in TemMed-Bench includes historical condition information, which challenges models to analyze changes in patient conditions over time.
• Multi-image input: Each sample in TemMed-Bench contains multiple images from different visits as input, emphasizing the need for models to process and reason over multiple images.
• Diverse task suite: TemMed-Bench comprises three tasks, including VQA, report generation, and image-pair selection. Additionally, TemMed-Bench includes a knowledge corpus with more than 17,000 instances to support retrieval-augmented generation (RAG).

Benchmark Overview

• Examples of the three tasks in TemMed-Bench:

• Key statistics of TemMed-Bench:

Load Dataset

Please refer to 🐱 Github

Contact

• Junyi Zhang: JunyiZhang2002@g.ucla.edu

Citation

@misc{zhang2025temmedbenchevaluatingtemporalmedical,
      title={TemMed-Bench: Evaluating Temporal Medical Image Reasoning in Vision-Language Models}, 
      author={Junyi Zhang and Jia-Chen Gu and Wenbo Hu and Yu Zhou and Robinson Piramuthu and Nanyun Peng},
      year={2025},
      eprint={2509.25143},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2509.25143}, 
}