ClinBench-HPB: A Clinical Benchmark for Evaluating LLMs in Hepato-Pancreato-Biliary Diseases

ClinBench-HPB is a comprehensive benchmark dataset specifically designed to evaluate the performance of Large Language Models (LLMs) in the field of hepatobiliary and pancreatic diseases.

Dataset Overview

This dataset consists of five carefully constructed subsets covering different data sources and question types:

1. CNQA (Chinese Questions & Answers)

• 2000 Chinese multiple-choice questions
• Includes single-choice and multiple-choice questions
• Each question may have one or more correct answers

2. ENQA (English Questions & Answers)

• 1535 English multiple-choice questions
• Each question has exactly one correct answer

3. Journal

• 120 open diagnostic cases from authoritative medical journals

4. Website

• 167 diagnostic cases from professional medical websites and case-sharing platforms

5. Hospital

• 50 real-world diagnostic cases from partner hospitals (anonymized)

Key Features

• Comprehensiveness: Covers multiple subspecialties of HPB diseases

• Diversity: Includes both multiple-choice questions and real clinical cases

• High Quality: All data reviewed by medical experts

• Multilingual: Supports both Chinese and English evaluation

Usage Suggestions

This dataset is suitable for:

1. Evaluating LLMs' medical knowledge in HPB diseases

2. Comparing model performance on medical QA tasks

3. Developing medical education applications

4. Evaluating clinical decision support systems

Notice

This benchmark is for research purposes only and does not constitute medical advice. Always consult healthcare professionals for clinical decisions. This dataset is licensed for academic research only. Commercial use is strictly prohibited. No modification, distribution, or use in commercial products is permitted without explicit authorization. If you believe this dataset infringes your rights, please contact us via: Email: yuchong.li@connect.polyu.hk

Citation

If you use ClinBench-HPB in your research, please cite our work:

@misc{li2025clinbenchhpbclinicalbenchmarkevaluating,
      title={ClinBench-HPB: A Clinical Benchmark for Evaluating LLMs in Hepato-Pancreato-Biliary Diseases}, 
      author={Yuchong Li and Xiaojun Zeng and Chihua Fang and Jian Yang and Fucang Jia and Lei Zhang},
      year={2025},
      eprint={2506.00095},
      archivePrefix={arXiv},
      primaryClass={cs.CY},
      url={https://arxiv.org/abs/2506.00095}, 
}