ECGBench

Benchmark Dataset for paper "Teach Multimodal LLMs to Comprehend Electrocardiographic Images".

🌐 Project Page: https://aimedlab.github.io/PULSE/

📄 Paper: https://arxiv.org/abs/2410.19008

🧑‍💻 Code: https://github.com/AIMedLab/PULSE

🤗 Model: https://huggingface.co/PULSE-ECG/PULSE-7B

👩‍⚕️ ECGInstruct: https://huggingface.co/datasets/PULSE-ECG/ECGInstruct

Introduction

We introduce ECGBench, a comprehensive benchmark designed to evaluate ECG image interpretation across a range of tasks involving both real-world and synthesized images. The curation of ECGBench focuses on four key tasks: (1) two repurposed tasks—abnormality detection and report generation—derived from existing ECG datasets, where images are synthesized from raw signals, and queries and answers are extracted from diagnostic and clinical reports; and (2) two newly developed tasks that leverage external resources, where ECG images, along with corresponding questions and answers, are collected and generated from real-world sources.

Dataset Statistics

ECGBench Leaderboard

In-domain

Results marked as † are copied from original papers. Results marked as * are obtained using the provided online software to collect prediction results. N/A: methods are not applicable or not designed for certain tasks. -: scores are not reported in the original papers. Note that the experimental setup of some domain-specific methods is not exactly the same as ours, thus the results are for reference purposes.

Out-of-domain

Results marked as † are copied from original papers. Results marked as * are obtained using the provided online software to collect prediction results. N/A: methods are not applicable or not designed for certain tasks. -: scores are not reported in the original papers.

Citation

If you find this work helpful, please cite our paper:

@article{liu2024teach,
  title={Teach Multimodal LLMs to Comprehend Electrocardiographic Images},
  author={Ruoqi Liu, Yuelin Bai, Xiang Yue, Ping Zhang},
  journal={arXiv preprint arXiv:2410.19008},
  year={2024}
}