Description

This dataset can be used to evaluate the capabilities of large language models in traditional Chinese medicine and contains multiple-choice, multiple-answer, and true/false questions.

Changelog

• 2024-08-28: Added 7226 questions.
• 2024-08-09: The benchmark code is available at https://github.com/huangxinping/HWTCMBench.
• 2024-08-02: System prompts are removed to ensure the purity of the evaluation results.
• 2024-07-20: Debut.

Examples

multiple-answers questions（多选题）

[
  {
    "instruction": "便秘的预防调护应注意\nA.保持心情舒畅\nB.少吃辛辣刺激性食物\nC.适当摄入油脂\nD.积极治疗肛门直肠疾病\nE.按时登厕",
    "input": "",
    "output": "ABCDE"
  }
]

multiple-choice questions（单选题）

[
  {
    "instruction": "患者，男，50岁。眩晕欲仆，头摇而痛，项强肢颤，腰膝疫软，舌红苔薄白，脉弦有力。其病机是\nA．肝阳上亢\nB．肝肾阴虚\nC．肝阳化风\nD．阴虚风动\nE．肝血不足",
    "input": "",
    "output": "C"
  }
]

True/False questions（判断题）

[
  {
    "instruction": "秦医医和提出了“六气病源说”。",
    "input": "",
    "output": "正确"
  },
  {
    "instruction": "中风中经络邪盛时也可出现神志改变",
    "input": "",
    "output": "错误"
  }
]

Evaluation

Canggong-14b-chat is an LLM of traditional Chinese medicine still in training.

Citation

If you find this project useful in your research, please consider cite:

@misc{hwtcm2024,
    title={{hwtcm} a traditional Chinese medicine QA dataset for evaluating large language models},
    author={Haiwei AI Team},
    howpublished = {\url{https://huggingface.co/datasets/Monor/hwtcm}},
    year={2024}
}