Diabetica-o1-SFT

Code ｜Paper

Introduction

Specifically, we use Deepseek-R1-Distilled-Qwen-32B as our teacher model. Our data augmentation strategy follows a two-step approach: (1) We prompt Qwen2.5-72B-Instruct to generate diverse synthetic questions based on existing datasets. (2) We then use Deepseek-R1-Distilled-Qwen-32B to generate responses for both the collected and synthetic instructions, resulting in an enriched dataset of 70K samples with extensive CoT reasoning steps. After that, we use the 70K dataset to fine-tune Qwen2.5-7B-Instruct and get Diabetica-o1-7B.

Please refer to our Paper for more details.

Citation

@article{wei2024adapted,
  title={An adapted large language model facilitates multiple medical tasks in diabetes care},
  author={Wei, Lai and Ying, Zhen and He, Muyang and Chen, Yutong and Yang, Qian and Hong, Yanzhe and Lu, Jiaping and Li, Xiaoying and Huang, Weiran and Chen, Ying},
  journal={arXiv preprint arXiv:2409.13191},
  year={2024}
}