🩺 Reasoning as the Engine: The Evolution from Medical LLMs to Versatile Medical Agents

---

 

---

💡 Welcome Contributions to Awesome-Medical-LLM-Agent 💡

If you find missing work or have suggestions, feel free to open an issue or contact us at yucheng.zhou@connect.um.edu.mo. We will promptly update the repository.

If you find our survey is useful in your research or applications, please consider giving us a star 🌟 and citing it by the following BibTeX entry.

@article{zhoureasoning,
  title={From Medical LLMs to Versatile Medical Agents: A Comprehensive Survey},
  author={Zhou, Yucheng and Zheng, Huan and Chen, Dubing and Yang, Hongji and Han, Wencheng and Shen, Jianbing},
  booktitle={OpenReview},
  year={2025},
  url={https://openreview.net/pdf?id=75M55jtwj6},
}

---

📌 Quick Links

• 1. Medical LLM
• 2. Medical Single Agent
• 3. Medical Multi Agent
• 4. Medical LLM & Agent Survey & Benchmark
• 🌱 Acknowledgements

---

1. Medical LLM

• A foundation model to predict and capture human cognition, Nature, 2025.
• A Foundational Multimodal Vision Language AI Assistant for Human Pathology, arXiv, 2023.
• A generalist vision–language foundation model for diverse biomedical tasks, Nature Medicine, 2024.
• A study of generative large language model for medical research and healthcare, NPJ Digital Medicine, 2023.
• An Explainable Biomedical Foundation Model via Large-Scale Concept-Enhanced Vision-Language Pre-training, arXiv, 2025.
• A Knowledge-enhanced Pathology Vision-language Foundation Model for Cancer Diagnosis, arXiv, 2024.
• Augmenting intensive care unit nursing practice with generative AI: A formative study of diagnostic synergies using simulation-based clinical cases, J Clin Nurs., 2025.
• [AutoRG-Brain: Grounded Report Generation for Brain MRI](AutoRG-Brain: Grounded Report Generation for Brain MRI), arXiv, 2024.
• Beyond Self‑Consistency: Ensemble Reasoning Boosts Consistency and Accuracy of LLMs in Cancer Staging, AIME, 2024.
• BianQue: Balancing the Questioning and Suggestion Ability of Health LLMs with Multi-turn Health Conversations Polished by ChatGPT, arXiv, 2023.
• BioGPT: Generative Pre‑Trained Transformer for Biomedical Text Generation and Mining, Briefings in Bioinformatics, 2022.
• Can Generalist Foundation Models Out‑Compete Special‑Purpose Tuning? Case Study in Medicine, arXiv, 2023.
• Can Large Language Models Reason About Medical Questions?, Patterns, 2024.
• Can GPT-4V(ision) Serve Medical Applications? Case Studies on GPT-4V for Multimodal Medical Diagnosis, arXiv, 2023.
• Capabilities of Gemini Models in Medicine, arXiv, 2024.
• Chain‑of‑Thought Prompting Strategies for Medical Error Detection and Correction, arXiv, 2024.
• Chain‑of‑Thought Strategy for Smaller LLMs for Medical Reasoning, Studies in Health Technology & Informatics, 2025.
• Chain‑of‑Thought Utilization in Large Language Models and Application in Nephrology, Medicina, 2024.
• ChatDoctor: A Medical Chat Model Fine‑Tuned on a Large Language Model (LLaMA) Using Medical Domain Knowledge, Cureus, 2023.
• ChestGPT: Integrating Large Language Models and Vision Transformers for Disease Detection and Localization in Chest X-Rays, arXiv, 2025.
• CITRUS: Leveraging Expert Cognitive Pathways in a Medical Language Model for Advanced Medical Decision Support, arXiv, 2025.
• ClinicalGPT: Large Language Models Finetuned with Diverse Medical Data and Comprehensive Evaluation, arXiv, 2023.
• Closing the Gap Between Open‑Source and Commercial LLMs for Medical Evidence Summarization, NPJ Digital Medicine, 2024.
• CXR-LLAVA: a multimodal large language model for interpreting chest X-ray images, arXiv, 2023.
• DeID‑GPT: Zero‑Shot Medical Text De‑Identification by GPT‑4, arXiv, 2023.
• DeepSeek Deployed in 90 Chinese Tertiary Hospitals: How Artificial Intelligence Is Transforming Clinical Practice, Journal of Medical Systems, 2025.
• Development, Evaluation, and Assessment of Large Language Models (DEAL) Checklist: A Technical Report, NEJM AI, 2025.
• Diagnostic Reasoning Prompts Reveal the Potential for LLM Interpretability in Medicine, NPJ Digital Medicine, 2024.
• DiagX-DT: diagnostic exclusion reasoning with dialectical thinking for traditional Chinese medicine in large language models, Journal of King Saud University Computer and Information Sciences, 2025.
• DISC-MedLLM: Bridging General Large Language Models and Real-World Medical Consultation, arXiv, 2023.
• DoctorGLM: Fine‑tuning your Chinese Doctor is not a Herculean Task, arXiv, 2023.
• ELMTEX: Fine‑Tuning Large Language Models for Structured Clinical Information Extraction, arXiv, 2025.
• Evaluating and mitigating bias in AI-based medical text generation, Nature Computational Science, 2025.
• Freeze the backbones: A Parameter-Efficient Contrastive Approach to Robust Medical Vision-Language Pre-training, ICASSP, 2024.
• From MedPrompt to o1: Exploration of Run‑Time Strategies for Medical Challenge Problems and Beyond, arXiv, 2024.
• GatorTron: A Large Clinical Language Model to Unlock Patient Information from Unstructured EHRs, arXiv, 2022.
• Health-LLM: Personalized Retrieval-Augmented Disease Prediction System, ACL 2025 NLP4PosImpact Workshop, 2025.
• HealthGPT: A Medical Large Vision-Language Model for Unifying Comprehension and Generation via Heterogeneous Knowledge Adaptation, arXiv, 2025.
• Hidden Flaws Behind Expert‑Level Accuracy of Multimodal GPT‑4 Vision in Medicine, NPJ Digital Medicine, 2024.
• Huatuo: Tuning LLaMA with Chinese Medical Knowledge, arXiv, 2023.
• HuatuoGPT: Taming Language Models to Be a Doctor, arXiv, 2023.
• HuatuoGPT-II, One-stage Training for Medical Adaption of LLMs, COLM, 2024.
• HuatuoGPT‑o1: Towards Medical Complex Reasoning with LLMs, arXiv, 2024.
• HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scale, EMNLP, 2024.
• Improving Medical Large Vision-Language Models with Abnormal-Aware Feedback, ACL, 2025.
• IvyGPT: InteractiVe Chinese Pathway Language Model in Medical Domain, CICAI 2023, 2024.
• Knowledge-tuning Large Language Models with Structured Medical Knowledge Bases for Trustworthy Response Generation in Chinese, ACM Transactions on Knowledge Discovery from Data, 2025.
• Large Language Model‑Informed ECG Dual Attention Network for Heart Failure Risk Prediction, IEEE Transactions on Big Data, 2025.
• Large Language Models as Biomedical Hypothesis Generators: A Comprehensive Evaluation, arXiv, 2024.
• Large Language Models Encode Clinical Knowledge, Nature, 2023.
• Large Language Models to Identify Advance Care Planning in Patients With Advanced Cancer, J Pain Symptom Manage., 2025.
• LoRKD: Low-Rank Knowledge Decomposition for Medical Foundation Models, arXiv, 2024.
• MCA-RG: Enhancing LLMs with Medical Concept Alignment for Radiology Report Generation, arXiv, 2025.
• Med-Flamingo: a Multimodal Medical Few-shot Learner, Proceedings of the 3rd Machine Learning for Health Symposium, PMLR 225:353-367, 2023.
• MedAlign: A Clinician-Generated Dataset for Instruction Following with Electronic Medical Records, arXiv, 2023.
• MedAlpaca—An Open‑Source Collection of Medical Conversational AI Models and Training Data, arXiv, 2023.
• MedBLIP: Bootstrapping Language-Image Pre-training from 3D Medical Images and Texts, arXiv, 2023.
• MedGemma Technical Report, arXiv, 2025.
• MedGo: A Chinese Medical Large Language Model, arXiv, 2025.
• Medical Hallucinations in Foundation Models and Their Impact on Healthcare, arXiv, 2025.
• MedPAIR: Measuring Physicians and AI Relevance Alignment in Medical Question Answering, arXiv, 2025.
• MED‑Prompt: A Novel Prompt Engineering Framework for Medicine Prediction on Free‑Text Clinical Notes, Journal of King Saud University – Computer & Information Sciences, 2024.
• Medical Reasoning in LLMs: An In‑Depth Analysis of DeepSeek R1, Frontiers in Artificial Intelligence, 2025.
• Medvlm-r1: Incentivizing medical reasoning capability of vision-language models (vlms) via reinforcement learning, arXiv, 2025.
• Med-r1: Reinforcement learning for generalizable medical reasoning in vision-language models, arXiv, 2025.
• MiniGPT-Med: Large Language Model as a General Interface for Radiology Diagnosis, arXiv, 2024.
• MOSS-MED: A Family of Multimodal Models Serving Medical Image Analysis, ACM Transactions on Management Information Systems, 2025.
• MRGen: Segmentation Data Engine For Underrepresented MRI Modalities, arXiv, 2025.
• Multimodal generative AI for interpreting 3D medical images and videos, npj Digital Medicine, 2025.
• OpenMedLM: Prompt Engineering Can Out‑Perform Fine‑Tuning in Medical Question‑Answering with Open‑Source LLMs, Scientific Reports, 2024.
• OphGLM: Training an Ophthalmology Large Language-and-Vision Assistant based on Instructions and Dialogue, Artificial Intelligence in Medicine, 2024.
• Orthodoc: Multimodal large language model for assisting diagnosis in computed tomography, arXiv, 2024.
• PathVLM-R1: A Reinforcement Learning-Driven Reasoning Model for Pathology Visual-Language Tasks, arXiv, 2025.
• Path to Medical AGI: Unify Domain-specific Medical LLMs with the Lowest Cost, arXiv, 2023.
• PediatricsGPT: Large Language Models as Chinese Medical Assistants for Pediatric Applications, NeurIPS, 2024.
• PMC-LLaMA: Towards Building Open-source Language Models for Medicine, Journal of the American Medical Informatics Association, 2024.
• Polaris: A Safety-Focused LLM Constellation Architecture for Healthcare, arXiv, 2024.
• Pre-trained multimodal large language model enhances dermatological diagnosis using SkinGPT-4, Nature Communications, 2024.
• Qilin-Med: Multi-stage Knowledge Injection Advanced Medical Large Language Model, arXiv, 2023.
• Qilin-Med-VL: Towards Chinese Large Vision-Language Model for General Healthcare, arXiv, 2023.
• Reasoning with Large Language Models for Medical Question Answering, Journal of the American Medical Informatics Association, 2024.
• Rethinking Whole-Body CT Image Interpretation: An Abnormality-Centric Approach, arXiv, 2025.
• Taiyi: a bilingual fine-tuned large language model for diverse biomedical tasks, Journal of the American Medical Informatics Association, 2024.
• Thinking About GPT‑3 In‑Context Learning for Biomedical IE? Think Again, arXiv, 2022.
• Towards Expert‑Level Medical Question Answering with Large Language Models, Nature Medicine, 2025.
• Towards Generalist Biomedical AI, NEJM AI, 2024.
• Towards Generalist Foundation Model for Radiology by Leveraging Web-scale 2D&3D Medical Data, arXiv, 2023.
• Towards Human-AI Collaboration in Healthcare: Guided Deferral Systems With Large Language Models, arXiv, 2024.
• UniBiomed: A Universal Foundation Model for Grounded Biomedical Image Interpretation), arXiv, 2025.
• Unibrain: Universal brain mri diagnosis with hierarchical knowledge-enhanced pre-training, Computerized Medical Imaging and Graphics, 2025.
• Visual Med-Alpaca: A Parameter-Efficient Biomedical LLM with Visual Capabilities, 2024.
• XrayGLM: The first Chinese Medical Multimodal Model that Chest Radiographs Summarization, 2023.
• XrayGPT: Chest Radiographs Summarization using Medical Vision-Language Models, ACL 2024-BIONLP Workshop, 2024.
• Zhongjing: Enhancing the Chinese Medical Capabilities of Large Language Model through Expert Feedback and Real-world Multi-turn Dialogue, arXiv, 2023.

2. Medical Single Agent

• Almanac—Retrieval‑Augmented Language Models for Clinical Medicine, NEJM AI, 2024.
• AI-VaxGuide: An Agentic RAG-Based LLM for Vaccination Decisions, arXiv, 2025.
• Augmentation of ChatGPT with Clinician‑Informed Tools Improves Performance on Medical Calculation Tasks, medRxiv, 2023.
• Autonomous Artificial Intelligence Agents for Clinical Decision Making in Oncology, arXiv, 2024.
• Biomni: A General-Purpose Biomedical AI Agent, arXiv, 2025.
• ChestX-Reasoner: Advancing Radiology Foundation Models with Reasoning through Step-by-Step Verification, arXiv, 2025.
• CoD: Towards an Interpretable Medical Agent Using Chain of Diagnosis, arXiv, 2024.
• CT-Agent: A Multimodal-LLM Agent for 3D CT Radiology Question Answering, bioRXiv, 2025.
• DrHouse: An LLM‑Empowered Diagnostic Reasoning System, PACM IMWUT, 2024.
• Ehragent: Code empowers large language models for few-shot complex tabular reasoning on electronic health records, CEMNLP, 2024.
• Emerging cyber attack risks of medical ai agents, arXiv, 2025.
• Future Prospects of Agentic RAG in Medical Research, SSRN, 2025.
• GeneGPT: Augmenting LLMs with Domain Tools for Improved Access to Biomedical Information, Bioinformatics, 2024.
• HEAL-KGGen: A Hierarchical Multi-Agent LLM Framework with Knowledge Graph Enhancement for Genetic Biomarker-Based Medical Diagnosis, arXiv, 2025.
• KGARevion: An AI Agent for Knowledge‑Intensive Biomedical Question Answering, ICLR, 2025.
• Large Language Model Agents Can Use Tools to Perform Clinical Calculations, NPJ Digital Medicine, 2025.
• MMedAgent: Learning to Use Medical Tools with Multi‑Modal Agent, EMNLP Findings, 2024.
• Medrax: Medical reasoning agent for chest x-ray, arXiv, 2025.
• MeNTi: Bridging Medical Calculator and LLM Agent With Nested Tool Calling, NAACL, 2025.
• "My Nose Is Running." "Are You Also Coughing?": Building a Medical Diagnosis Agent with Interpretable Inquiry Logics, IJCAI 2022, 2022.
• Towards Next-Generation Medical Agent: How o1 is Reshaping Decision-Making in Medical Scenarios, arXiv, 2024.
• VoxelPrompt: A Vision‑Language Agent for Grounded Medical Image Analysis, arXiv, 2024.
• VILA-M3: Enhancing Vision-Language Models with Medical Expert Knowledge, arXiv, 2024.

3. Medical Multi Agent

• A Dual-Agent Collaboration Framework Based on Ilms for Nursing Robots to Perform Bimanual Coordination Tasks, ICRA, 2025.
• Adaptive Reasoning and Acting in Medical Language Agents, arXiv, 2024.
• ADAgent: LLM Agent for Alzheimer's Disease Analysis with Collaborative Coordinator, arXiv, 2025.
• Agent Hospital: A Simulacrum of Hospital with Evolvable Medical Agents, arXiv, 2024.
• Agentic LLM Workflows for Generating Patient-Friendly Medical Reports, arXiv, 2024.
• A-IDE : Agent-Integrated Denoising Experts, arXiv, 2025.
• AIPatient: Simulating Patients with EHRs and LLM Powered Agentic Workflow, arXiv, 2024.
• AKIBoards: A Structure‑Following Multi‑Agent System for Predicting Acute Kidney Injury, arXiv, 2025.
• An Agentic System for Rare Disease Diagnosis with Traceable Reasoning, arXiv, 2025.
• An LLM-based Framework for Biomedical Terminology Normalization in Social Media via Multi-Agent Collaboration, ACL 2025.
• An LLM-Driven Multi-Agent Debate System for Mendelian Diseases, arXiv, 2025.
• A Self-Evolving Framework for Multi-Agent Medical Consultation Based on Large Language Models, ICASSP, 2025.
• Beyond Direct Diagnosis: LLM-based Multi-Specialist Agent Consultation for Automatic Diagnosis, arXiv, 2024.
• ClinicalAgent: Clinical Trial Multi‑Agent System with LLM‑Based Reasoning, ACM BCB & Health Informatics, 2024.
• ColaCare: Enhancing Electronic Health Record Modeling Through Large Language Model-Driven Multi-Agent Collaboration, WWW, 2025.
• Coordinated AI agents for advancing healthcare, Nature Biomedical Engineering, 2025.
• DDO: Dual-Decision Optimization via Multi-Agent Collaboration for LLM-Based Medical Consultation, arXiv, 2025.
• Di-Agent: A Multi-Agent Llm Framework for Full Lifecycle Bearing Health Management, SSRN, 2025.
• DoctorAgent-RL: A Multi-Agent Collaborative Reinforcement Learning System for Multi-Turn Clinical Dialogue, arXiv, 2025.
• DynamiCare: A Dynamic Multi-Agent Framework for Interactive and Open-Ended Medical Decision-Making, arXiv, 2025.
• Enhancing diagnostic capability with multi-agents conversational large language models, NPJ Digital Medicine, 2025.
• Empowering LLM Agents With Zero-Shot Optimal Decision-Making Through Q-Learning, ICLR, 2025.
• Few‑Shot Chain‑of‑Thought‑Driven Reasoning to Prompt LLMs for Open‑Ended Medical QA, arXiv, 2024.
• Map: Evaluation and multi-agent enhancement of large language models for inpatient pathways, arXiv, 2025.
• From RAG to Agentic: Validating Islamic-Medicine Responses with LLM Agents, arXiv, 2025.
• KERAP: A Knowledge-Enhanced Reasoning Approach for Accurate Zero-shot Diagnosis Prediction Using Multi-agent LLMs, arXiv, 2025.
• Learning to Be A Doctor: Searching for Effective Medical Agent Architectures, arXiv, 2025.
• LLMs Can Simulate Standardized Patients via Agent Co‑Evolution, arXiv, 2024.
• M3Builder: A Multi-Agent System for Automated Machine Learning in Medical Imaging, arXiv, 2025.
• Magda: Multi-Agent Guideline-Driven Diagnostic Assistance, MICCAI, 2024.
• MAM: Modular Multi-Agent Framework for Multi-Modal Medical Diagnosis via Role-Specialized Collaboration, ACL, 2025.
• MAP: Multi‑Agent Enhancement of LLMs for In‑Patient Pathways, arXiv, 2025.
• MC‑CoT: A Modular Collaborative Chain‑of‑Thought Framework for Zero‑Shot Medical VQA, arXiv, 2024.
• MDAgents: An Adaptive Collaboration of LLMs for Medical Decision‑Making, NeurIPS 2024, 2024.
• MedAgentGym: Training LLM Agents for Code-Based Medical Reasoning at Scale, arXiv, 2025.
• MedAgent-Pro: Towards Evidence-Based Multi-Modal Medical Diagnosis via Reasoning Agentic Workflow, arXiv, 2025.
• Mdteamgpt: A self-evolving llm-based multi-agent framework for multi-disciplinary team medical consultation, arXiv, 2025.
• MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning, arXiv, 2023.
• MedAide: Towards an Omni Medical Aide via Specialized LLM‑Based Multi‑Agent Collaboration, arXiv, 2024.
• MedChat: A Multi-Agent Framework for Multimodal Diagnosis with Large Language Models, arXiv, 2025.
• MedCoT: Medical Chain‑of‑Thought via Hierarchical Expert, arXiv, 2024.
• MedDxAgent: A Unified Modular Agent Framework for Explainable Automatic Differential Diagnosis, arXiv, 2025.
• MediRAG: Hierarchical Intent-Driven Medical Knowledge Framework with Multi-Agent Specialized Answering, arXiv, 2025.
• MedSentry: Understanding and Mitigating Safety Risks in Medical LLM Multi-Agent Systems, arXiv, 2025.
• MMedAgent-RL: Optimizing Multi-Agent Collaboration for Multimodal Medical Reasoning, arXiv, 2025.
• MRGAgents: A Multi-Agent Framework for Improved Medical Report Generation with Med-LVLMs, arXiv, 2025.
• Multi-Agent Reasoning for Cardiovascular Imaging Phenotype Analysis, arXiv, 2025.
• M^3Builder: A Multi-Agent System for Automated Machine Learning in Medical Imaging, arXiv, 2025.
• Pathfinder: A multi-modal multi-agent system for medical diagnostic decision-making applied to histopathology, arXiv, 2025.
• PatientSim: A Persona‑Driven Simulator for Realistic Doctor‑Patient Interactions, arXiv, 2025.
• PiORS: Personalized Intelligent Outpatient Reception Based on Multi‑Agents Medical Scenario Simulation, arXiv, 2024.
• RareAgents: Autonomous Multi‑Disciplinary Team for Rare Disease Diagnosis and Treatment, arXiv, 2024.
• Reinforcing clinical decision support through multi-agent systems and ethical ai governance, arXiv, 2025.
• Rx Strategist: Prescription Verification Using LLM Agent Systems, arXiv, 2024.
• Self‑Evolving Multi‑Agent Simulations for Realistic Clinical Interactions, arXiv, 2025.
• Sequential Diagnosis with Language Models, arXiv, 2025.
• STELLA: Self-Evolving LLM Agent for Biomedical Research, arXiv, 2025.
• SurgBox: Agent-Driven Operating Room Sandbox With Surgery Copilot, arXiv, 2024.
• Tiered Agentic Oversight: A Hierarchical Multi-Agent System for AI Safety in Healthcare, arXiv, 2025.
• WSI-Agents: A Collaborative Multi-Agent System for Multi-Modal Whole Slide Image Analysis, MICCAI, 2025.

4. Medical LLM & Agent Survey & Benchmark

• 3MDBench: Medical Multimodal Multi-agent Dialogue Benchmark, arXiv, 2025.
• A Comprehensive Survey of Electronic Health Record Modeling: From Deep Learning Approaches to Large Language Models, arXiv, 2025.
• Addressing Cognitive Bias in Medical Language Models, arXiv, 2024.
• AgentClinic: A Multimodal Agent Benchmark to Evaluate AI in Simulated Clinical Environments, arXiv, 2024.
• AI Agents in Modern Healthcare: A Comprehensive Review and Implementation Roadmap, Preprints, 2025.
• AI Hospital: Benchmarking LLMs in a Multi‑Agent Medical Interaction Simulator, arXiv, 2024.
• A perspective for adapting generalist AI to specialized medical AI applications and their challenges, npj Digital Medicine, 2025.
• Artificial Intelligence and Automation in Endoscopy and Surgery, Nature Reviews Gastroenterology & Hepatology, 2023.
• Baymax: A Survey of LLM‑Based Agents in Medicine – How Far Are We from Baymax?, ACL Findings, 2025.
• Benchmarking Large Language Models on Communicative Medical Coaching: A Novel System and Dataset, ACL, 2024.
• ClinicalLab: Aligning Agents for Multi-Departmental Clinical Diagnostics in the Real World, arXiv, 2024.
• CliBench: A Multifaceted and Multigranular Evaluation of LLMs for Clinical Decision Making, arXiv, 2024.
• EHRNoteQA: An LLM Benchmark for Real‑World Clinical Practice Using Discharge Summaries, NeurIPS, 2024.
• Evaluating Large Language Models as Agents in the Clinic, npj Digital Medicine, 2024.
• Exploring LLM-based Data Annotation Strategies for Medical Dialogue Preference Alignment, arXiv, 2024.
• Generalization in Healthcare AI: Evaluation of a Clinical LLM, arXiv, 2024.
• Improving Clinical Documentation with AI: A Comparative Study of Sporo AI Scribe and GPT-4o mini, arXiv, 2024.
• How generative AI voice agents will transform medicine, npj Digital Medicine, 2025.
• How Well Can Modern LLMs Act as Agent Cores in Radiology Environments?, arXiv, 2025.
• KG4Diagnosis: A Hierarchical Multi-Agent Ilm Framework With Knowledge Graph Enhancement for Medical Diagnosis, AAAI, 2025.
• LLM-Based Agentic Systems in Medicine and Healthcare, Nature Machine Intelligence volume, 2024.
• M$^3$-Med: A Benchmark for Multi-lingual, Multi-modal, and Multi-hop Reasoning in Medical Instructional Video Understanding, arXiv, 2025.
• MediQ: Question‑Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoning, NeurIPS, 2024.
• MedAgentBench: Dataset for Benchmarking LLMs as Agents in Medical Applications, arXiv, 2025.
• MedAgentBoard: Benchmarking Multi‑Agent Collaboration with Conventional Methods for Diverse Medical Tasks, arXiv, 2025.
• MedAgentsBench: Benchmarking Agent Frameworks for Complex Medical Reasoning, arXiv, 2025.
• MedChain: Bridging the Gap Between LLM Agents and Clinical Practice Through Interactive Sequential Benchmarking, arXiv, 2024.
• MedHallBench: A New Benchmark for Assessing Hallucination in Medical Large Language Models, AAAI, 2025.
• Mitigating Cognitive Biases in Clinical Decision-Making Through Multi-Agent Conversations Using Large Language Models: Simulation Study, Journal of Medical Internet Research, 2024.
• Multimodal generative AI for medical image interpretation, AAAI, 2025.
• Omnimedvqa: A new large-scale comprehensive evaluation benchmark for medical lvlm, CVPR, 2024.
• Optimizing AI solutions for population health in primary care, npj Digital Medicine, 2025.
• Quantifying the Reasoning Abilities of LLMs on Real-world Clinical Cases, arXiv, 2025.
• RadGenome-Chest CT: A Grounded Vision-Language Dataset for Chest CT Analysis, arXiv, 2024.
• RadRevise: A Benchmark Dataset for Instruction-Based Radiology Report Editing, AAAI, 2025.
• RareBench: Can LLMs Serve as Rare Diseases Specialists?, KDD, 2024.
• Reflectool: Towards reflection-aware tool-augmented clinical agents, arXiv, 2024.
• ReXVQA: A Large-scale Visual Question Answering Benchmark for Generalist Chest X-ray Understanding, arXiv, 2025.
• R-Judge: Benchmarking Safety Risk Awareness for LLM Agents, EMNLP, 2024.
• The generative era of medical AI, Cell, 2025.
• Towards Automatic Evaluation for LLMs' Clinical Capabilities: Metric, Data, and Algorithm, arXiv, 2024.
• Towards evaluating and building versatile large language models for medicine, npj Digital Medicine, 2025.
• WHO Guidance: Ethics and Governance of Artificial Intelligence for Health – Large Multi‑Modal Models, World Health Organization, 2024.

🌱 Acknowledgements

We would like to express our sincere appreciation to the researchers, clinicians, medical practitioners, and engineers dedicated to advancing medical artificial intelligence.
Their continuous efforts are driving the transformation from Medical Large Language Models to reliable and versatile Medical Agents, enabling safer, more intelligent, and more accessible healthcare for all.