Fu-TCM: a multimodal large language model that learns traditional Chinese medicine diagnostic reasoning

🤗 Fu-TCM-9B | 🤗 Fu-TCM-27B

Overview

FU-TCM is a multimodal large-language-model family for Traditional Chinese Medicine (TCM), trained on approximately 7.12 million examples derived from classical texts, textbooks, medical images, clinical cases, and public datasets. It is designed to connect evidence from the four diagnostic methods to explicit intermediate bianzheng fields and a final syndrome prediction.

Training proceeds in two stages. TCM domain-specific learning uses text QA, image-text VQA, and case-reasoning examples to establish TCM knowledge, multimodal understanding, and structured reasoning. Bianzheng-Grounded Policy Optimization (BGPO) then optimizes response format, tree-based syndrome consistency, and fidelity across 30 intermediate bianzheng fields.

FU-TCM framework: multisource data construction, two-stage training, benchmark performance, and external validation.

Key Features

• 🧠 From knowledge to bianzheng. Fu-TCM shifts TCM large-model training from knowledge accumulation toward learning how clinical evidence is organized into bianzheng decisions.
• 🔎 Multimodal four-examination reasoning. A unified architecture integrates evidence from the four diagnostic methods and generates a complete, traceable evidence-to-syndrome chain for clinician review.
• 📊 Process–outcome evaluation. A dual-dimensional evaluation system measures both final predictions and the intermediate bianzheng process. Fu-TCM achieves state-of-the-art results across six benchmarks and professional-level external validation.
• 🔓 Computable and open TCM knowledge. TCM bianzheng experience is encoded as 30 learnable fields and a syndrome taxonomy, while open data, code, and model weights lower the cost of reuse and accelerate international research.

🚀 Model Zoo

We provide two model variants with different parameter scales:

🏆 Performance Highlights

TCM Reasoning, Text, and Vision Benchmarks

Accuracy comparison across six benchmarks. Bold indicates Fu-TCM models; macro averages include only models with results on all six benchmarks.

🛠️ Installation

# Clone the repository
git clone https://github.com/HC-Guo/FU-TCM.git
cd FU-TCM

# Create the Conda environment
conda create -n fu-tcm python=3.12 -y
conda activate fu-tcm

# Install PyTorch for CUDA 12.8
pip install torch==2.11.0 torchvision==0.26.0 torchaudio==2.11.0 \
  --index-url https://download.pytorch.org/whl/cu128

# Install FlashAttention
pip install flash-attn==2.8.3 --no-build-isolation

# Install the project dependencies
pip install -r requirements.txt

💻 Quick Start

import torch
from transformers import AutoProcessor, Qwen3_5ForConditionalGeneration

model_id = "fudanxai/Fu-TCM-9B"

processor = AutoProcessor.from_pretrained(
    model_id,
    trust_remote_code=True,
)
model = Qwen3_5ForConditionalGeneration.from_pretrained(
    model_id,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    attn_implementation="flash_attention_2",
).eval()

messages = [
    {
        "role": "user",
        "content": "请根据四诊信息分析患者的中医证型，并说明辨证依据。",
    }
]

text = processor.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,
)
inputs = processor(text=text, return_tensors="pt").to(model.device)

with torch.inference_mode():
    generated_ids = model.generate(
        **inputs,
        max_new_tokens=512,
        do_sample=False,
    )

input_length = inputs["input_ids"].shape[1]
response = processor.decode(
    generated_ids[0][input_length:],
    skip_special_tokens=True,
)
print(response)

📊 Training

Data Preparation

Our training data consists of three categories:

• Text QA (1.53M samples): Classical TCM texts, modern medical textbooks, and syndrome-focused knowledge
• Image-Text VQA (8,331 samples): Image-grounded questions covering tongue, facial, pulse, and other visual diagnostic evidence
• Bianzheng Reasoning (1,867 cases): Structured clinical cases linking four-examination evidence, 30 intermediate bianzheng fields, and final syndrome labels

🏗️ Model Architecture

Fu-TCM consists of four core components:

1. Multimodal Backbone: Qwen3.5-9B and Qwen3.6-27B provide unified text and image understanding for the two Fu-TCM model scales.
2. Four-Examination Evidence Encoder: Clinical text and diagnostic images are organized as evidence from inspection, auscultation and olfaction, inquiry, and palpation.
3. Structured Bianzheng Representation: Evidence is mapped to 30 intermediate fields: four eight-principle fields, five zang-fu fields, 15 qi–blood–body-fluid fields, and six pathogenic-factor fields.
4. Syndrome Prediction and BGPO Alignment: The model produces the final syndrome prediction, while format, syndrome-tree, and intermediate-field rewards jointly optimize the reasoning path.