MediSimplifier: LoRA Adapters for Medical Text Simplification

Fine-tuned LoRA adapters that simplify medical discharge summaries to 6th-grade reading level for patient comprehension.

Model Overview

Key Achievement: ~50% readability reduction (FK 14.5 → ~7.0), matching ground truth quality.

Repository Contents

├── openbiollm_8b_lora/          # Best overall quality
├── mistral_7b_lora/             # Best readability (FK 6.91)
├── biomistral_7b_dare_lora/     # Medical domain baseline
└── checkpoints/
    ├── full_training_llama3/    # Epochs 1-3 (checkpoint-500/1000/1500)
    ├── full_training_mistral/   # Epochs 1-3
    └── full_training_biomistral/ # Epochs 1-3

Quick Start

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load base model + adapter (example: OpenBioLLM - best performer)
base_model = AutoModelForCausalLM.from_pretrained(
    "aaditya/Llama3-OpenBioLLM-8B",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
model = PeftModel.from_pretrained(base_model, "GuyDor007/MediSimplifier-LoRA-Adapters/openbiollm_8b_lora")
tokenizer = AutoTokenizer.from_pretrained("aaditya/Llama3-OpenBioLLM-8B")

# Inference
SYSTEM_MESSAGE = "You are a helpful medical assistant that simplifies complex medical text for patients."

TASK_INSTRUCTION = """Simplify the following medical discharge summary in plain language for patients with no medical background.
Guidelines:
- Replace medical jargon with everyday words (e.g., "hypertension" → "high blood pressure")
- Keep all important information (diagnoses, medications, follow-up instructions)
- Use short, clear sentences (aim for 15-20 words per sentence)
- Aim for a 6th-grade reading level
- Maintain the same structure as the original
- Do not add or omit information"""

# ChatML format for OpenBioLLM
prompt = f"""<|im_start|>system
{SYSTEM_MESSAGE}<|im_end|>
<|im_start|>user
{TASK_INSTRUCTION}

{complex_medical_text}<|im_end|>
<|im_start|>assistant
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7)
simplified = tokenizer.decode(outputs[0], skip_special_tokens=True)

Using Mistral-based Models

# For mistral_7b_lora or biomistral_7b_dare_lora
base_model = AutoModelForCausalLM.from_pretrained(
    "mistralai/Mistral-7B-Instruct-v0.2",  # or "BioMistral/BioMistral-7B-DARE"
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
model = PeftModel.from_pretrained(base_model, "GuyDor007/MediSimplifier-LoRA-Adapters/mistral_7b_lora")

# Mistral format
prompt = f"""[INST] <<SYS>>
{SYSTEM_MESSAGE}
<</SYS>>

{TASK_INSTRUCTION}

{complex_medical_text} [/INST]"""

Training Configuration

Ablation Study Findings

Key Research Findings

1. Ranking Reversal: Worst zero-shot model (OpenBioLLM) achieved best fine-tuned performance
2. Medical Pretraining: Advantage disappears after domain-specific fine-tuning
3. All Models Succeed: 50% readability reduction achieved (college → 7th grade level)
4. Statistical Significance: All ROUGE-L differences significant (p < 0.001)

Dataset

• Source: Asclepius-Synthetic-Clinical-Notes (10K samples)
• Ground Truth: Claude Opus 4.5 generated simplifications
• Processed Dataset: GuyDor007/medisimplifier-dataset

Evaluation Metrics

Checkpoints

Training checkpoints saved at each epoch for reproducibility and analysis:

Citation

@misc{medisimplifier2026,
  author = {Dor, Guy and Avraham, Shmulik},
  title = {MediSimplifier: LoRA Fine-Tuning for Medical Discharge Summary Simplification},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/GuyDor007/MediSimplifier-LoRA-Adapters}}
}

License

Apache 2.0

Acknowledgments

• Technion DS25 Deep Learning Course
• Base models: OpenBioLLM, Mistral, BioMistral teams
• Dataset: Asclepius-Synthetic-Clinical-Notes (starmpcc)

Links

• 📊 Dataset: GuyDor007/medisimplifier-dataset
• 💻 Code: GitHub - MediSimplifier (coming soon)
• 📄 Paper: IEEE-format technical report included in GitHub repo