Pocket-Dentist: Benchmarking Compact Vision-Language Models for Dental Image Understanding

Table of Contents

1. Qualitative Analysis
2. Introduction
3. Development Guide — Environment setup, inference, and training

---

Qualitative Analysis

To illustrate the practical impact of compact-model adaptation, we present representative test-set cases across all six task types in which the LoRA-adapted Qwen3-VL-4B (4B parameters) produces correct predictions, whereas the substantially larger Lingshu-32B (32B parameters), adapted with the same LoRA budget, fails.

All cases are from the SFT (LoRA) setting and selected from test-set predictions.

📄 For higher resolution, see the full PDF version.

Training Loss

Training loss curves for LoRA-based instruction tuning across seven dental datasets. Each subplot displays the training loss over one epoch for all 13 evaluated VLMs, grouped into Compact VLMs (≤4B parameters, solid lines) and Larger VLMs (≥7B parameters, dashed lines). All models are fine-tuned with identical LoRA configurations (r=16, α=32) and a cosine learning rate schedule. Most models converge rapidly within the first 10–20% of training, with InternVL3.5 and PaliGemma2 exhibiting notably higher initial loss due to their less-aligned vision–language representations. The consistently low final loss of MedMO-8B-Next and Qwen2.5-VL-7B across all datasets reflects their stronger pre-trained foundations for medical image understanding.

📄 For higher resolution, see the full PDF version.

---

Overview

Pocket-Dentist is a large-scale multimodal benchmark and deploy-aware evaluation pipeline for dental vision-language models (VLMs). It curates and standardizes seven heterogeneous dental datasets into a unified vision-language benchmark, enabling systematic evaluation of VLMs across diverse imaging modalities, clinical task types, adaptation strategies, and deployment constraints.

Key Highlights

• 7 dental datasets spanning panoramic radiographs, intraoral photographs, periapical radiographs, and cephalometric radiographs
• 71,000+ images from 6,000+ patients, covering 6 task types and 14 evaluation metrics
• 14 VLMs evaluated under zero-shot, few-shot (1-shot, 2-shot), and LoRA fine-tuning settings
• Core finding: Under a uniform low-cost LoRA adaptation budget, compact VLMs — especially Qwen3-VL-4B — match or outperform substantially larger open-weight models (7B–32B) on most primary task metrics

Benchmark Datasets

Evaluated Models

---

Getting Started

For detailed instructions on environment setup, running evaluation, SFT training, and data format specifications, see the Development Guide.

Quick Start

# Install dependencies
pip install -r requirements.txt

# Run zero-shot evaluation on MetaDent with a single model
bash scripts/run_metadent.sh --models Qwen3-VL-4B-Instruct --tasks baseline

# Run LoRA fine-tuning
bash scripts/run_metadent_sft.sh --models Qwen3-VL-4B-Instruct

Hardware Requirements

---

Citation

@inproceedings{pocket-dentist-2026,
  title     = {Pocket-Dentist: Benchmarking Compact Vision-Language Models for Dental Image Understanding},
  author    = {Anonymous},
  booktitle = {NeurIPS 2026 Evaluations \& Datasets Track},
  year      = {2026},
  note      = {Under review}
}

License

This project is licensed under the Apache License 2.0.