ClinFusion Medical VLM

This Space runs ClinFusion-8B, a vision-centric multimodal LLM system for holistic medical understanding.

Features

• 2D Medical Image Understanding: Upload a medical image and ask questions about it
• Text-Only Medical QA: Ask medical questions without an image
• Cascaded Vision Encoder: Combines Qwen3-VL's native vision encoder with DINOv2 and ConvNeXt for enhanced medical image understanding

Model Architecture

ClinFusion is built on Qwen3-VL-8B-Instruct and adds:

• DINOv2-large — self-supervised vision encoder
• ConvNeXt-Large (OpenCLIP) — additional vision features
• 3D PE ViT — custom 3D positional encoding vision encoder (for 3D volumes)

The encoders are fused via a Cascade Spatial-Aware Locality Fusion operator.

References

• Paper
• GitHub
• Model Weights