SeePhys Pro: Diagnosing Modality Transfer and Blind-Training Effects in Multimodal RLVR for Physics Reasoning

SeePhys Pro is the benchmark from our paper for diagnosing modality transfer in multimodal physics reasoning. It evaluates the same underlying physics across progressively more visual representations, making it useful for measuring whether a model is grounding its reasoning in diagrams, images, and visual evidence rather than relying only on text priors.

This unified repository combines the full public benchmark and the compact development benchmark. Each level is provided as a Hugging Face dataset config/subset, and each subset contains only two splits: test and testmini.

Levels 1-4 correspond to the main benchmark levels described in the paper. Level 5 is an additional real-world education subset provided by Transsion, a leading smart device and mobile internet company focused on emerging markets. It consists of anonymized service data from Transsion's AI photo-based problem-solving service and evaluates models under realistic conditions, including diverse question formats and natural image quality variations.

Subsets and Splits

All subsets use the same public fields: question_id, row_id, idx, problem, images, answer, unit, answer_type, error_range, discipline, domain, and field.

Loading

from datasets import load_dataset

level1_test = load_dataset("Kun-Xiang/SeePhysPro", "level1", split="test")
level1_testmini = load_dataset("Kun-Xiang/SeePhysPro", "level1", split="testmini")

Project Family

• Benchmark: SeePhysPro
• RL training set: PhysRL

Links

• Paper: SeePhys Pro: Diagnosing Modality Transfer and Blind-Training Effects in Multimodal RLVR for Physics Reasoning
• Project page: seephyspro.github.io
• GitHub: AI4Phys/SeePhy-Pro
• Workshop challenge: AI for Math Workshop at ICML 2026
• Challenge submission: Codabench Track 3

Citation

@article{xiang2026seephyspro,
  title={SeePhys Pro: Diagnosing Modality Transfer and Blind-Training Effects in Multimodal RLVR for Physics Reasoning},
  author={Xiang, Kun and Zhang, Terry Jingchen and Liu, Zirong and Zhou, Bokai and Tang, Yueling and Yu, Junjie and Lu, Jiacong and Huang, Shangrui and Li, Heng and Zhang, Likui and Liu, Kunkun and Zhang, Changzheng and Fang, Yangle and Guo, Boqiang and Zhen, Hui-Ling and Tu, Dandan and Huang, Yinya and Liang, Xiaodan},
  journal={arXiv preprint arXiv:2605.09266},
  year={2026}
}