Virgo: A Preliminary Exploration on Reproducing o1-like MLLM

Virgo-Visual-Long-Thought is the long thought visual reasoning data we use to train Virgo-7B. This dataset is built upon 8 visual question answering datasets: four geometry datasets (Geos, GeoQA+, Geometry3K, and UniGeo), three table and figure datasets (TabMWP, FigureQA, and ChartQA), and an object dataset (CLEVR).

• The "question" and "answer" are the original question and answer from the source dataset collected by LLaVA-OneVision.
• The "response" is a list, which contains five responses generated by Virgo-72B.
• The "conversation" is the final data we used to train our model (Virgo-7B). Since there are 5 responses, we select the correct response with the shortest length.
• The "images" are the paths to the images in the LLaVA-OneVision dataset. Since LLaVA-OneVision stores the images in base64 format, you can directly save the images to those paths.
• The "score" is a list of 0s or 1s, with the same length as the "response", indicating whether each response is correct or not.

Please kindly cite our reports if they are helpful for your research.

@article{du2025virgo,
      title={Virgo: A Preliminary Exploration on Reproducing o1-like MLLM}, 
      author={Yifan Du and Zikang Liu and Yifan Li and Wayne Xin Zhao and Yuqi Huo and Bingning Wang and Weipeng Chen and Zheng Liu and Zhongyuan Wang and Ji-Rong Wen},
      journal={arXiv preprint arXiv:2501.01904},
      year={2025}
}