CrossVid: A Comprehensive Benchmark for Evaluating Cross-Video Reasoning in Multimodal Large Language Models

Dataset Description

CrossVid is a large-scale, multi-task dataset designed to advance cross-video understanding capabilities in vision-language models. The dataset encompasses 10 diverse task types that require models to reason across multiple videos, understand temporal dynamics, spatial relationships, and complex narrative structures. Unlike existing benchmarks focusing on single-video analysis, CrossVid is the first comprehensive benchmark designed to evaluate cross-video understanding capabilities in MLLMs.

Key Features

• 🎥 Multi-Domain Videos: Includes assembly tutorials, animal/human behaviors, cooking demonstrations, movie scenes, and UAV footage
• 🎯 10 Challenging Tasks: Covering behavioral analysis, content comparison, temporal reasoning, spatial understanding, and more
• 📊 Rich Annotations: Question-answer pairs with temporal segments, spatial object tracking, and procedural step sequences
• 🌐 Cross-Video Reasoning: Tasks explicitly require understanding relationships and patterns across multiple video clips

Task Types

Dataset Structure

CrossVid/
│── uav/
│   ├── bbox/
│   └── frames/
├── videos/
│   ├── assembly/
│   ├── behavior/
│   ├── cook/
│   └── movie/
│── QA/
│   ├── BU.json
│   ├── CC.json
│   ├── CCQA.json
│   ├── FSA.json
│   ├── MOC.json
│   ├── MSR.json
│   ├── NC.json
│   ├── PEA.json
│   ├── PI.json
│   └── PSS.json
└── README.md

📧 Contact

For questions or issues, please:

• Open an issue on GitHub
• Contact us at: chuntianli666666@gmail.com

🙏 Acknowledgements

Note: The CC BY 4.0 license applies only to the original annotations, benchmark design, and documentation provided in this repository, and does not apply to the underlying source videos or datasets listed below.

We thank the creators of the following datasets that made CrossVid possible:

• Animal Kingdom
• MovieChat-1K
• YouCook2
• VisDrone
• Charades
• Assembly101

We do not own and do not claim any copyright or related rights over these source datasets or their content.

All copyrights and related rights remain with the original dataset creators and copyright holders.

📝 Citation

If you find CrossVid useful for your research, please cite our paper:

@article{li2025crossvid,
  title={CrossVid: A Comprehensive Benchmark for Evaluating Cross-Video Reasoning in Multimodal Large Language Models},
  author={Li, Jingyao and Wang, Jingyun and Tan, Molin and Wang, Haochen and Yan, Cilin and Shi, Likun and Cai, Jiayin and Jiang, Xiaolong and Hu, Yao},
  journal={arXiv preprint arXiv:2511.12263},
  year={2025}
}