VRR-QA: Visual Relational Reasoning in Videos Beyond Explicit Cues

Sirnam Swetha  |  Rohit Gupta  |  Parth Parag Kulkarni  |  David G Shatwell  |  Jeffrey A Chan Santiago  |  Nyle Siddiqui  |  Joseph Fioresi  |  Mubarak Shah

University of Central Florida 

VRRQA Dataset

The VRRQA dataset was introduced in the paper VRR-QA: Visual Relational Reasoning in Videos Beyond Explicit Cues.

Project page: https://swetha5.github.io/ImplicitQA/

VRRQA is a novel benchmark specifically designed to test models on implicit reasoning in Video Question Answering (VideoQA). Unlike existing VideoQA benchmarks that primarily focus on questions answerable through explicit visual content (actions, objects, events directly observable within individual frames or short clips), VRRQA addresses the need for models to infer motives, causality, and relationships across discontinuous frames. This mirrors human-like understanding of creative and cinematic videos, which often employ storytelling techniques that deliberately omit certain depictions.

The dataset comprises 1,000 meticulously annotated QA pairs derived from 1,000 high-quality creative video clips. These QA pairs are systematically categorized into key reasoning dimensions, including:

• Lateral vertical spatial reasoning
• Vertical spatial reasoning
• Relative Depth and proximity
• Viewpoint and visibility
• Motion and trajectory Dynamics
• Causal and motivational reasoning
• Social interactions and Relations
• Physical and Environmental context
• Inferred counting

The annotations are deliberately challenging, crafted to ensure high quality and to highlight the difficulty of implicit reasoning for current VideoQA models. By releasing both the dataset and its data collection framework, the authors aim to stimulate further research and development in this crucial area of AI.

---

Citation

If you use this dataset and/or this code in your work, please cite our paper:

@article{swetha2025vrrqa,
title={VRR-QA: Visual Relational Reasoning in Videos Beyond Explicit Cues},
author={Swetha, Sirnam and Gupta, Rohit and Kulkarni, Parth Parag and Shatwell, David G and Santiago, Jeffrey A Chan and Siddiqui, Nyle and Fioresi, Joseph and Shah, Mubarak},
journal={arXiv preprint arXiv:2506.21742},
year={2026}
}