zhang9302002/MultiTaskVideoReasoning

Dataset

Multi Task Video Reasoning Dataset

6 commits

6 linked in READMEs

updated Aug 25, 2025

See the code

README

Multi Task Video Reasoning Dataset

This is the official training dataset for Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoning.

[Project] [arXiv] [Code]

Data Structure

└── MultiTaskVideoReasoning
    ├── MTVR_CoT
    │   ├── actnet.json
    │   ├── charades.json
    │   ├── longvideo-reason.json
    │   ├── nextgqa.json
    │   ├── rextime.json
    │   ├── vidchapters.json
    │   ├── Video-R1-data-image.json
    │   └── Video-R1-data-video.json
    ├── MTVR_RL
    │   ├── actnet.json
    │   ├── charades.json
    │   ├── nextgqa.json
    │   ├── rextime.json
    │   ├── vidchapters.json
    │   ├── Video-R1-data-image.json
    │   └── Video-R1-data-video.json
    ├── MTVR_Tool_CoT
    │   ├── longvideo-reason.json
    │   └── vidchapters.json
    ├── MTVR_Tool_RL
    │   ├── longvideo-reason.json
    │   └── vidchapters.json
    └── README.md

Data Preperation

Please download original videos from these sources.

Dataset	Abbr.	Downlaod Link
Charades-STA	charades	https://github.com/jiyanggao/TALL
ActivityNet-MR	actnet	https://cs.stanford.edu/people/ranjaykrishna/densevid/
VidChapters-7M	vidchapters	https://github.com/antoyang/VidChapters
Video-R1	Video-R1	https://huggingface.co/datasets/Video-R1/Video-R1-data
LongVideo-Reason	longvideo-reason	https://github.com/NVlabs/Long-RL/tree/main/longvideo-reason
NExT-GQA	nextgqa	https://github.com/doc-doc/NExT-GQA
ReXTime	rextime	https://huggingface.co/datasets/ReXTime/ReXTime

Citation

If you use this dataset or find it useful, please cite our paper:

@article{zhang2025thinking,
  title={Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoning},
  author={Zhang, Haoji and Gu, Xin and Li, Jiawen and Ma, Chixiang and Bai, Sule and Zhang, Chubin and Zhang, Bowen and Zhou, Zhichao and He, Dongliang and Tang, Yansong},
  journal={arXiv preprint arXiv:2508.04416},
  year={2025}
}

Contributors

zhanghaoji.1

5 commits

zhang9302002

1 commits

Similar repos

zhang9302002/ThinkingWithVideos

The official code of "Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for…

103

longvideotool/LongVT-Source

Dataset

LongVT-Source

longvideotool/LongVT-Parquet

Dataset

LongVT-Parquet

yeliudev/VideoMind-Dataset

Dataset

VideoMind Datasets

QiWang98/VideoRFT-Data

Dataset

🎥 VideoRFT: Incentivizing Video Reasoning Capability in MLLMs via Reinforced Fine-Tuning

ParaVT/ParaVT-Parquet

Dataset

ParaVT-Parquet

ParaVT/ParaVT-Source

Dataset

ParaVT-Source

OneThink/OneThinker-train-data

Dataset

OneThinker-600k Training Data

Linked in READMEs

NVlabs/Long-RL

Long-RL: Scaling RL to Long Sequences (NeurIPS 2025)

729

README badge

yunlong10/Awesome-Video-LMM-Post-Training

🔥🔥🔥 Latest Papers, Codes and Datasets on Video-LMM Post-Training

303

README badge

jiyanggao/TALL

TALL: Temporal Activity Localization via Language Query

219

README badge

antoyang/VidChapters

[NeurIPS 2023 D&B] VidChapters-7M: Video Chapters at Scale

214

README badge

zhang9302002/ThinkingWithVideos

The official code of "Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for…

103

README badge

doc-doc/NExT-GQA

Can I Trust Your Answer? Visually Grounded Video Question Answering (CVPR'24, Highlight)

README badge

zhang9302002/MultiTaskVideoReasoning

Dataset

Multi Task Video Reasoning Dataset

6 commits

6 linked in READMEs

updated Aug 25, 2025

See the code

README

Multi Task Video Reasoning Dataset

This is the official training dataset for Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoning.

[Project] [arXiv] [Code]

Data Structure

└── MultiTaskVideoReasoning
    ├── MTVR_CoT
    │   ├── actnet.json
    │   ├── charades.json
    │   ├── longvideo-reason.json
    │   ├── nextgqa.json
    │   ├── rextime.json
    │   ├── vidchapters.json
    │   ├── Video-R1-data-image.json
    │   └── Video-R1-data-video.json
    ├── MTVR_RL
    │   ├── actnet.json
    │   ├── charades.json
    │   ├── nextgqa.json
    │   ├── rextime.json
    │   ├── vidchapters.json
    │   ├── Video-R1-data-image.json
    │   └── Video-R1-data-video.json
    ├── MTVR_Tool_CoT
    │   ├── longvideo-reason.json
    │   └── vidchapters.json
    ├── MTVR_Tool_RL
    │   ├── longvideo-reason.json
    │   └── vidchapters.json
    └── README.md

Data Preperation

Please download original videos from these sources.

Dataset	Abbr.	Downlaod Link
Charades-STA	charades	https://github.com/jiyanggao/TALL
ActivityNet-MR	actnet	https://cs.stanford.edu/people/ranjaykrishna/densevid/
VidChapters-7M	vidchapters	https://github.com/antoyang/VidChapters
Video-R1	Video-R1	https://huggingface.co/datasets/Video-R1/Video-R1-data
LongVideo-Reason	longvideo-reason	https://github.com/NVlabs/Long-RL/tree/main/longvideo-reason
NExT-GQA	nextgqa	https://github.com/doc-doc/NExT-GQA
ReXTime	rextime	https://huggingface.co/datasets/ReXTime/ReXTime

Citation

If you use this dataset or find it useful, please cite our paper:

@article{zhang2025thinking,
  title={Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoning},
  author={Zhang, Haoji and Gu, Xin and Li, Jiawen and Ma, Chixiang and Bai, Sule and Zhang, Chubin and Zhang, Bowen and Zhou, Zhichao and He, Dongliang and Tang, Yansong},
  journal={arXiv preprint arXiv:2508.04416},
  year={2025}
}