Datasets and Recipes for Temporal Video Grounding via Reinforcement Learning

TL;DR: This repository presents datasets and recipes to enhance temporal grounding and reasoning capabilities in large vision-language models (LVLMs) for video temporal grounding tasks.

Acknowledgement

Our codes and repository are built on EasyR1 and VideoMind. Thanks for their awesome work!

📦 TVG-Cold-Start Dataset

Our Raw Annotations

We release full 56K Chain-of-Thought temporal video grounding annotation by Gemini-2.5-Pro, which can be accessed by TVG_raw_annotation.

Dataset Files

The processed Cold-Start and RL datasets can be accessed by TVG_processed_annotation.

Raw Videos

Raw videos of TVG-Cold-Start Dataset can be downloaded from VideoMind and Grounded-VideoLLM. Here we provide the script for downloading and preprocessing data.

bash Download_raw_videos.sh

The list of source datasets is shown below.

🚀 Training

Installation

bash init_Easy_R1.sh

GRPO Training

cd EasyR1
bash example/TVG_R1.sh

More training settings and details can be referred to EasyR1.

Evaluation

Benchmarks

Raw videos of TVG-Cold-Start Dataset can be downloaded from VideoMind and Grounded-VideoLLM. Here we provide the script for downloading and preprocessing data.

bash Download_evaluation_raw_videos.sh

The list of benchmarks is shown below.

Here we provide the script for running the evaluation.

bash evaluation.sh

Then you can view the results by

python videomind/eval/eval_auto.py ./outputs/<your-output-name>

📖 Citation

Please kindly cite our paper if you find this project helpful.

@misc{chen2025datasetsrecipesvideotemporal,
      title={Datasets and Recipes for Video Temporal Grounding via Reinforcement Learning}, 
      author={Ruizhe Chen and Zhiting Fan and Tianze Luo and Heqing Zou and Zhaopeng Feng and Guiyang Xie and Hansheng Zhang and Zhuochen Wang and Zuozhu Liu and Huaijian Zhang},
      year={2025},
      eprint={2507.18100},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2507.18100}, 
}