VideoMind: A Chain-of-LoRA Agent for Temporal-Grounded
Video Reasoning

Ye Liu^1†, Kevin Qinghong Lin^2†, Chang Wen Chen¹, Mike Zheng Shou²

¹The Hong Kong Polytechnic University ²Show Lab, National University of Singapore

TL;DR: Pioneer DeepSearch-like Video Understanding.

VideoMind is a multi-modal agent framework that enhances video reasoning by emulating human-like processes, such as breaking down tasks, localizing and verifying moments, and synthesizing answers. This approach addresses the unique challenges of temporal-grounded reasoning in a progressive strategy.

🔥 News

• 2026.01.27 🌴 Our full paper has been accepted by ICLR 2026.
• 2025.09.23 🧠 VideoMind has been accepted by LAW Workshop @ NeurIPS 2025 as Spotlight.
• 2025.04.05 📊 See BENCHMARK.md for evaluation results of VideoMind on public benchmarks.
• 2025.03.28 🚀 VideoMind-2B is ready on Hugging Face Spaces. Check it out!
• 2025.03.21 ⭐️ Code, model, and dataset release.
• 2025.03.17 🎉 Our tech report is available online.

🏆 VideoMind on Public Benchmarks

ZS and FT refer to zero-shot and fine-tuned settings, respectively. * means third-party results.

See BENCHMARK.md for full evaluation results.

🕹️ Gradio Demo

https://github.com/user-attachments/assets/a4d99c05-aa73-4ed9-a275-2362a201bfec

Play with our online demo or see DEMO.md for guidelines about how to deploy it locally.

📦 VideoMind-SFT Dataset

We provide raw videos, compressed videos, and pre-processed annotations of 27 video grounding / QA datasets, including our VideoMind-SFT (481K) for training and multiple benchmarks for evaluation. We also release the datasets used during our early exploration (but not included in the final version) to facilitate future research.

The list of source datasets is shown below. See our dataset repo for more details.

Grounder (210K):

Verifier (232K):

Planner (39K):

Benchmarks

The following datasets are not used in our project (partially used during early exploration), but we still share them to facilitate future research.

Notes:

1. For some datasets (e.g., ReXTime), the annotations and videos are stored in different folders. All the directories in Processed need to be downloaded.
2. Use the following commands to concatenate and extract video tar splits (e.g., videos.tar.gz.00, videos_3fps_480_noaudio.tar.gz.00).

# videos.tar.gz.00, videos.tar.gz.01
cat videos.tar.gz.* | tar -zxvf -

# videos_3fps_480_noaudio.tar.gz.00, videos_3fps_480_noaudio.tar.gz.01
cat videos_3fps_480_noaudio.tar.gz.* | tar -zxvf -

🚀 Training

Our codebase supports training and evaluating on 27 video datasets and benchmarks with the following features.

• Flexible hardware settings: NVIDIA GPU / Ascend NPU, Single-Node / Multi-Node
• Efficient training techniques: DeepSpeed ZeRO, BF16, LoRA, SDPA, FlashAttention2, Liger-Kernel
• Customizing the base LLM and conversation templates
• Monitoring the training process via Tensorboard / Wandb
• Group sampling for mixed dataset training
• Multi-process / multi-device evaluation on public benchmarks

See TRAIN.md for a quick start guide.

🔮 Evaluation

See EVAL.md for details about evaluating VideoMind on public benchmarks.

📖 Citation

Please kindly cite our paper if you find this project helpful.

@inproceedings{liu2026videomind,
  title={VideoMind: A Chain-of-LoRA Agent for Temporal-Grounded Video Reasoning},
  author={Liu, Ye and Lin, Kevin Qinghong and Chen, Chang Wen and Shou, Mike Zheng},
  booktitle={International Conference on Learning Representations (ICLR)},
  year={2026}
}