LongVT-Parquet

This repository contains the training data annotations and evaluation benchmark for the LongVT project.

Overview

LongVT is an end-to-end agentic framework that enables "Thinking with Long Videos" via interleaved Multimodal Chain-of-Tool-Thought. This dataset provides the training annotations and evaluation benchmark in Parquet format, with source media files available in LongVT-Source.

Important Notes

For privacy reasons, media paths in the Parquet files were sanitized before release. Please replace them with your own local paths after downloading the corresponding media from LongVT-Source. The annotations and media files follow a one-to-one correspondence across the two repos.

Dataset Structure

The dataset is organized into three training subsets and one evaluation benchmark:

Training Data

Evaluation Benchmark

We have transferred the annotation file of VideoSIAH-Eval to longvideotool/VideoSIAH-Eval.

SFT Data Composition

RL Data

RFT Data

Download

Install huggingface_hub

pip install huggingface_hub

Download all annotation files

huggingface-cli download longvideotool/LongVT-Parquet --repo-type dataset --local-dir ./data

Download source media files

huggingface-cli download longvideotool/LongVT-Source --repo-type dataset --local-dir ./source## Usage with Datasets

from datasets import load_dataset

Load SFT data

sft_data = load_dataset("longvideotool/LongVT-Parquet", "sft", split="train")

Load RL data

rl_data = load_dataset("longvideotool/LongVT-Parquet", "rl", split="train")

Load RFT data

rft_data = load_dataset("longvideotool/LongVT-Parquet", "rft", split="train")

Data Format

Each sample contains:

• id: Unique identifier
• messages: Conversation turns with system prompt, user query, and assistant response
  • Includes <think>, <tool_call>, <tool_response>, and <answer> tags for reasoning traces

Evaluation benchmark format:

• video_path: Path to video file
• question: Question about the video
• answer: Ground truth answer

Related Resources

• 📄 Paper: arXiv:2511.20785
• 🌐 Project Page: LongVT Website
• 💻 Code: GitHub Repository
• 🎬 Source Media: LongVT-Source
• 🤗 Models: LongVT Collection

Citation

If you find LongVT useful for your research and applications, please cite using this BibTeX:

@misc{yang2025longvtincentivizingthinkinglong,
      title={LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Calling}, 
      author={Zuhao Yang and Sudong Wang and Kaichen Zhang and Keming Wu and Sicong Leng and Yifan Zhang and Bo Li and Chengwei Qin and Shijian Lu and Xingxuan Li and Lidong Bing},
      year={2025},
      eprint={2511.20785},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2511.20785}, 
}

License

This dataset is released under the Apache 2.0 License.