ParaVT-Parquet

This repository hosts the training annotations for the ParaVT project: a 97 K-row cold-start SFT corpus and a 4406-row diverse RL corpus. The source media files live in ParaVT/ParaVT-Source; the two repos are designed for a one-step materialize pass (see Usage).

Overview

ParaVT is a multi-agent agentic framework for long-video understanding, post-trained with PARA-GRPO (Parseability-Anchored and Ratio-gAted GRPO). This dataset is the exact training corpus used to produce ParaVT/ParaVT-8B and its cold-start initialization mwxely/ParaVT-8B-SFT.

Path sanitization

For privacy reasons, the absolute media paths inside the parquets have been replaced with relative sentinel paths keyed to four virtual roots:

Materialize the sentinels back to absolute paths with paravt.data.materialize.

Dataset structure

sft config (cold-start; 97 K rows total)

rl config (4406 rows)

Usage

# 1. Download the parquets (~200 MB)
huggingface-cli download ParaVT/ParaVT-Parquet --repo-type dataset --local-dir ./paravt-parquet

# 2. Download the source videos + images and extract every zip into the same root.
#    Each archive's members carry the full sentinel path (e.g. `longvt_source/videor1_7/...`),
#    so the extraction target must be the top-level root.
huggingface-cli download ParaVT/ParaVT-Source --repo-type dataset --local-dir ./paravt-source
( cd ./paravt-source && find . -name "*.zip" -exec unzip -q -o -d . {} \; )

# 3. Materialize sentinel paths -> absolute file:// URIs (one shot)
python -m paravt.data.materialize \
    --root        ./paravt-source \
    --parquet-dir ./paravt-parquet \
    --output-dir  ./paravt-parquet-materialized

The materialized parquets are drop-in for the lmms-engine SFT recipe and the AReaL RL recipe shipped in ParaVT/paravt/sft and ParaVT/paravt/rl. For programmatic access via datasets:

from datasets import load_dataset
sft_videor1 = load_dataset("ParaVT/ParaVT-Parquet", "sft", split="videor1")
rl_train    = load_dataset("ParaVT/ParaVT-Parquet", "rl",  split="train")

(Run materialize first if you intend to read the videos; the raw downloaded parquets carry sentinel paths.)

Citation

@article{yang2026paravt,
  title={ParaVT: Taming the Tool Prior Paradox for Parallel Tool Use in Agentic Video Reinforcement Learning},
  author={Yang, Zuhao and Zhang, Kaichen and Wang, Sudong and Wu, Keming and Yang, Zhongyu and Li, Bo and Qi, Xiaojuan and Lu, Shijian and Li, Xingxuan and Bing, Lidong},
  journal={arXiv preprint arXiv:2605.20342},
  year={2026}
}

Acknowledgements

The SFT corpus reuses subsets of the LongVT training data (longvideotool/LongVT-Parquet); thanks to the LongVT authors for releasing the cleaned data. The MuSeG, Charades-STA, HACS, and Ego4D source clips are attributed to their respective original publications.