ShotPlan Training Dataset

Multi-shot video training data for ShotPlan: Cinematic Video Generation with Learnable Planning Token.

• 💻 Code: https://github.com/Pensioner-11/ShotPlan
• 🤖 Models: ShotPlan-Wan2.1-T2V-14B · ShotPlan-Wan2.2-T2V-A14B-HighNoise

Contents

Each sample is an 80-frame (5 s @ 16 fps) window over temporally contiguous shots of a source video, containing 1–3 hard cuts.

Metadata format

data/train_meta_16fps.json is a JSON list. Each record:

{
  "file_path": "videos/V000001_16fps.mp4",
  "start_frame": 102,
  "end_frame": 182,
  "cut_at": [26, 64],
  "type": "hardcut",
  "text": "Global caption ... Shot 1: <shot caption> Shot 2: <shot caption> ..."
}

Curation pipeline

Starting from the open-source VidEvent dataset (movie recap videos with naturally dense shot transitions):

1. Standard preprocessing (subtitle and border removal), re-encoding to 16 fps.
2. Shot boundary detection with TransNet V2, segmenting each video into shots.
3. Sliding-window extraction of 5-second windows over contiguous shots (≥2 shots per window, ≥20 frames per shot).
4. Semantic filtering with Gemini 2.5: only windows depicting a consistent event with stable scene context and subject continuity are kept.
5. Hierarchical captioning with Gemini 2.5: one global description (environment, subjects, lighting) plus per-shot captions (action, camera viewpoint, motion).

Intended use

Training and evaluation of multi-shot / shot-transition-controllable video generation models. This is the dataset used to train the ShotPlan checkpoints listed above.

License and provenance

The videos are derived from the VidEvent dataset (Liang et al., 2025), which is publicly released for research at http://www.videvent.top. This derived dataset is distributed for non-commercial research purposes only, inheriting the terms of the original VidEvent release. If you use this data, please cite both ShotPlan and VidEvent, and comply with VidEvent's usage terms.

We sincerely thank the VidEvent authors for making their dataset publicly available.

@article{liang2025videvent,
  title={VidEvent: A Large Dataset for Understanding Dynamic Evolution of Events in Videos},
  author={Liang, Baoyu and Su, Qile and Zhu, Shoutai and Liang, Yuchen and Tong, Chao},
  journal={arXiv preprint arXiv:2506.02448},
  year={2025}
}