ShotPlan-Wan2.1-T2V-14B

Full-parameter fine-tune of Wan2.1-T2V-14B for ShotPlan: Cinematic Video Generation with Learnable Planning Token — controllable multi-shot video generation with hard cuts at user-specified frame indices.

• 🌐 Project Page: https://pensioner-11.github.io/ShotPlan/
• 📄 Paper: arXiv:2607.17675
• 💻 Code (required to run): https://github.com/Pensioner-11/ShotPlan
• 🗂️ Training data: Pensioner/shotplan
• 🤝 Companion model: ShotPlan-Wan2.2-T2V-A14B-HighNoise

What's inside

ShotPlan-Wan2.1-T2V-14B.safetensors — full state dict of the fine-tuned DiT (step 3500, the checkpoint evaluated in the paper), containing:

• all standard Wan2.1-T2V-14B DiT weights, and
• hardcut_embedding [1, 1, 5120] — the learnable planning token.

Usage

This checkpoint requires the ShotPlan inference code (planning-token injection + fractional temporal RoPE). It is not a drop-in diffusers checkpoint.

git clone https://github.com/Pensioner-11/ShotPlan.git && cd ShotPlan
pip install -r requirements.txt

huggingface-cli download Wan-AI/Wan2.1-T2V-14B --local-dir ./models/Wan2.1-T2V-14B
huggingface-cli download Pensioner/ShotPlan-Wan2.1-T2V-14B --local-dir ./models/shotplan_wan21

python inference/infer_wan21.py \
    --model_root ./models/Wan2.1-T2V-14B \
    --ckpt ./models/shotplan_wan21/ShotPlan-Wan2.1-T2V-14B.safetensors \
    --prompt "Global scene description. Shot 1: ... Shot 2: ..." \
    --cut_at 40 \
    --output_dir ./results

--cut_at takes comma-separated frame indices (81-frame video @ 16 fps). Internally the script registers hardcut_embedding on the DiT, loads this state dict, and injects one planning token per cut at fractional RoPE coordinate t = 1 + frame/4.

Training

Fine-tuned from Wan2.1-T2V-14B on 6.4K multi-shot samples (480×832, 81 frames) for 3,500 steps with learning rate 1e-5 (AdamW), batch size 1 per GPU on 8 GPUs. The planning token and DiT weights are optimized jointly. See the code repository for the exact training script.

License

Apache-2.0, inherited from Wan2.1.

Citation

@article{guo2026shotplan,
  title={ShotPlan: Cinematic Video Generation with Learnable Planning Token},
  author={Guo, Su and Liu, Guangce and Yang, Haosen and Wang, Jiepeng and Liu, Cong and Liu, Junqi and Huang, Haibin and Yao, Hongxun and Zhang, Chi and Li, Xuelong},
  year={2026}
}