ShotPlan-Wan2.2-T2V-A14B-HighNoise

🌐 Project Page  ·  📄 Paper  ·  💻 Code

Fine-tuned high-noise expert of Wan2.2-T2V-A14B for ShotPlan: Cinematic Video Generation with Learnable Planning Token — controllable multi-shot video generation with hard cuts at user-specified frame indices.

• 💻 Code (required to run): https://github.com/Pensioner-11/ShotPlan
• 🗂️ Training data: Pensioner/shotplan
• 🤝 Companion model: ShotPlan-Wan2.1-T2V-14B

What's inside

ShotPlan-Wan2.2-T2V-A14B-HighNoise.safetensors — full state dict of the fine-tuned high-noise DiT (step 3000), containing:

• all standard Wan2.2 high-noise DiT weights, and
• hardcut_embedding [1, 1, 5120] — the learnable planning token.

Only the high-noise expert is fine-tuned: shot structure is decided early in the denoising trajectory. At inference the low-noise expert runs with the original Wan2.2 weights and the stock model function; a router injects planning tokens only into the high-noise expert.

Usage

This checkpoint requires the ShotPlan inference code (planning-token injection + fractional temporal RoPE). It is not a drop-in diffusers checkpoint.

git clone https://github.com/Pensioner-11/ShotPlan.git && cd ShotPlan
pip install -r requirements.txt

huggingface-cli download Wan-AI/Wan2.2-T2V-A14B --local-dir ./models/Wan2.2-T2V-A14B
huggingface-cli download Pensioner/ShotPlan-Wan2.2-T2V-A14B-HighNoise --local-dir ./models/shotplan_wan22

python inference/infer_wan22.py \
    --wan22_root ./models/Wan2.2-T2V-A14B \
    --ckpt ./models/shotplan_wan22/ShotPlan-Wan2.2-T2V-A14B-HighNoise.safetensors \
    --prompt "Global scene description. Shot 1: ... Shot 2: ..." \
    --cut_at 40 \
    --output_dir ./results

--cut_at takes comma-separated frame indices (81-frame video @ 16 fps). Internally the script registers hardcut_embedding on the high-noise DiT, loads this state dict, and injects one planning token per cut at fractional RoPE coordinate t = 1 + frame/4.

Training

Fine-tuned from Wan2.2-T2V-A14B high-noise expert on 6.4K multi-shot samples (480×832, 81 frames) with learning rate 1e-5, timestep boundary restricted to the high-noise segment (max_timestep_boundary 0.358). The planning token and DiT weights are optimized jointly. See the code repository for the exact training script.

License

Apache-2.0, inherited from Wan2.2.

Citation

@article{guo2026shotplan,
  title={ShotPlan: Cinematic Video Generation with Learnable Planning Token},
  author={Guo, Su and Liu, Guangce and Yang, Haosen and Wang, Jiepeng and Liu, Cong and Liu, Junqi ... Zhang, Chi and Li, Xuelong},
  journal={arXiv preprint arXiv:2607.17675},
  year={2026}
}