DistillAlign: Coordinating Mode Covering and Mode Seeking in Autoregressive Video Distillation

Jiaxing Li^1,2*, Kai Zou^1*, Cindy Zhou^1,3, Kaichen Huang^1,2, Junyao Gao¹, Zile Wang¹,
Yang Liu¹, Bin Liu¹, Bo An², Yangguang Li^1†

¹Riemann Dynamics    ²Nanyang Technological University    ³Wellington College, UK

_{^*Equal contribution    ^†Corresponding author}

Project Page · Paper · Code · Models

DistillAlign aligns and balances the mode-covering and mode-seeking objectives of the multi-stage video distillation pipeline — using only a 1.3B DMD teacher, it already surpasses baselines refined with a 14B DMD teacher.

DistillAlign-1p3b-25K

Pre-computed clean VAE latents generated by the Wan2.1-T2V-1.3B teacher, each paired with its text prompt. Intended for video-diffusion distillation (as regression / distribution-matching targets) and latent-space research.

Dataset summary

Latent shape [21, 16, 60, 104]

Video resolution: 480 x 832, ~81 frames.

Files

part0/00000.pt
part0/00001.pt
...

Each *.pt is torch.save-d as a single-entry dict: the key is the prompt string, the value is the clean video latent of shape [21, 16, 60, 104].

Usage

import torch

d = torch.load("part0/00000.pt", map_location="cpu")
prompt = list(d.keys())[0]          # text prompt
latent = d[prompt]                  # torch.FloatTensor, shape [21, 16, 60, 104]
print(prompt, latent.shape, latent.dtype)

Loading the whole set:

import torch, glob

for f in sorted(glob.glob("part*/*.pt")):
    d = torch.load(f, map_location="cpu")
    prompt, latent = next(iter(d.items()))
    # ... use (prompt, latent) as a (text, clean-latent) training pair

To decode a latent back to pixels, use the Wan2.1 VAE decoder (add back the batch dim if your decoder expects [B, T, C, H, W]).

A matching dataset generated by the Wan2.1-T2V-14B teacher is released at DistillAlign_14B_25K.

Citation

@misc{li2026distillalign,
  title         = {DistillAlign: Coordinating Mode Covering and Mode Seeking in Autoregressive Video Distillation},
  author        = {Li, Jiaxing and Zou, Kai and Zhou, Cindy and Huang, Kaichen and Gao, Junyao and Wang, Zile and Liu, Yang and Liu, Bin and An, Bo and Li, Yangguang},
  year          = {2026},
  eprint        = {2607.26811},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url           = {https://arxiv.org/abs/2607.26811}
}