Pretrained DiT decoders for TT-VidT (Decoupling the Temporal Axis for Efficient Motion-Centric Video Pretraining, NeurIPS 2026). Paper · Project page
Each TT-VidT encoder pretraining run initialises its decoder from one of
these; the training code downloads them automatically through DECODER_PRETRAINED,
e.g. "KBlueLeaf/TTVidT-decoders/pretrain_video_S_qknorm".
The decoders work in the 32x32x4 latent space of the frozen f8 frame VAE
KBlueLeaf/latentmaid-vae and are
conditioned on DINOv3 ViT-B/16 features (768-d).
Each decoder is <name>.safetensors plus <name>.json, which holds the decoder
config (decoder_config), the latent size and the latent statistics used in training.
pretrain_{imgnet,video}_{S,B,L}_<variant>:
| Size | Hidden | Layers | Heads |
|---|---|---|---|
| S | 768 | 12 | 12 |
| B | 1024 | 16 | 16 |
| L | 1152 | 28 | 16 |
| Variant | Meaning |
|---|---|
qknorm | QK-norm attention, diffusion head |
qknorm_nofinal | as above, without the final norm |
qknorm_reg, qknorm_nofinal_reg | latent-regression head instead of diffusion |
768d12l, 1024d16l, 1152d28l (_80k) | earlier runs without QK-norm |
The ones used in the paper's configs:
| Decoder | Used by |
|---|---|
pretrain_imgnet_S_qknorm | objective / encoder sweep |
pretrain_video_S_qknorm | TT-VidT |
pretrain_imgnet_S_qknorm_nofinal | DisMo-2D3D with dual augmentation |
pretrain_{imgnet,video}_B_qknorm_nofinal, pretrain_{imgnet,video}_L_qknorm | decoder ablation |
Recipe: 100k steps (80k for _80k), global batch 256, AdamW lr 1e-4. See the
code repository for training and usage.
Pretrained DiT decoders for TT-VidT (Decoupling the Temporal Axis for Efficient Motion-Centric Video Pretraining, NeurIPS 2026). Paper · Project page
Each TT-VidT encoder pretraining run initialises its decoder from one of
these; the training code downloads them automatically through DECODER_PRETRAINED,
e.g. "KBlueLeaf/TTVidT-decoders/pretrain_video_S_qknorm".
The decoders work in the 32x32x4 latent space of the frozen f8 frame VAE
KBlueLeaf/latentmaid-vae and are
conditioned on DINOv3 ViT-B/16 features (768-d).
Each decoder is <name>.safetensors plus <name>.json, which holds the decoder
config (decoder_config), the latent size and the latent statistics used in training.
pretrain_{imgnet,video}_{S,B,L}_<variant>:
| Size | Hidden | Layers | Heads |
|---|---|---|---|
| S | 768 | 12 | 12 |
| B | 1024 | 16 | 16 |
| L | 1152 | 28 | 16 |
| Variant | Meaning |
|---|---|
qknorm | QK-norm attention, diffusion head |
qknorm_nofinal | as above, without the final norm |
qknorm_reg, qknorm_nofinal_reg | latent-regression head instead of diffusion |
768d12l, 1024d16l, 1152d28l (_80k) | earlier runs without QK-norm |
The ones used in the paper's configs:
| Decoder | Used by |
|---|---|
pretrain_imgnet_S_qknorm | objective / encoder sweep |
pretrain_video_S_qknorm | TT-VidT |
pretrain_imgnet_S_qknorm_nofinal | DisMo-2D3D with dual augmentation |
pretrain_{imgnet,video}_B_qknorm_nofinal, pretrain_{imgnet,video}_L_qknorm | decoder ablation |
Recipe: 100k steps (80k for _80k), global batch 256, AdamW lr 1e-4. See the
code repository for training and usage.
