h3-contact-sheet — Contact-Sheet diffusion for MiniMax-H3

MiniMax-H3 five-view (ref2va) training as a standalone, drop-in ai-toolkit extension. Zero edits to ai-toolkit core.

What it trains

One reference image + one orbit instruction → five coherent, progressively rotated views, trained as a 5-position latent set on top of MiniMax-H3's ref2va partition:

• the target is [B, 24, 5, h, w]: five independent one-frame VAE encodes concatenated on the (latent) time axis — never a video encode, which would land on the 17n+5 grid and produce 2 latents instead of 5;
• one clean image-ref latent enters as ref-block condition rows at an earlier integer RoPE time cursor, pinned at timestep 0.999, noise-augmented with the released recipe from a per-sample request seed, and stripped from the prediction before the loss;
• all five views are jointly denoised in a single packed sequence and independently decodable (frame bookkeeping fixed at num_frames = 17, 28 silent audio latents — see five_view_forward.py);
• audio rides along as noised silence with zero loss; the fl2va keyframe / i2v path is hard-asserted OFF.

Training previews decode all five slots independently and save one horizontal contact sheet [ref | slot0..slot4].

The full dataset/batch interface is specified in BATCH_CONTRACT.md.

Install

Clone (or symlink) this repo into ai-toolkit's extensions/ directory under a Python-identifier name:

cd /path/to/ai-toolkit
git clone <this repo> extensions/minimax_h3_five_view
# or: ln -s /path/to/h3-contact-sheet extensions/minimax_h3_five_view

That is the whole install. ai-toolkit's stock extension scan (toolkit/util/get_model.py::get_all_models and toolkit/extension.py::get_all_extensions both iterate extensions/) picks up this package's AI_TOOLKIT_MODELS (model arch minimax_h3_five_view) and AI_TOOLKIT_EXTENSIONS (process uid five_view_trainer).

Requirements: none beyond ai-toolkit itself. The base minimax_h3 extension this builds on ships built-in with ai-toolkit (extensions_built_in/diffusion_models/minimax_h3). ref_packing.py (ref-block packed-sequence geometry) is not upstream yet, so this package carries its own copy.

Run

cp extensions/minimax_h3_five_view/config/train_five_view_example.yaml config/my_run.yaml
# edit the <placeholder> paths, then:
python run.py config/my_run.yaml

The example config uses job: extension with process type: 'five_view_trainer' and model arch: "minimax_h3_five_view".

Inference — sheets from a trained LoRA (full walkthrough)

If you're coming from ComfyUI: this is a Python command-line tool, not a ComfyUI workflow. There is no node graph — you run one command and get PNGs. A ComfyUI custom node is planned; the LoRA file alone dropped into ComfyUI's loras/ folder will NOT make sheets (stock samplers can't do the five-slot packing; applying it to normal video generation actively degrades motion).

Hardware you need (honest numbers, measured on an RTX PRO 6000 running at a 450 W power cap — the card's stock limit is 600 W, so an uncapped card runs faster than these numbers):

That VRAM floor is the quantized base model — it does not shrink with image size, so you currently need a ≥48 GB card (A6000/6000-Ada/ RTX PRO 6000/A100/H100 class). Disk: ~43 GB of base weights download automatically on first run (21 GB DiT + 15.7 GB text encoder + 5.8 GB VAEs, all from the public Comfy-Org repackage — no token or login needed) plus the 63 MB LoRA you download yourself.

Step 1 — install ai-toolkit (this extension runs inside it):

git clone https://github.com/ostris/ai-toolkit
cd ai-toolkit
python3 -m venv .venv && .venv/bin/pip install -r requirements.txt

Step 2 — install this extension. The folder name matters (it must be a valid Python identifier, exactly as below):

git clone https://github.com/matlowai/h3-contact-sheet \
    extensions/minimax_h3_five_view

Step 3 — download a LoRA from matlod/minimax-h3-turnaround (or this repo's release assets). Start with minimax_h3_five_view_512_s1500.safetensors.

Step 4 — run (from the ai-toolkit root, using its venv):

.venv/bin/python extensions/minimax_h3_five_view/inference.py \
    --lora /path/to/minimax_h3_five_view_512_s1500.safetensors \
    --ref  /path/to/my_character.png \
    --out  out/ \
    --size 1024

First run stalls for a while at "Loading..." — that's the 43 GB download plus a cold model load; subsequent runs go straight to sampling. Output:

out/my_character_contact_sheet.png   <- [reference | view0 .. view4] strip
out/my_character_slot0.png .. slot4.png   <- the five views, full size
out/my_character_ref.png             <- the reference as conditioned on

Flags that matter:

• --size 512|1024|2048 — per-view resolution. 512 is the training size and fastest; 1024 is the sweet spot; 2048 works but the rotation arrives mostly in the later views (measured, expected).
• --strength — 1.0 (default) favors rotation; 0.7 favors scene fidelity / subtlety. This is the main quality dial.
• --prompt — default is the plain orbit instruction. Add ", neutral studio background, constant lighting" and switch to the _s400_instruct LoRA if you want the background replaced with a studio sweep instead of reproducing the reference's scene.
• --ref — any image size; it is resized internally. Subjects photographed square-on frontal rotate most predictably (the rotation axis follows the subject's facing direction — see the model card's limitations).
• --seed — sheets are deterministic per seed; reroll for a different take.

Troubleshooting:

• CUDA out of memory at load → the 48 GB floor is real; there is no low-VRAM mode yet.
• Long silent pause on first run → it's downloading ~43 GB; watch ~/.cache/huggingface. No HF account is required.
• ModuleNotFoundError: extensions.minimax_h3_five_view → the clone in step 2 isn't named exactly minimax_h3_five_view, or you're not running from the ai-toolkit root.
• WARNING unconsumed weights: [...] after loading the LoRA → wrong file (that message means keys didn't match this architecture — is it one of the five-view LoRAs from step 3?).
• Views barely rotate → try --strength 1.0 (not lower), a frontal reference, and see the model card's under-rotation notes.

How it registers without core edits

The in-tree prototype needed three ai-toolkit core edits. Each is replaced by a stock mechanism:

FiveViewTrainer subclasses the stock SDTrainer; dataset construction is its only override.

Testing

CPU tests (run from the ai-toolkit root, or set AI_TOOLKIT_ROOT):

.venv/bin/python extensions/minimax_h3_five_view/testing/test_ref_packing.py           # numeric parity vs ComfyUI's PackedLayout (needs a ComfyUI checkout; COMFYUI_ROOT to override)
.venv/bin/python extensions/minimax_h3_five_view/testing/test_five_view_dry.py         # full forward-prep dry run, no transformer
.venv/bin/python extensions/minimax_h3_five_view/testing/test_five_view_dataset_logic.py        # manifest/cache-identity logic (needs a dataset manifest)
.venv/bin/python extensions/minimax_h3_five_view/testing/test_five_view_dataset_integration.py  # batch contract through the real config path (needs the latent cache)

GPU utilities (not part of CI-style verification): testing/test_five_view_cache_gpu.py (cache builder), testing/test_heldvideo_sanity.py (v1b held-video encode checks).

License

GPL-3.0-or-later. Copyright (C) 2026 MATLOWAI (see LICENSE).