Viggle Turbo v0.1 (preview) — 4-step Qwen-Image-2.1

A DMD-distilled student of Qwen-Image-2.1 that generates and edits images in 4 sampling steps with no classifier-free guidance, against the teacher's 40 steps.

One model does both tasks, exactly as the base does: leave the reference slots empty for text-to-image, or fill one to three of them for editing / composition / style transfer.

The Examples table below the form is pre-rendered: every row carries the output this exact app produced for it (fixed seed; 4 steps and prompt enhancement on unless the row's status line says otherwise — the launch-graphic row is 8 steps at the 3:2 2048² size with the 3k-character brief sent as written), so you can see what the model does without spending GPU time — click a row to load the prompt, the references, the result, and the steps / size / enhancement settings that produced it. release/render_examples.py regenerates the rows and examples/manifest.json whenever the weights change. The reference photos woman1/woman2/cat/cat_window/bird.webp and the three-reference prompt come from the black-forest-labs/flux-klein-9b-kv Space.

Built with Qwen. Distilled from Qwen/Qwen-Image-2.1.

Status: v0.1 preview, work in progress — this release still falls short of the base model. On complicated image editing (multi-reference composition, face swaps, identity-preserving edits, instructions with several constraints) it is clearly worse than the 40-step base model. Text-to-image at 4 steps is usable. We are still working on it and will update this repository as the distillation improves; treat the current weights as a preview, not a replacement for the base model.

What the app loads

The model repo holds two students; the Space runs the full fine-tuned transformer by default:

transformer = QwenImage21Transformer2DModel.from_pretrained(STUDENT_REPO, subfolder="transformer", torch_dtype=torch.bfloat16)
pipe = QwenImage21Pipeline.from_pretrained(BASE_MODEL_ID, transformer=transformer, dtype=torch.bfloat16)

transformer/ is a full-parameter fine-tune of the base transformer (bf16, 14.2 GB); passing it as a component means the base transformer is never downloaded, so memory and download size are the same as for the base model. STUDENT=lora switches to the r64 LoRA at the repo root, loaded at runtime with pipe.load_lora_weights on top of the base transformer and never merged (merging into bf16 keeps only ~47 % of the adapter delta). The status line under each result names the student in use.

Environment: STUDENT (full, default, or lora), STUDENT_REPO (default Viggle/Qwen-Image-2.1-viggle-turbo; LORA_REPO is still accepted), BASE_MODEL_ID (default Qwen/Qwen-Image-2.1), HF_TOKEN. HF_TOKEN is required while the model repo is private — add it as a Space secret with read scope; it can be removed once the repo is public.

The 4-step contract

The student is trained against a specific schedule, and the demo reproduces it exactly:

• 4 Euler steps, raw sigma nodes linspace(1, 1/4, 4) — the pipeline's own default for num_inference_steps=4, so no sigmas= argument is passed. (Passing sigmas= would not be a way to bypass the shift: the pipeline shifts whatever it is given.) The Steps slider lets you try 3–8 (num_inference_steps passed through unchanged, same shift rule); the student was distilled at 4 and other counts are off-contract — more steps do not mean better here.
• Resolution-dependent time shift only. mu = calculate_shift((H/16)*(W/16)) with the base scheduler's base_image_seq_len=256, max_image_seq_len=8192, base_shift=0.5, max_shift=0.9.
• shift_terminal must be null. The stock Qwen-Image-2.1 scheduler ships shift_terminal: 0.02, which stretches the final sigma node to 0.02 instead of 0 and costs the last step. app.py rebuilds the scheduler from the base config with the key cleared, so the contract holds no matter which repo BASE_MODEL_ID points at: pipe.scheduler = FlowMatchEulerDiscreteScheduler.from_config(pipe.scheduler.config, shift_terminal=None)
• No CFG. true_cfg_scale=1.0 and no negative prompt. (Qwen-Image-2.1 has no guidance_scale argument at all; CFG needs both true_cfg_scale > 1 and a negative prompt.)
• Condition images are encoded at 1024-area (output_resolution=1024), matching distillation.

Verified against dmd_common.student_sigmas(..., terminal="none") at every size the app offers: max absolute gap 0.0. Nodes at 1024×1024: [1.0, 0.857192, 0.666756, 0.400096, 0.0]; at 2048×2048: [1.0, 0.917701, 0.787998, 0.553369, 0.0].

Output sizes

Text-to-image and editing were distilled on different target buckets, so the size menu follows the mode and rewrites itself as soon as a reference image is attached:

Every entry is calculate_dimensions(area, ratio) rounded to a multiple of 32 — the pipeline's own rule — for 1:1, 16:9, 9:16, 4:3, 3:4, 3:2, 2:3. Auto takes the aspect ratio of the last reference at 1024² area, which is what the pipeline does when height/width are left unset. generate() additionally snaps any editing call whose size is not one of the offered editing dimensions onto the 1536² bucket of the nearest aspect ratio, and says so in the status line, so an API client or a stale dropdown cannot push a three-reference call out of budget.

Reference images are always resized to 1024² area before encoding regardless of the output size, so extra references cost VRAM and time but not output resolution.

Prompt enhancement

Enhance prompt (on by default) rewrites the prompt before generation with the system prompts published with Qwen/Qwen-Image-2.1-PE-T2I and -PE-I2I, run on the pipeline's own text encoder: Qwen-Image-2.1 ships the complete Qwen3-VL-8B-Instruct, LM head included, so the rewrite costs no extra weights. Decoding is greedy (a given prompt always rewrites the same way), capped at 512 new tokens. Text-to-image prompts become a 200–300 token scene paragraph and the rewriter is told the chosen aspect ratio; editing instructions become an explicit instruction that spells out what to keep, in the instruction's language, with the reference images shown to the rewriter at 1024² area. The rewritten prompt is displayed under Prompt sent to the model; if the reply does not parse, the original prompt is used and the info line says so.

Two caveats. The official rewriters are separate Qwen3.5-VL 9B fine-tunes (~18 GiB) that would not fit next to the pipeline on the 48 GB card and are hosted behind an API in Qwen's own Space; what runs here is the base encoder prompted with the same system prompts — same output format, not the same quality. And the student was distilled on raw user prompts, so enhancement is a convenience layered on top of the recipe, not part of it: untick it when the wording matters (exact text to render, deliberately terse prompts).

Hardware

Weights are ~30.6 GiB resident in bf16 (13.25 transformer + 16.33 text encoder + 0.63 VAE; the LoRA flavour adds the r=64 adapter). Measured on one B200, around a single call including VAE encode/decode, with the allocator cache dropped before each measurement:

Above the 1024² bucket the peak is the VAE decode, not the denoising loop (33–40 GiB at every size). With vae.enable_tiling() the decode never rises above the denoising peak, so the largest call on the menu is a three-reference edit at 40.2 GiB allocated, inside the 48 GB (44.7 GiB) of ZeroGPU large. app.py therefore requests @spaces.GPU(duration=90, size="large") at 1× quota and switches tiling on only for the 1536² and 2048² buckets: 1024²-bucket outputs are bit-identical to the untiled pipeline, tiled outputs differ from untiled ones by 1.6/255 mean absolute (measured at 2048×2048). The 45.96 GiB reserved figure is what the caching allocator held on a card with room to spare; on the 44.7 GiB card it releases cached blocks before failing, so the allocated peak is the binding number, with ~4 GiB of margin on the largest call. Prompt enhancement runs before the denoise with nothing else live (~2 GiB of KV cache over the resident weights) and does not raise the peak. Wall-clock on the ZeroGPU card (half an RTX Pro 6000 Blackwell) will be slower than the B200 numbers above, and the prompt rewrite adds 4–11 s on a B200; duration=90 covers both.

Cold start is dominated by the download: ~30.9 GiB of base safetensors from Qwen/Qwen-Image-2.1 (13.25 transformer + 16.33 text encoder + 1.26 VAE + 0.02 processor) plus 0.32 GiB for the LoRA — the shipped root adapter is bf16, and load_lora_weights(weight_name=…) fetches that single file, not the F32 peft/ copy — then ~30–40 s to load and pack the pipeline onto the GPU. Only the adapter changes between releases, so a cached Space restarts far faster than it first boots.

Alpha channels

gr.Image(..., image_mode="RGB") drops any alpha channel on upload. The pipeline then calls img.convert("RGBA") on every condition image itself — the Qwen-Image-2.1 VAE genuinely has in_channels=4 — so what it actually sees is your image with a fully opaque alpha. A transparent PNG is therefore not composited onto a background: PIL's RGBA→RGB keeps the raw colour of transparent pixels (often black), and those colours become visible content. Flatten transparent images onto the background you want before uploading.

The vision patch-embed workaround

app.py rebinds Qwen3VLVisionPatchEmbed.forward to a matmul. That module applies an nn.Conv3d whose kernel_size equals its stride, which is exactly a linear map over each flattened patch, but cuDNN has no usable bf16 Conv3d kernel for the shape: it costs 29.9 s per reference image in this pipeline (356 ms in isolation with memory to spare; the same conv in fp32 is 2.9 ms). The matmul is 0.8 ms and its output is bitwise equal to the conv's — all verification images are unchanged byte-for-byte. Without it, a 1-reference edit takes 32.3 s and a 2-reference edit 63.1 s, which is why @spaces.GPU(duration=90) is enough only with the fix in place. Remove it if transformers ever ships a fast path here.

Dependencies

QwenImage21Pipeline is not in any released diffusers — 0.40.0, the latest on PyPI, contains no qwenimage21 files. requirements.txt therefore installs diffusers from git at a pinned commit (80c7ed262aeffbeb43ef13ae04baeb9b84515a69, upstream main, 2026-09-20), and pins transformers==5.17.0 (the pipeline installs a forward hook on the text encoder's final RMSNorm to undo the transformers>=5.0 hidden-state tying; a different major may break it), plus accelerate==1.15.0, peft==0.18.0 and spaces==0.51.3 — the versions this release was validated against. peft is required: without it load_lora_weights has no runtime to attach the adapter to. torch and gradio are not listed: sdk_version above owns the Gradio version, and the ZeroGPU image provides torch (its docs list 2.8.0–2.13.0 as supported, which includes the 2.9.1 this release was validated on). That last point is an assumption about the image, not a measurement — check the first build log and pin torch==2.9.1 if it resolves something outside that range.

License

The base model is released under the Qwen RESEARCH LICENSE AGREEMENT — Non-Commercial: research or evaluation purposes only (§1.i, §2.a). This distilled derivative and this demo inherit that restriction. Commercial use requires a separate license from Alibaba (model-business@notice.qwencloud.com). See NOTICE for the attribution required by §3.c.

Qwen is licensed under the Qwen RESEARCH LICENSE AGREEMENT, Copyright (c) 2026 Hangzhou Tongyi Laboratory Technology Co., Ltd. All Rights Reserved.