Qwen-Image-2.1-viggle-turbo — v0.1 (preview)

Built with Qwen. A 4-step distilled student of Qwen/Qwen-Image-2.1, trained by Viggle with Distribution Matching Distillation. It does both text-to-image and instruction-driven editing with 1–3 reference images in 4 transformer passes instead of 40, with no classifier-free guidance.

Two students are shipped — pick one:

• transformer/ — full fine-tuned transformer (bf16, 14.2 GB). Replaces the base transformer; exact, no adapter. This is what the demo Space runs by default and the one we currently recommend — in our qualitative comparison it edits more faithfully than the LoRA.
• Qwen-Image-2.1-viggle-turbo-4step-lora-r64.safetensors — LoRA adapter (rank 64, 340 MB) loaded on top of the base transformer at runtime. Smaller download, slightly weaker.

Status: v0.1 preview, work in progress — this release still falls short of the base model. On complicated image editing (multi-reference composition, face swaps, identity-preserving edits, instructions with several constraints) it is clearly worse than the 40-step base model. Text-to-image at 4 steps is usable. We are still working on it and will update this repository as the distillation improves; treat the current weights as a preview, not a replacement for the base model.

What's inside

transformer/                                             # full fine-tuned transformer (config.json + bf16 safetensors)
Qwen-Image-2.1-viggle-turbo-4step-lora-r64.safetensors   # the LoRA adapter (rank 64, alpha 64), diffusers key format, bf16
peft/                                                    # the same adapter in peft key format, F32 as trained
scheduler/scheduler_config.json                          # base scheduler config with shift_terminal: null
LICENSE, NOTICE, README.md

Both students are step-400 EMA checkpoints of their respective runs (the full fine-tune adds a low-frequency teacher anchor to the DMD objective). The LoRA is never merged into the transformer — merging into bf16 is lossy, loading it at runtime is exact. Text encoder, VAE and processor are not redistributed; they load from the base repo.

Install

pip install -U torch "transformers>=5.17,<6" accelerate safetensors peft pillow
pip install "git+https://github.com/huggingface/diffusers.git@80c7ed262aeffbeb43ef13ae04baeb9b84515a69"

QwenImage21Pipeline is not in a released diffusers yet, hence the pinned git install. peft is required.

Usage

Load the full fine-tuned transformer (recommended)

import torch
from diffusers import QwenImage21Pipeline, QwenImage21Transformer2DModel, FlowMatchEulerDiscreteScheduler

transformer = QwenImage21Transformer2DModel.from_pretrained(
    "Viggle/Qwen-Image-2.1-viggle-turbo", subfolder="transformer", torch_dtype=torch.bfloat16
)
pipe = QwenImage21Pipeline.from_pretrained("Qwen/Qwen-Image-2.1", transformer=transformer, dtype=torch.bfloat16)
pipe.scheduler = FlowMatchEulerDiscreteScheduler.from_pretrained(
    "Viggle/Qwen-Image-2.1-viggle-turbo", subfolder="scheduler"
)
pipe.to("cuda")

… or load the LoRA instead

pipe = QwenImage21Pipeline.from_pretrained("Qwen/Qwen-Image-2.1", dtype=torch.bfloat16)
pipe.load_lora_weights(
    "Viggle/Qwen-Image-2.1-viggle-turbo",
    weight_name="Qwen-Image-2.1-viggle-turbo-4step-lora-r64.safetensors",
)
pipe.scheduler = FlowMatchEulerDiscreteScheduler.from_pretrained(
    "Viggle/Qwen-Image-2.1-viggle-turbo", subfolder="scheduler"
)
pipe.to("cuda")

Do not load the LoRA on top of the fine-tuned transformer — they are separate students.

Text to image

image = pipe(
    prompt="A studio portrait of an old fisherman mending a net, warm rim light, 85mm.",
    height=1024,
    width=1024,
    num_inference_steps=4,
    true_cfg_scale=1.0,                                   # no CFG (also the default)
    generator=torch.Generator("cuda").manual_seed(0),
).images[0]
image.save("out.png")

Image editing (1–3 reference images)

from diffusers.utils import load_image

image = pipe(                                             # same pipe object as above
    prompt="Replace the background with a sunset beach, keep the subject unchanged.",
    image=[load_image("input.png")],                      # list; order fixes <image1>, <image2>, ...
    output_resolution=1024,
    num_inference_steps=4,
    true_cfg_scale=1.0,
    generator=torch.Generator("cuda").manual_seed(0),
).images[0]

Rules that matter

• num_inference_steps=4, true_cfg_scale=1.0, no negative prompt. The adapter was distilled for exactly this; more steps or CFG do not help.
• Use the shipped scheduler config (or FlowMatchEulerDiscreteScheduler.from_config(pipe.scheduler.config, shift_terminal=None)). The base config's shift_terminal: 0.02 would wreck the last of the four steps. Do not pass sigmas= — the pipeline shifts them again.
• LoRA flavour: leave the LoRA scale at 1.0 (alpha equals rank).
• Reference-image order determines which image image 1 / image 2 in the prompt refers to. Without height/width, the output aspect ratio follows the last reference. Up to 3 references were used in training.
• Prompt rewriting is optional and was not used in training; the official PE-T2I / PE-I2I rewriters still apply and generally help composition and rendered text.
• peft users can load peft/ directly: pipe.transformer.load_lora_adapter("Viggle/Qwen-Image-2.1-viggle-turbo", subfolder="peft", weight_name="adapter_model.safetensors", prefix=None) — same weights, different key names; pick one path, not both.

Resolutions

Training sizes follow the base model's calculate_dimensions(area, aspect_ratio) rounded to a multiple of 32. Text-to-image was trained at 1024² and 2048² area, editing at 1024² and 1536² area (references always encoded at 1024² area). Other sizes work; these are where the training mass sits.

Known limitations

Overall the student still falls short of the base model; the gap is obvious on complicated editing, while text-to-image is usable. Specifically:

• Complicated edits degrade: multi-reference composition, face swaps and identity-document edits can produce duplicated or ghosted figures; "keep everything the same" requests drift in identity.
• Small or long rendered text can garble more often than with the 40-step base model.
• 2K output is not validated against the teacher.
• Untested at 4 steps: RGBA output, more than 3 reference images, mask-based local editing.
• Evaluation so far is qualitative, on held-out user requests; no quantitative metric is claimed.

License

This model is a derivative work of Qwen-Image-2.1 and is distributed under the Qwen RESEARCH LICENSE AGREEMENT (LICENSE): non-commercial use only — research or evaluation purposes. Commercial use requires a separate licence from the licensor (model-business@notice.qwencloud.com). See NOTICE for the required attribution.

Qwen is licensed under the Qwen RESEARCH LICENSE AGREEMENT, Copyright (c) 2026 Hangzhou Tongyi Laboratory Technology Co., Ltd. All Rights Reserved.

Relative to Qwen/Qwen-Image-2.1 this repository ships a modified transformer (transformer/, full fine-tune), adds a LoRA adapter and a scheduler config with shift_terminal changed from 0.02 to null; text encoder, VAE and processor are not redistributed.

Distillation method: DMD2 (Yin et al., 2024) / SenseFlow-style distribution matching. Distillation and release by Viggle. Built with Qwen.