Qwen-Image-2.1-PE-T2I — Heretic — NVFP4 (mixed precision, vLLM)

Not affiliated with, or endorsed by, Alibaba / Qwen. Community derivative of Qwen/Qwen-Image-2.1-PE-T2I, redistributed under the Qwen Research License (copy included as LICENSE, per §3.a). Non-commercial use only; commercial use needs a separate licence from Qwen.

Mixed-precision NVFP4 + FP8 build of pottokao/Qwen-Image-2.1-PE-T2I-Heretic — the Qwen-Image-2.1 prompt rewriter, refusals removed (3/100 from 98/100, KL 0.036 — best of a 1,600-trial search; strictly dominates the earlier 7/100 @ 0.039, tag v1-trial384). 7.76 GB, from 17.5 GB bf16.

system_prompt.txt is included and the model is useless without it. That 10 KB document is the output contract — structure, register, length, and the JSON shape.

Ready-made workflow: PE-T2I Photo Style Prompt

This rewriter is what drives ComfyUI-QwenImage-PhotoStyles — a small ComfyUI node with 17 photographic styles (camera language + mood) that turns one short prompt into a full styled prompt. The node's 68-image gallery, and the showcases on the DiT-NVFP4 repo, were all produced with this rewriter. If you're interested in how those images are made, that's the place to look.

⚠️ For vLLM, not ComfyUI

"NVFP4" names a numeric format, not a file layout:

For ComfyUI use the GGUF build through an LLM node. Needs Blackwell (sm_120+) for the native FP4 path.

The recipe

Not NVFP4_DEFAULT_CFG. This follows NVIDIA's published mixed-precision recipe for the same model generation (nvidia/Qwen3.6-27B-NVFP4, quant_algo: MIXED_PRECISION):

Attention stays at FP8, not 4-bit. NVFP4_DEFAULT_CFG would take it to W4A4 — weights and activations at 4-bit. No NVIDIA checkpoint for this family does that.

The embedding is never touched. It is absent from the quantized-layer list of the official mixed recipe and from exclude_modules of every plain-NVFP4 checkpoint in the family — modelopt disables nn.Embedding at the class level, so it never appears.

Three traps, if you build this yourself

These cost me a whole afternoon; they are all silent.

1. export_hf_checkpoint does not carry processor_config.json. This model is a VL architecture, so vLLM builds a multimodal processor at startup and dies with OSError: Can't load image processor before it ever touches the weights. Copy the file in from the source. Same for system_prompt.txt.

2. Don't swap FP8_DEFAULT_CFG for FP8_PER_CHANNEL_PER_TOKEN_CFG. The per-token dynamic variant is tempting because it needs no calibration set. It exports as quant_algo: FP8_PER_CHANNEL_PER_TOKEN, and vLLM then fails during weight loading with AttributeError: 'MergedColumnParallelLinear' object has no attribute 'data' — the fused gate/up projection never gets its quantized parameters created. Use W8A8 with a real calibration pass.

3. Verify before exporting, not after. Checking tensor dtypes in the finished file tells you the shapes are right, not that the recipe was applied. Scan every TensorQuantizer before export and assert three things: nothing forbidden is enabled (conv1d, in_proj_a/b, vision tower, embeddings, norms), everything enabled matches the intended algorithm and role, and the counts are complete. In this build that gate checks 97 NVFP4 modules / 104 FP8 modules / 104 FP8 input scales.

Run it

vllm serve pottokao/Qwen-Image-2.1-PE-T2I-Heretic-NVFP4 \
  --max-model-len 16384 --port 8000

The system prompt is 2,426 tokens, so a request costs roughly 2,426 + prompt + ~2,000 output ≈ 4,500 tokens. 36 K of KV cache holds about 8 concurrent requests. Prefix caching helps a lot here — the first 2,426 tokens are identical on every call.

import json, requests
sp = open("system_prompt.txt").read().strip()
r = requests.post("http://localhost:8000/v1/chat/completions", json={
    "model": "pottokao/Qwen-Image-2.1-PE-T2I-Heretic-NVFP4",
    "messages": [{"role": "system", "content": sp},
                 {"role": "user", "content": "夕陽下的燈塔"}],
    "max_tokens": 4096, "temperature": 0.3})
gen = r.json()["choices"][0]["message"]["content"]
print(json.loads(gen.partition("</think>")[2].strip()))

Use max_tokens ≥ 4096. At low temperature the model writes to the full length the system prompt asks for, and 2048 truncates it mid-JSON.

Verified

• Gate, pre-export — 97 / 104 / 104, all seven module-group assertions pass.
• Text, 3/3 — strictly valid JSON with exactly rewritten_prompt + wh_ratio, 2795–3229 characters. The bf16 source gives 2450–3320 on the same inputs, so the length discipline survives.
• Vision tower — fed an image, returned an accurate description. This is why the tower is left in BF16.
• End to end — driving a full Qwen-Image-2.1 NVFP4 pipeline (this rewriter → NVFP4 text encoder → NVFP4 DiT), producing finished images with legible rendered Japanese signage. Text rendering is the first thing aggressive quantization breaks; it held.

Not verified: a numeric quality comparison against the bf16 source.

Showcase

Outputs from the full pipeline this rewriter drives (rewriter → Qwen-Image-2.1 text encoder → DiT) — the image engine behind a "photography masters" system that originally ran on DiffusionGemma and has since been moved onto this pipeline. Each was generated from a one-line prompt; the rewriter expanded it into the scene, palette and framing below. One seed per cell, no retouching.

_{AI-generated. Style homage only — not affiliated with or endorsed by the photographers.}

Other builds