Qwen-Image-2.1-PE-T2I — Heretic — GGUF

Not affiliated with, or endorsed by, Alibaba / Qwen. Community derivative of Qwen/Qwen-Image-2.1-PE-T2I, redistributed under the Qwen Research License (copy included as LICENSE, per §3.a). Non-commercial use only; commercial use needs a separate licence from Qwen.

Q4_K_M build of pottokao/Qwen-Image-2.1-PE-T2I-Heretic — the Qwen-Image-2.1 prompt rewriter with refusal behaviour removed (3/100 refusals @ KL 0.036, from 98/100 — best of a 1,600-trial search; dominates the earlier 7/100 @ 0.039, tag v1-trial384).

system_prompt.txt is included and the model is useless without it. That 10 KB document is the output contract — structure, register, length, and the JSON shape. Load it as the system message.

Made with a simple node

The sample / showcase images in this repo were generated with the technique behind ComfyUI-QwenImage-PhotoStyles — a small ComfyUI node (17 photographic styles; one short prompt becomes a full styled prompt via the PE-T2I rewriter). If you're interested in how the images are made, the node and its recipe are there.

What this is for

A short request in any language goes in; one JSON line comes out with a detailed English image prompt and a recommended aspect ratio:

{"rewritten_prompt": "...", "wh_ratio": "3:2"}

It is an LLM, not a text encoder. In ComfyUI it belongs in an LLM/GGUF node that rewrites your prompt before it reaches the DiT — it does not load into CLIPLoader. For the Qwen-Image-2.1 text encoder, see …-Text-Encoder-Heretic.

Available quantizations

All load the same way (llama-server -m <file>); pick by available VRAM.

One deviation from stock Q4_K_M

token_embd is kept at Q6_K instead of Q4_K. Everything else is standard Q4_K_M (llama.cpp already promotes output to Q6_K on its own).

The reason: every official NVIDIA NVFP4 checkpoint for this model family leaves the token embedding untouched — across Qwen3-8B, Qwen3-32B, Qwen3-30B-A3B, Qwen3.5-122B-A10B and Qwen3.5-397B-A17B, and also in the one official mixed-precision recipe (Qwen3.6-27B), which quantizes lm_head to 4-bit but still does not touch the embedding. Stock Q4_K_M does compress it. Raising it back costs 0.25 GB and cannot be worse than leaving it at Q4_K.

Run it

llama-server -m pe_t2i_heretic-Q4_K_M.gguf --host 0.0.0.0 --port 8080 \
             -ngl 99 -c 16384 --jinja -a pe-t2i

Then POST to /v1/chat/completions with system_prompt.txt as the system message. Loads in a few seconds; fits on an 8 GB card.

Conversion

python convert_hf_to_gguf.py <bf16 model> --outfile pe_bf16.gguf --outtype bf16 --no-mtp
llama-quantize --token-embedding-type q6_K --output-tensor-type q6_K \
               pe_bf16.gguf pe_t2i_heretic-Q4_K_M.gguf Q4_K_M

--no-mtp is required. This model's config.json carries text_config.mtp_num_hidden_layers with the value null. Without the flag the converter does int += None and dies with a TypeError. The key existing with a null value is the trap — .get(key, 0) returns None, not the default.

What was checked, and what wasn't

Checked — it still does its job. Four requests (three Chinese, one English) through llama-server: 4/4 produced strictly valid JSON with exactly the rewritten_prompt / wh_ratio keys, 2053–3351 characters. The bf16 source produces 2450–3320 on the same style of input, so the length discipline survives quantization.

Not checked — a clean perplexity comparison. I ran one and it returned self-contradictory numbers (the unquantized bf16 GGUF scored worse than its own quantizations, which is not physically sensible), so I am not reporting it. Take the functional check above as the evidence, and nothing more.

Tried and not shipped — an imatrix build. I generated an importance matrix from in-domain data and quantized with it. It passed the same functional check, but the calibration corpus was badly designed: the 10 KB system prompt appears in every sample, so ~88% of the 341 KB corpus was one repeated document, which would dominate the importance statistics. Rather than ship a build I can't justify, it's left out. A properly diversified corpus would be the way to revisit this.

🖼️ Showcase

This is what Q4_K_M across the board looks like — every image above was made with:

_{AI-generated. Style homage only — not affiliated with or endorsed by the photographers.}

Showcase

Outputs from the full pipeline this rewriter drives (rewriter → Qwen-Image-2.1 text encoder → DiT) — the image engine behind a "photography masters" system that originally ran on DiffusionGemma and has since been moved onto this pipeline. Each was generated from a one-line prompt; the rewriter expanded it into the scene, palette and framing below. One seed per cell, no retouching.

_{AI-generated. Style homage only — not affiliated with or endorsed by the photographers.}

Other builds

Results, the Pareto front of other operating points, and the measurement caveats are in the base repo.