Qwen-Image-2.1 Text Encoder (Heretic) — GGUF · FP8 · bf16

[!IMPORTANT] Got got input of size [1, 512, 12288] with the GGUF? It's fixed. The GGUF files were fine all along — ComfyUI-GGUF just didn't load the Qwen3-VL vision tower. Install the add-on node ComfyUI-GGUF-Qwen3VL-TE, make sure the mmproj file sits next to the GGUF, restart ComfyUI. No need to re-download anything.

📥 Which file should I use?

All three are in this repo. Every loader uses type qwen_image and feeds TextEncodeQwenImage21.

⚙️ Recommended sampler settings

Qwen-Image-2.1 is a full (non-distilled) model. Use 25 steps — more steps don't make text sharper and start to burn the highlights.

• No text in the image: plain KSampler, cfg 1.0 (fastest, softest look; the negative prompt is ignored at cfg 1).
• Text in the image (signs, labels, posters): switch cfg partway through — the first ½–⅔ of the steps at cfg 1 lock in composition and materials, the rest at cfg 3 with a negative prompt redraws the lettering. You get cfg 1's look and cfg 3's readable text, in one pass with one seed.

In ComfyUI that's two chained KSamplerAdvanced nodes (same model, seed and steps):

Split later (17) keeps more of the cfg 1 look; split earlier (12) gives crisper text. Example negative prompt: oversaturated, overexposed, gibberish text.

GGUF in ComfyUI — 3 steps

1. Install the add-on (you also need city96/ComfyUI-GGUF):

   cd ComfyUI/custom_nodes
   git clone https://github.com/pottokao-dotcom/ComfyUI-GGUF-Qwen3VL-TE
   

2. Put both files in ComfyUI/models/text_encoders/

   • qwen3vl_8b_heretic-Q4_K_M.gguf
   • mmproj-qwen3vl_8b_heretic-f16.gguf ← the vision tower (required — also used for image editing)

   Don't rename either file — the vision tower is matched by name. If it's missing, the add-on stops with a Missing vision tower error telling you which file to download.

3. Restart ComfyUI, then CLIPLoaderGGUF → type qwen_image → TextEncodeQwenImage21 → the rest of the official workflow. The console should show [GGUF-Qwen3VL-TE] added 351 Qwen3-VL vision tensors from mmproj.

Verified 2026-09-23 on ComfyUI 0.36.0 + ComfyUI-GGUF 6ea2651 (NVIDIA GPU): text-to-image and reference-image editing both work and match the bf16 encoder's output for the same seed up to Q4 quantization noise. Not tested on a Mac.

RuntimeError: Given normalized_shape=[4096], expected input with shape [*, 4096],
but got input of size [1, 512, 12288]

Files in this repo

Other formats of the same model

Ablation (inherited from the bf16 source)

Produced with Heretic directional ablation (o_proj + down_proj), 200 trials / 60 startup trials, knee point of the Pareto front. Independently re-checked on the bf16 source: 0/20 refusals, 4/4 benign questions answered correctly. Full methodology in the bf16 repo.

Format details

GGUF: Q4_K_M language model, vision tower as a separate f16 mmproj.

FP8: self-quantized float8_e4m3fn (Comfy-Org doesn't ship an FP8 encoder):

Remapped to the ComfyUI key layout (model.layers.…, no language_model. prefix).

Requires a ComfyUI build with QwenImage21 support (0.36.0 or newer).

🎨 Also for Qwen-Image-2.1: the prompt rewriters (Heretic)

Qwen-Image-2.1 ships two prompt rewriters — small LLMs that expand a short request into the detailed prompt the model was trained on. Both are available refusal-ablated, and both have GGUF builds that run anywhere llama.cpp runs (Mac included):

A short request goes in; one JSON line comes out (real output of the PE-T2I GGUF, trimmed):

in:  雨夜的東京小巷，一隻流浪狗，森山大道風格
out: {"rewritten_prompt": "A vertical nighttime street photograph in the style of Daido Moriya shows a
      narrow, rain-soaked urban alley with a stray dog standing in the lower foreground. The scene is
      composed with strong depth: dark building walls and utility poles frame the left and right sides,
      while the wet pavement stretches into a softly blurred background. … (2,137 characters)",
      "wh_ratio": "2:3"}

Showcase — generated with the NVFP4 builds through the full pipeline (PE-T2I rewriter → Qwen-Image-2.1 text encoder → DiT), each from a one-line prompt, one seed per image, no retouching. Also shown in the PE-T2I NVFP4 repo.

Not affiliated with, or endorsed by, Alibaba / Qwen. Community derivative (refusal-ablated) of Qwen/Qwen3-VL-8B-Instruct — the model Qwen-Image-2.1 uses, unmodified, as its text encoder. Qwen releases that model under Apache-2.0, so this derivative is redistributed under Apache-2.0 (see LICENSE and NOTICE).