SP-PE-Qwen3.5-35B-A3B

Structured-Prompt Prompt-Expansion model built on top of Qwen3.5-35B-A3B.
Takes a natural-language prompt (+ target aspect ratio) and emits a
Structured Prompt JSON suitable for a QwenImage-style text-to-image DiT.

Part of the Context-Scaling project.

• Paper: Scaling Properties of Text Conditioning in Visual Generation
• Project page: https://heheyas.github.io/context-scaling/
• Code: https://github.com/heheyas/context-scaling

Usage

Serve via vLLM:

vllm serve heheyas/SP-PE-Qwen3.5-35B-A3B \
    --quantization fp8 \
    --kv-cache-dtype fp8 \
    --language-model-only \
    --max-model-len 8192

Or load with transformers (bf16, needs ~70 GB):

from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained("heheyas/SP-PE-Qwen3.5-35B-A3B", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    "heheyas/SP-PE-Qwen3.5-35B-A3B",
    torch_dtype="bfloat16",
    device_map="auto",
    trust_remote_code=True,
)

Notes

• 16 safetensors shards, ~66 GB in bf16.
• The --language-model-only flag is used at serving time because we only need the text path for the PE task; the vision tower ships along for compatibility but is not exercised.