QwenDLM-4B-ORL

QwenDLM + ORL (4B): the block diffusion language model QwenDLM-4B, post-trained with On-Policy Residual Learning (ORL)

Highlights

• 6.2× tokens per forward (TPF) over QwenDLM-4B on HumanEval (11.1 vs 1.8), with higher accuracy on all four benchmarks
• 1,298 tokens/s for a single request on one H100 with dlmrt (mean of GSM8K and HumanEval): about 6.9× QwenDLM-4B on SGLang, and 1.74× DFlash speculative decoding at matched accuracy
• Bit-exact reproducible: the released training recipe reproduces this checkpoint byte for byte on the same hardware and runtime

Results

Accuracy (%) / TPF; each model uses its selected decoder.

Speed

Tokens per second for a single request on one H100 (BF16), with the paper's prompts and decoders:

At the same setting, the autoregressive Qwen3-4B-Thinking-2507 on vLLM decodes 240 tokens/s, 353 / 405 with EAGLE3 and 573 / 746 with DFlash (strict / relaxed verification). Strict verification keeps the accuracy of the autoregressive model, 2.4 points above QwenDLM-4B-ORL's 91.0%; relaxed verification uses the fastest setting at most 0.5 points below it.

Usage

Plain transformers generation does not run the paper's decoding. Two engines do: dlmrt, our inference engine, and the ORL repository's SGLang engine and decoder configuration, which the paper's evaluation uses.

dlmrt

dlmrt, our inference engine, decodes one request at a time with the paper's decoder (role="orl") and measures the paper's throughput; see its README for installation, tensor parallelism and the commands of every paper cell.

from dlmrt import LLM

llm = LLM("yifanyu/QwenDLM-4B-ORL", role="orl")
out = llm.generate("Write a Python function that reverses a linked list.", max_new_tokens=4096)
print(out.text)
print(f"{out.request_tps:.0f} tokens/s")

python -m dlmrt serve --model yifanyu/QwenDLM-4B-ORL --role orl --port 18080
curl -N localhost:18080/generate -d '{"messages": [{"role": "user", "content": "Hi!"}], "max_new_tokens": 512}'

The paper's evaluation pipeline

Reproduce the paper's numbers (GSM8K, MATH-500, HumanEval, MBPP) with the repository's evaluation pipeline:

git clone https://github.com/Introspective-Diffusion/ORL orl && cd orl/evaluation
export ORL_EVAL_WORK=/path/to/eval-work
bash scripts/build_images.sh && bash scripts/prepare.sh
hf download yifanyu/QwenDLM-4B-ORL --local-dir $ORL_EVAL_WORK/hf/QwenDLM-4B-ORL
bash scripts/main_table.sh qwen-dlm-orl $ORL_EVAL_WORK/hf/QwenDLM-4B-ORL

Serve it with the engine settings of the evaluation (paper_decoding.json) and a paper decoder (here GSM8K's):

python3 -m repro.paper.render_config --suite main --model qwen-dlm-orl --benchmark gsm8k --seed 1234 --out decoder.yaml
apptainer exec --nv -B "$PWD,$ORL_EVAL_WORK" $ORL_EVAL_WORK/images/sglang-gen.sif \
  env PYTHONPATH=$ORL_EVAL_WORK/runtime/qwen-dlm/engine/inference/sglang \
      SGLANG_FLASHINFER_PREFILL_BACKEND=fa3 SDAR_FUSED_ROW_STATS=1 SDAR_COMMIT_GRAPH=1 \
  python3 -m sglang.launch_server --model-path $ORL_EVAL_WORK/hf/QwenDLM-4B-ORL --trust-remote-code \
    --tp-size 1 --max-running-requests 1 --context-length 65536 --attention-backend flashinfer \
    --chunked-prefill-size -1 --sdar-rollout-cuda-graph --sdar-rollout-cuda-graph-capture-sizes 1 \
    --dllm-algorithm SDARPrefixKV --dllm-algorithm-config decoder.yaml --port 30000

curl http://localhost:30000/v1/chat/completions -H "Content-Type: application/json" \
  -d '{"model":"default","messages":[{"role":"user","content":"Write a function that reverses a linked list."}],"max_tokens":4096}'

Training

ORL trains on fresh, verified trajectories of the current model, replays the exact residual inputs each denoising step consumed, and adds adjacent entropy contraction (AEC) so that refinement gets faster, not only more accurate. This checkpoint is step 350 of the paper run: stage 1 (steps 1-325) and stage 2 (steps 326-350) on 819 Python problems, on one node of 8 × H100. training/scripts/reproduce_qwen_dlm.sh in the code repository reproduces it byte for byte.

Checkpoint identity

Paper role qwen-dlm-orl, training step 350. checkpoint_manifest.json records the hashes of the weights, tokenizer, configuration and custom model code, and paper_decoding.json the engine and benchmark-specific decoding settings, which downloading the model does not apply.

Related Models

Citation

@misc{yu2026orl,
  title         = {On-Policy Residual Learning for Diffusion Language Models},
  author        = {Yu, Yifan and Jian, Yuqing and Li, Jisen and Hu, Yuezhou and Zhang, Tianyi and Wu, Xiaoxia and
                   Zhu, Rui-Jie and Wang, Junxiong and Zhou, Zhongzhu and Lai, Fan and Zou, James and Xu, Chenfeng},
  year          = {2026},
  eprint        = {2610.20687},
  archivePrefix = {arXiv},
  primaryClass  = {cs.LG},
  url           = {https://arxiv.org/abs/2610.20687}
}