QwenDLM-4B

QwenDLM (4B): Qwen3-4B-Thinking-2507 converted into a block diffusion language model (SDAR-style, 256-token blocks) with a self-conditioning residual adapter, the start model of QwenDLM-4B-ORL

Results

Accuracy (%) / tokens per forward (TPF); each model uses its selected decoder.

Speed

Tokens per second for a single request on one H100 (BF16), with the paper's prompts and decoders. dlmrt decodes QwenDLM-4B about 1.5× faster than SGLang, and ORL raises it to 1,298 tokens/s:

Usage

Plain transformers generation does not run the paper's decoding. Two engines do: dlmrt, our inference engine, and the ORL repository's SGLang engine and decoder configuration, which the paper's evaluation uses.

dlmrt

dlmrt, our inference engine, decodes one request at a time with the paper's decoder (role="base") and measures the paper's throughput; see its README for installation, tensor parallelism and the commands of every paper cell.

from dlmrt import LLM

llm = LLM("yifanyu/QwenDLM-4B", role="base")
out = llm.generate("Write a Python function that reverses a linked list.", max_new_tokens=4096)
print(out.text)
print(f"{out.request_tps:.0f} tokens/s")

python -m dlmrt serve --model yifanyu/QwenDLM-4B --role base --port 18080
curl -N localhost:18080/generate -d '{"messages": [{"role": "user", "content": "Hi!"}], "max_new_tokens": 512}'

The paper's evaluation pipeline

Reproduce the paper's numbers (GSM8K, MATH-500, HumanEval, MBPP) with the repository's evaluation pipeline:

git clone https://github.com/Introspective-Diffusion/ORL orl && cd orl/evaluation
export ORL_EVAL_WORK=/path/to/eval-work
bash scripts/build_images.sh && bash scripts/prepare.sh
hf download yifanyu/QwenDLM-4B --local-dir $ORL_EVAL_WORK/hf/QwenDLM-4B
bash scripts/main_table.sh qwen-dlm-base $ORL_EVAL_WORK/hf/QwenDLM-4B

Serve it with the engine settings of the evaluation (paper_decoding.json) and a paper decoder (here GSM8K's):

python3 -m repro.paper.render_config --suite main --model qwen-dlm-base --benchmark gsm8k --seed 1234 --out decoder.yaml
apptainer exec --nv -B "$PWD,$ORL_EVAL_WORK" $ORL_EVAL_WORK/images/sglang-gen.sif \
  env PYTHONPATH=$ORL_EVAL_WORK/runtime/qwen-dlm/engine/inference/sglang \
      SGLANG_FLASHINFER_PREFILL_BACKEND=fa3 SDAR_FUSED_ROW_STATS=1 SDAR_COMMIT_GRAPH=1 \
  python3 -m sglang.launch_server --model-path $ORL_EVAL_WORK/hf/QwenDLM-4B --trust-remote-code \
    --tp-size 1 --max-running-requests 1 --context-length 65536 --attention-backend flashinfer \
    --chunked-prefill-size -1 --sdar-rollout-cuda-graph --sdar-rollout-cuda-graph-capture-sizes 1 \
    --dllm-algorithm SDARPrefixKV --dllm-algorithm-config decoder.yaml --port 30000

Training ORL from this model

This is checkpoint 38750 of the QwenDLM conversion run. The ORL runs start from it after rescaling the residual adapter (the result's weights and configuration are byte-identical to the paper's start model); the training data is the dataset yifanyu/ORL-training-data:

hf download yifanyu/ORL-training-data --repo-type dataset --local-dir $DATA
apptainer exec -B $PWD $SIF $PY training/tools/make_scrp_base.py QwenDLM-4B QwenDLM-4B-scrp8 --gamma 8
bash training/scripts/reproduce_qwen_dlm.sh $RUNS/qwen_dlm QwenDLM-4B-scrp8 $DATA --sif $SIF --python $PY

Checkpoint identity

Paper role qwen-dlm-base. checkpoint_manifest.json records the hashes of the weights, tokenizer, configuration and custom model code, and paper_decoding.json the engine and benchmark-specific decoding settings, which downloading the model does not apply.

Related Models

Citation

@misc{yu2026orl,
  title         = {On-Policy Residual Learning for Diffusion Language Models},
  author        = {Yu, Yifan and Jian, Yuqing and Li, Jisen and Hu, Yuezhou and Zhang, Tianyi and Wu, Xiaoxia and
                   Zhu, Rui-Jie and Wang, Junxiong and Zhou, Zhongzhu and Lai, Fan and Zou, James and Xu, Chenfeng},
  year          = {2026},
  eprint        = {2610.20687},
  archivePrefix = {arXiv},
  primaryClass  = {cs.LG},
  url           = {https://arxiv.org/abs/2610.20687}
}