LLaDA2.1-mini-ORL

LLaDA-2.1-mini + ORL: LLaDA-2.1-mini post-trained with On-Policy Residual Learning (ORL), with a residual adapter

Highlights

• Up to 1.5× tokens per forward (TPF) and up to 3× throughput at comparable quality
• 2,124 tokens/s for a single request on HumanEval with dlmrt on one H100; with NVFP4, 5,046 on one B200 and 2,788 on a consumer RTX 5090
• Data-efficient: 4.2M self-generated tokens and 36 H100-hours of post-training, about 10× less additional data than earlier LLaDA parallel-decoding recipes

Results

Accuracy (%) / TPF; each model uses its selected decoder.

Speed

Tokens per second for a single request, measured with dlmrt unless noted, with the paper's prompts and decoders (the paper reports the base model on GSM8K and HumanEval):

Usage

Plain transformers generation does not run the paper's decoding. Two engines do: dlmrt, our inference engine, and the ORL repository's SGLang engine and decoder configuration, which the paper's evaluation uses.

dlmrt

dlmrt, our inference engine, decodes one request at a time with the paper's decoder (role="orl") and measures the paper's throughput; see its README for installation, tensor parallelism and the commands of every paper cell.

from dlmrt import LLM

llm = LLM("yifanyu/LLaDA2.1-mini-ORL", role="orl")
out = llm.generate("Write a Python function that reverses a linked list.", max_new_tokens=4096)
print(out.text)
print(f"{out.request_tps:.0f} tokens/s")

python -m dlmrt serve --model yifanyu/LLaDA2.1-mini-ORL --role orl --port 18080
curl -N localhost:18080/generate -d '{"messages": [{"role": "user", "content": "Hi!"}], "max_new_tokens": 512}'

The paper's evaluation pipeline

Reproduce the paper's numbers (GSM8K, MATH-500, HumanEval, MBPP) with the repository's evaluation pipeline:

git clone https://github.com/Introspective-Diffusion/ORL orl && cd orl/evaluation
export ORL_EVAL_WORK=/path/to/eval-work
bash scripts/build_images.sh && bash scripts/prepare.sh
hf download yifanyu/LLaDA2.1-mini-ORL --local-dir $ORL_EVAL_WORK/hf/LLaDA2.1-mini-ORL
bash scripts/main_table.sh llada-orl $ORL_EVAL_WORK/hf/LLaDA2.1-mini-ORL

The evaluation generates with SGLang's offline engine (here with the GSM8K decoder, rendered by python3 -m repro.paper.render_config --suite main --model llada-orl --benchmark gsm8k --seed 12345 --out decoder.yaml), inside $ORL_EVAL_WORK/images/sglang-gen.sif with PYTHONPATH=$ORL_EVAL_WORK/runtime/llada/engine/inference/sglang:

import sglang as sgl

engine = sgl.Engine(model_path="LLaDA2.1-mini-ORL", trust_remote_code=True, tp_size=1, mem_fraction_static=0.60,
                    disable_cuda_graph=False, sdar_rollout_cuda_graph=True, sdar_rollout_cuda_graph_capture_sizes=[1, 2],
                    attention_backend="flashinfer", disable_radix_cache=True,
                    dllm_algorithm="SDARPrefixKV", dllm_algorithm_config="decoder.yaml")

Training

ORL trains on fresh, verified trajectories of the current model, replays the exact residual inputs each denoising step consumed, and adds adjacent entropy contraction (AEC) so that refinement gets faster, not only more accurate. This checkpoint is step 625 of the paper run, trained on a mixture of 819 GSM8K, 819 MATH and 819 Python problems read in order, on one node of 8 × H100. training/scripts/reproduce_llada2.sh in the code repository reruns the recipe in one launch; MoE kernels accumulate in a nondeterministic order, so reruns differ slightly.

Checkpoint identity

Paper role llada-orl, training step 625. Weights, tokenizer, configuration and custom model code are byte-identical to the inventory used in the paper reproduction; checkpoint_manifest.json records their hashes, and paper_decoding.json the engine and benchmark-specific decoding settings, which downloading the model does not apply.

Related Models

Citation

@misc{yu2026orl,
  title         = {On-Policy Residual Learning for Diffusion Language Models},
  author        = {Yu, Yifan and Jian, Yuqing and Li, Jisen and Hu, Yuezhou and Zhang, Tianyi and Wu, Xiaoxia and
                   Zhu, Rui-Jie and Wang, Junxiong and Zhou, Zhongzhu and Lai, Fan and Zou, James and Xu, Chenfeng},
  year          = {2026},
  eprint        = {2610.20687},
  archivePrefix = {arXiv},
  primaryClass  = {cs.LG},
  url           = {https://arxiv.org/abs/2610.20687}
}