Research dLLM

Codebase for Diffusion Language Models Research

---

Research dLLM is a research-focused library for Diffusion Language Models (dLLMs), providing a comprehensive collection of baseline methods (primarily KV caching and decoding strategies) for reproducible experiments.

Why Use This Codebase?

• Unified Evaluation Framework: Research dLLM provides a standardized testing environment that allows users to seamlessly switch between different baseline methods.
• Clean and Well-Documented Code: Research dLLM is written with a strong emphasis on clarity and readability.
• Active and Ongoing Maintenance: Research dLLM is actively maintained and continuously updated. More out-of-the-box baselines will be included in the future.

News

[25/9/30] We released code reading guides, hoping this can help you to grasp our work :)

[25/10/13] Now, batch inference is supported!

[25/12/6] 🚀 We are looking for help to reproduce our experimental results using A100 GPUs. Please contact us if you can assist with the benchmarking.

[26/5/2] We now support SDAR series models, more block diffusion dLLMs will be added in the future!

Supported Models & Methods

Models

The following models are supported out-of-the-box:

KV Caching

The corresponding usages can be found here.

Decoding Strategies

The corresponding usages can be found here.

Setup

# Create and activate the environment
conda create -n d2cache python=3.11 -y
conda activate d2cache

# Install dependencies
pip install -r requirements/common.txt

# Prepare dotenv file, and set model path manually 
cp .env.example .env

Evaluation

Please check docs/ for detailed instructions on how to run evaluations with different methods. A quick example is shown in srcipts/run_eval.sh.

Available models:

• llada-base: GSAI-ML/LLaDA-8B-Base
• llada-inst: GSAI-ML/LLaDA-8B-Instruct
• llada-1.5: GSAI-ML/LLaDA-1.5
• dream-base: Dream-org/Dream-v0-Base-7B
• dream-inst: Dream-org/Dream-v0-Instruct-7B
• sdar-8b-chat: JetAstra/SDAR-8B-Chat

Available datasets:

• gsm8k
• humaneval / humaneval_instruct
• math-500
• mbpp / mbpp_instruct
• ... (all tasks specified in lm-eval are available)

[!IMPORTANT] To evaluate humaneval dataset on Dream-v0-Instruct-7B, please use its corresponding instruct variants, i.e., humaneval_instruct.

Additional general arguments can be specified in configs/generation/*.yaml. If gen_args_script is provided, dynamic defaults will be loaded from that script.

Starchart

Citation

If you find d²Cache or this repository useful for your research and applications, please cite using this BibTeX:

@article{jiang2025d2cache,
  title={d $\^{} 2$ Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching},
  author={Jiang, Yuchu and Cai, Yue and Luo, Xiangzhong and Fu, Jiale and Wang, Jiarui and Liu, Chonghan and Yang, Xu},
  journal={arXiv preprint arXiv:2509.23094},
  year={2025}
}

Acknowledgment

We would like to thank the authors of all models and baseline methods for their excellent work and open-source contributions.

License

This project is licensed under the Apache 2.0 License. See the LICENSE file for details.