discrete-diffusion-papers

Find :small_blue_diamond: Gemini Diffusion amazing and interested in discrete diffusion? Here is a collection of papers on discrete diffusion models, which will be updated (likely) weekly or biweekly! Feel free to submit a PR or raise an issue if you would like to add relevant papers!

Proprietary Discrete Diffusion Models - Mercury, Gemini Diffusion

Other References - Another earlier great collection: awesome-discrete-diffusion-models

Table of Contents

• 📑 Research Papers
  • Masked Diffusion Models
  • Other Formulations
  • Unified Perspectives
  • Other Topics
  • Post Training

---

📑 Research Papers

Masked Diffusion Models

• 2021.07 Structured Denoising Diffusion Models in Discrete State-Spaces (D3PM, Masked Diffusion Models), NeurIPS 2021 [Paper]
• 2023.10 Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution (SEDD-Absorb), ICML 2024 [Paper]
• 2024.06 Simplified and Generalized Masked Diffusion for Discrete Data, NeurIPS 2024 [Paper]
• 2024.06 Simple and Effective Masked Diffusion Language Models, NeurIPS 2024 [Paper]
• 2024.06 Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data, ICLR 2025 [Paper]
• 2024.10 Energy-Based Diffusion Language Models for Text Generation, ICLR 2025 [Paper]
• 2024.10 Scaling up Masked Diffusion Models on Text, ICLR 2025 [Paper]
• 2025.02 Large language diffusion model (LLaDA), NeurIPS 2025 [Paper]
• 2025.03 Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models, ICLR 2025 [Paper]
• 2025.03 Dream 7B (HKU NLP Group) [Blog | Code]
• 2025.05 LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Models, [Paper]
• 2025.09 Planner Aware Path Learning in Diffusion Language Models Training (PAPL), ICLR 2026 [Paper]

Uniform Diffusion / Discrete Flow Matching

• 2021.02 Argmax Flows and Multinomial Diffusion: Learning Categorical Distributions, NeurIPS 2021 [Paper]
• 2022.05 A Continuous Time Framework for Discrete Denoising Models, NeurIPS 2022 [Paper]
• 2023.10 Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution (SEDD-Uniform), ICML 2024 [Paper]
• 2024.07 Discrete Flow Matching, NeurIPS 2024 [Paper]
• 2024.12 Flow Matching with General Discrete Paths: A Kinetic-Optimal Perspective, ICLR 2025 [Paper]
• 2025.01 Generalized Interpolating Discrete Diffusion (GIDD), ICML 2025 [Paper|Code]

Unified Perspectives

• 2024.10 How Discrete and Continuous Diffusion Meet: Comprehensive Analysis of Discrete Diffusion Models via a Stochastic Integral Framework, ICLR 2025 [Paper]
• 2024.10 Generator Matching: Generative Modeling with Arbitrary Markov Processes, ICLR 2025 [Paper]

Unified Models

• 2025.05 MMaDA: Multimodal Large Diffusion Language Models, [Paper]
• 2025.05 FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocities, [Paper]

Guidance

• 2024.10 Unlocking Guidance for Discrete State-Space Diffusion and Flow Models, ICLR 2025 [Paper]

Post Training

• 2024.06 Diffusion of Thoughts: Chain-of-Thought Reasoning in Diffusion Language Models, [Paper]
• 2025.02 Fine-Tuning Discrete Diffusion Models with Policy Gradient Methods, [Paper]
• 2025.04 d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learning, [Paper]

Application

• 2023.12 GraphGPT: Generative Pre-trained Graph Eulerian Transformer, [Paper]
• 2024.10 Speculative Diffusion Decoding: Accelerating Language Generation through Diffusion, [Paper]