Awesome Text-to-Audio Generation

This is a repository organizing papers, codes and other resources related to Text-to-Audio Generation (TTA), where audio denotes primarily ambient sound or music.

Welcome to PR if you want to add some resources !

Papers

TTA Foundation Models

• Woosh: Sound Effect Generative Models [arXiv 2026] Paper / Code

• STABLE AUDIO 3 [arXiv 2026] Paper / Code

• Generative Audio Language Modeling with Continuous-valued Tokens and Masked Next-Token Prediction [ICML 2025] Paper

• Fugatto 1: Foundational Generative Audio Transformer Opus 1 [ICLR 2025] Paper

• AudioX: Diffusion Transformer for Anything-to-Audio Generation [arXiv 2025] Paper

• TangoFlux: Super Fast and Faithful Text to Audio Generation with Flow Matching and Clap-Ranked Preference Optimization [arXiv 2025] Paper / Code

• CosyAudio: Improving Audio Generation with Confidence Scores and Synthetic Captions [arXiv 2025] Paper

• ETTA: Elucidating the Design Space of Text-to-Audio Models [arXiv 2024] Paper

• Make Some Noise: Towards LLM audio reasoning and generation using sound tokens [ICASSP 2025] Paper

• MambaFoley: Foley Sound Generation using Selective State-Space Models [ICASSP 2025] Paper

• EzAudio: Enhancing Text-to-Audio Generation with Efficient Diffusion Transformer [arXiv 2024] Paper / Code

• GenAU: Taming Data and Transformers for Audio Generation [arXiv 2024] Paper / Code

• Stable-Audio-Open [arXiv 2024] Paper / Code

• Stable-Audio 2: Long-form music generation with latent diffusion [arXiv 2024] Paper

• Stable Audio 1: Fast Timing-Conditioned Latent Audio Diffusion [ICML 2024] Paper

• Masked Audio Generation using a Single Non-Autoregressive Transformer [ICLR 2024] Paper / Code

• SpecMaskGIT: Masked Generative Modeling of Audio Spectrograms for Efficient Audio Synthesis and Beyond [arXiv 2024] Paper

• FlashAudio: Rectified Flows for Fast and High-Fidelity Text-to-Audio Generation [arXiv 2024] Paper

• AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining [TASLP 2024] Paper / Code

• LAFMA: A Latent Flow Matching Model for Text-to-Audio Generation [Interspeech 2024] Paper / Code

• Auffusion: Leveraging the Power of Diffusion and Large Language Models for Text-to-Audio Generation [arXiv 2024] Paper / Code

• AudioGen: Textually Guided Audio Generation [ICLR 2023] Paper

• Make-An-Audio 2: Temporal-Enhanced Text-to-Audio Generation [arXiv 2023] Paper / Code

• Tango: Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Model [ACM MM 2023] Paper / Code

• Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models [ICML 2023] Paper / Code

• AudioLDM: Text-to-Audio Generation with Latent Diffusion Models [ICML 2023] Paper / Code

• Diffsound: Discrete Diffusion Model for Text-to-sound Generation [TASLP 2022] Paper / Code

TTA with Fine-grained Control

• ControlAudio: Tackling Text-Guided, Timing-Indicated and Intelligible Audio Generation via Progressive Diffusion Modeling [arXiv 2025] Paper

• PicoAudio 2: Temporal Controllable Text-to-Audio Generation with Natural Language Description [arXiv 2025] Paper

• DegDiT: Controllable Audio Generation with Dynamic Event Graph Guided Diffusion Transformer [arXiv 2025] Paper

• FreeAudio: Training-Free Timing Planning for Controllable Long-Form Text-to-Audio Generatio [ACM MM 2025] Paper

• Sketch2Sound: Controllable Audio Generation via Time-Varying Signals and Sonic Imitations [ICASSP 2025] Paper

• AudioComposer: Towards Fine-grained Audio Generation with Natural Language Descriptions [ICASSP 2025] Paper / Code

• PicoAudio: Enabling Precise Timestamp and Frequency Controllability of Audio Events in Text-to-audio Generation [ICASSP 2025] Paper / Code

• SILA: Signal-to-Language Augmentation for Enhanced Control in Text-to-Audio Generation [arXiv 2024] Paper

• RiTTA: Modeling Event Relations in Text-to-Audio Generation [arXiv 2024] Paper

• Controllable Text-to-Audio Generation with Training-Free Temporal Guidance Diffusion [ICME 2024] Paper

• Audio Generation with Multiple Conditional Diffusion Model [AAAI 2024] Paper / Code

• T-FOLEY: A Controllable Waveform-Domain Diffusion Model for Temporal-Event-Guided Foley Sound Synthesis [ICASSP 2024] Paper / Code

• Continuous descriptor-based control for deep audio synthesis [ICASSP 2023] / Code

TTA with Inference Acceleration

• MeanAudio: Fast and Faithful Text-to-Audio Generation with Mean Flows [arXiv 2025] Paper

• AudioTurbo: Fast Text-to-Audio Generation with Rectified Diffusion [Interspeech 2025] Paper

• Stable-Audio-Small: Fast Text-to-Audio Generation with Adversarial Post-Training [arXiv 2025] Paper / Code

• SoundCTM: Uniting Score-based and Consistency Models for Text-to-Sound Generation [ICLR 2025] Paper

• FlashAudio: Rectified Flows for Fast and High-Fidelity Text-to-Audio Generation [arXiv 2024] Paper

• ConsistencyTTA: Accelerating Diffusion-Based Text-to-Audio Generation with Consistency Distillation [Interspeech 2024] Paper / Code

• AudioLCM: Text-to-Audio Generation with Latent Consistency Models [ACM MM 2024] Paper / Code

TTA with Reinforcement Learning

• Improving Text-to-Audio Instruction Following via Fine-Grained Feedback from Audio-Aware Large Language Models [Interspeech 2026] Paper

• Resonate: Reinforcing Text-to-Audio Generation via Online Feedback from Large Audio Language Models [Interspeech 2026] Paper

• T2A-Feedback: Improving Basic Capabilities of Text-to-Audio Generation via Fine-grained AI Feedback [ACL 2025] Paper

• TangoFlux: Super Fast and Faithful Text to Audio Generation with Flow Matching and Clap-Ranked Preference Optimization [arXiv 2025] Paper / Code

• Tango 2: Aligning Diffusion-based Text-to-Audio Generations through Direct Preference Optimization [ACM MM 2024] Paper / Code

• BATON: Aligning Text-to-Audio Model with Human Preference Feedback [IJCAI 2024] Paper / Code

TTA with Retrieval-Augmented Generation

• SonicRAG : High Fidelity Sound Effects Synthesis Based on Retrival Augmented Generation [arXiv 2025] Paper

• Audiobox TTA-RAG: Improving Zero-Shot and Few-Shot Text-To-Audio with Retrieval-Augmented Generation [arXiv 2024] Paper

• Retrieval-Augmented Text-to-Audio Generation [ICASSP 2024] Paper

Spatial Audio Generation

• OmniAudio: Generating Spatial Audio from 360-Degree Video [ICML 2025] Paper / Code

• ImmerseDiffusion: A Generative Spatial Audio Latent Diffusion Model [ICASSP 2025] Paper

• Both Ears Wide Open: Towards Language-Driven Spatial Audio Generation [ICLR 2025] Paper / Code

• AudioSpa: Spatializing Sound Events with Text [arXiv 2025] Paper

TTA Benchmarks

• AQAScore: Evaluating Semantic Alignment in Text-to-Audio Generation via Audio Question Answering [arXiv 2026] Paper

• TTA-Bench: A Comprehensive Benchmark for Evaluating Text-to-Audio Models [AAAI 2026] Paper Code

• AUDIOEVAL: AUTOMATIC DUAL-PERSPECTIVE AND MULTI-DIMENSIONAL EVALUATION OF TEXT-TO-AUDIO-GENERATION [arXiv 2025] Paper

• RELATE: Subjective evaluation dataset for automatic evaluation of relevance between text and audio [Interspeech 2025] Paper

Downstream Applications of TTA Models

• Synthio: Augmenting Small-Scale Audio Classification Datasets with Synthetic Data [ICLR 2025] Paper

• Mind the Prompt: Prompting Strategies in Audio Generations for Improving Sound Classification [ICASSP 2025 Workshop] Paper

• Can Synthetic Audio From Generative Foundation Models Assist Audio Recognition and Speech Modeling? [Interspeech 2024] Paper

Text-Guided Audio Editing:

• MMEDIT: A Unified Framework for Multi-Type Audio Editing via Audio Language Model [arXiv 2026] Paper

• RFM-Editing: Rectified flow matching for text-guided audio editing [arXiv 2025] Paper

• SAO-Instruct: Free-form Audio Editing using Natural Language Instructions [NeurIPS 2025] Paper / Code

• AudioEditor: A Training-Free Diffusion-Based Audio Editing Framework [ICASSP 2025] Paper

• AUDIT: Audio Editing by Following Instructions with Latent Diffusion Models [NeurIPS 2023] Paper

• Audio Editing with Non-Rigid Text Prompts [Interspeech 2024] Paper

Datasets

• AudioTime: A Temporally-aligned Audio-text Benchmark Dataset [ICASSP 2025] Paper / Code

• Sound-VECaps: Improving Audio Generation with Visual Enhanced Captions [ICASSP 2025] Paper / Code

• Improving Text-To-Audio Models with Synthetic Captions [Interspeech 2024] Paper

Metrics

TODO

Toolbox

TODO