Omni-R1: Do You Really Need Audio to Fine-Tune Your Audio LLM?

Paper Link

Introduction

We propose Omni-R1 which fine-tunes a recent multi-modal LLM, Qwen2.5-Omni, on an audio question answering dataset with the reinforcement learning method GRPO. This leads to new State-of-the-Art performance on the recent MMAU and MMAR benchmarks. Omni-R1 achieves the highest accuracies on the sounds, music, speech, and overall average categories, both on the Test-mini and Test-full splits. To understand the performance improvement, we tested models both with and without audio and found that much of the performance improvement from GRPO could be attributed to better text-based reasoning. We also made a surprising discovery that fine-tuning without audio on a text-only dataset was effective at improving the audio-based performance.

Our main contributions are:

• Omni-R1, a streamlined GRPO fine-tuning on Qwen2.5-Omni that achieves new SOTA on MMAU and MMAR without any complex prompts or explicit reasoning.
• Automatically generated audio question answering datasets which scale question–answer pairs across 182k VGGSound clips to further boost performance.
• Analysis of text-only fine-tuning, demonstrating that improving an audio LLM’s text reasoning yields larger-than-expected gains on audio benchmarks.

Setup

This repo is directly built on R1-AQA, please visit their repo to follow the setup instructions and also give them a star!

Qwen-Omni-2.5 setup:

• Update transformers to 4.52.0
• Model - deleted the "talker" part since we don't use it from this file miniconda3/envs/phi312/lib/python3.12/site-packages/transformers/models/qwen2_5_omni/modeling_qwen2_5_omni.py

Pre-trained Models

• Omni-R1 (Trained on VGGS-GPT): omni-r1-checkpoint-vggs-gpt.tar.gz
• Omni-R1 (Trained on AVQA-GPT): omni-r1-checkpoint-avqa-gpt.tar.gz
• Omni-R1 (Trained on AVQA): omni-r1-checkpoint-avqa.tar.gz

Download example:

ckpt=omni-r1-checkpoint-vggs-gpt
wget https://data.csail.mit.edu/public-release-sls/omni-r1/${ckpt}.tar.gz
mkdir -p exp/model/${ckpt}
tar --strip-components=8 -xvf ${ckpt}.tar.gz -C exp/model/${ckpt}

Testing

Test script: test_mmau_omni_pretrained.sh runs this python script src/test_omni.py

Note: for testing Qwen-Omni without audio:

• Comment out line 123 in src/test_omni.py and replace with line 124
• Comment out lines 233 - 235 and replace with if special_token == self.image_token: in this file: miniconda3/envs/phi312/lib/python3.12/site-packages/transformers/models/qwen2_5_omni/processing_qwen2_5_omni.py

Training

GRPO: run_grpo.sh. You can use sbatch train_4gpu.sh if you have SLURM. By default, we save a checkpoint every 100 epochs and you can check the performance of all of them with test_mmau_omni.sh. The step of the best checkpoint is random, so we recommend checking all of them and using the best one. Training for 1,500 steps takes around 10 hours on 4 A6000 GPUs with 48 GBs each.

Training Data

• Full VGGS-GPT: data/AVQA/train_qa_chatgpt_VGGSound.data
  • We trained on the split filtered to 30% of the data: data/AVQA/train_qa_chatgpt_VGGSound_filtered_0.3.data
• Full AVQA-GPT: data/AVQA/train_qa_chatgpt_v1_balanced.data
  • We trained on this and got OOM after ~2k steps but it worked better than filtering the data and training entirely on that split.
• Scripts for generating the data: openai/
• Script for filtering data: openai/filter_dataset.py

Output of test_mmau_omni.sh after training (evaluating on MMAU-v05.15.25). Note: the performance is even higher than reported in the paper since we selected the "best" checkpoints using the original MMAU and then evaluated on MMAU-v05.15.25.

iter       sound    music    speech    Total Accuracy
---------  -------  -------  --------  ----------------
test_1200  82.28%   73.35%   77.78%    77.80%
test_1300  82.58%   72.75%   77.48%    77.60%
test_1000  81.98%   73.65%   77.18%    77.60%
test_800   82.28%   72.75%   77.78%    77.60%
test_900   82.28%   73.35%   76.88%    77.50%
test_600   82.88%   72.16%   77.48%    77.50%
test_400   82.88%   72.75%   76.88%    77.50%
test_500   82.58%   72.16%   77.48%    77.40%
test_100   82.88%   71.86%   76.88%    77.20%
test_1500  81.68%   73.05%   76.88%    77.20%
test_1100  81.68%   73.05%   76.58%    77.10%
test_700   82.28%   71.86%   77.18%    77.10%
test_200   82.28%   72.46%   76.28%    77.00%
test_1400  81.38%   72.46%   76.58%    76.80%
test_300   82.58%   70.06%   75.38%    76.00%

Citation

@article{rouditchenko2025omni,
  title={Omni-R1: Do You Really Need Audio to Fine-Tune Your Audio LLM?},
  author={Rouditchenko, Andrew and Bhati, Saurabhchand and Araujo, Edson and Thomas, Samuel and Kuehne, Hilde and Feris, Rogerio and Glass, James},
  journal={arXiv preprint arXiv:2505.09439},
  year={2025}
}