PaperlayerView exact revision on arXiv

Source-linked AI summary

Omni-R1: Do You Really Need Audio to Fine-Tune Your Audio LLM?

Andrew Rouditchenko, Saurabhchand Bhati, Edson Araujo, Samuel Thomas, Hilde Kuehne, Rogerio Feris, James Glass

arXiv:2505.09439v3eess.AScs.SD

TL;DR

Collecting transcribed or human-annotated audio can be expensive, motivating investigation of alternatives for improving audio question answering. Omni-R1 fine-tunes Qwen2.5-Omni with GRPO and achieves new benchmark highs, while experiments indicate that text-only reasoning improvements account for much of the gains.

  • Problem

    Collecting transcribed or human-annotated audio can be expensive, motivating alternatives for improving Audio LLM capabilities.

    Source: §V · p. 6

  • Method

    Omni-R1 fine-tunes Qwen2.5-Omni with GRPO on audio question-answering data, including automatically generated questions derived from audio captions.

    Source: §I · p. 1, C. Automatic Q/A Dataset Generation… · p. 3

  • Results

    Omni-R1 reports SOTA across MMAU’s sounds, music, speech, and overall categories on both splits, and SOTA among open-source models on MMAR.

    Source: §V · p. 6

  • Takeaways & Limitations

    Text-only data can improve Audio LLM capabilities, and automatically generated questions may reduce reliance on human annotators.

    Source: §V · p. 6

Abstract

from arXiv · show

We propose Omni-R1 which fine-tunes a recent multi-modal LLM, Qwen2.5-Omni, on an audio question answering dataset with the reinforcement learning method GRPO. This leads to new State-of-the-Art performance on the recent MMAU and MMAR benchmarks. Omni-R1 achieves the highest accuracies on the sounds, music, speech, and overall average categories, both on the Test-mini and Test-full splits. To understand the performance improvement, we tested models both with and without audio and found that much of the performance improvement from GRPO could be attributed to better text-based reasoning. We also made a surprising discovery that fine-tuning without audio on a text-only dataset was effective at improving the audio-based performance.

I. INTRODUCTION

Omni-R1 applies GRPO fine-tuning to Qwen2.5-Omni for audio question answering and reports new benchmark highs using automatically generated training questions. The introduction also highlights that text-only fine-tuning can improve audio performance, with much of the gain attributed to better text-based reasoning.

I. INTRODUCTION

  • 71.3% on Test-mini and 71.2% on Test full are reported as new SOTA after scaling training with automatically generated questions.The automatically generated questions cover 40k AVQA audios and 182k VGGSound audios.Source: §I · p. 1
  • The paper proposes automatically generating audio question-answer data and reports that scaling such questions across 182k VGGSound audios further boosts performance.Source: §I · p. 1
  • Text-only science-question fine-tuning raises Qwen2.5-Omni’s MMAU Test-mini average from 65.9% to 68.2%, while audio-withheld evaluation attributes much of the gain to text-based reasoning.Source: §I · p. 1
  • Omni-R1 applies streamlined GRPO fine-tuning to Qwen2.5-Omni and reports new SOTA on MMAU and MMAR without complex prompts or explicit reasoning.Source: §I · p. 1

II. RELATED WORK

The related work situates Omni-R1 among audio question-answering benchmarks and reinforcement-learning approaches, contrasting its simpler pipeline with a concurrent method. It also frames text-only fine-tuning as a way to probe the source of audio-benchmark gains.

II. RELATED WORK

  • MMAU tests sound, speech, and music questions with one correct and three incorrect choices, while MMAR also includes mixtures such as audio containing both sound and speech.Source: §II · p. 1
  • R1-AQA previously fine-tuned Qwen2-Audio with GRPO on AVQA and achieved the prior MMAU SOTA, inspiring Omni-R1’s approach.Source: §II · p. 1
  • Omni-R1 reports new SOTA on MMAU and strong performance on MMAR, including SOTA among open-source models and competitive results against significantly larger closed-source models.Source: §II · p. 2
  • Unlike concurrent SARI, which combines supervised fine-tuning and RL with structured and unstructured reasoning, Omni-R1 uses RL alone without explicit reasoning.Source: §II · p. 2
  • Fine-tuning audio LLMs using text-only inputs can work almost as well as audio-based fine-tuning, and the authors attribute much of GRPO’s improvement to text-only reasoning.Source: §II · p. 2

III. METHOD

Omni-R1 fine-tunes Qwen2.5-Omni with GRPO using a short prompt that asks the model to choose an answer directly. The authors present the prompt’s simplicity as useful for memory-limited training while still supporting benchmark-leading performance.

A. Audio LLM and Prompt

  • The prompt asks the model to choose an answer from the provided options, encouraging direct output of one answer choice.Source: A. Audio LLM and Prompt · p. 2
  • The simple prompt enabled full fine-tuning on 48GB GPUs, compared with 80GB GPUs used in cited recent works.Source: A. Audio LLM and Prompt · p. 2
  • Despite omitting explicit reasoning prompts used by recent work, the simple prompt was sufficient for the reported MMAU and MMAR SOTA results.Source: A. Audio LLM and Prompt · p. 2

B. Reinforcement Learning

GRPO fine-tuning samples multiple outputs for each question, scores them for correctness, and uses group-normalized advantages to update the policy. Its objective maximizes policy performance over sampled questions and outputs.

B. Reinforcement Learning

  • Each response’s group-normalized advantage is applied uniformly to all its tokens.The advantage is derived from the response rewards normalized across the group.Source: B. Reinforcement Learning · p. 2
  • GRPO assigns each sampled response an accuracy reward of 1 when it contains the correct answer and 0 otherwise.The resulting rewards are normalized across the group to define the advantage.Source: B. Reinforcement Learning · p. 2
  • The policy update maximizes the GRPO objective over input questions and outputs sampled from the old policy.The objective uses a probability ratio between the current and sampling policies, with clipping and a reference-policy KL term.Source: B. Reinforcement Learning · pp. 2–3

C. Automatic Q/A Dataset Generation with ChatGPT

The authors generate audio question–answer training data by turning audio captions into multiple-choice questions and answers, then scale this process from AVQA to VGGSound. They note that caption hallucinations can lead to questions about sounds or music absent from the audio.

C. Automatic Q/A Dataset Generation with ChatGPT

  • The pipeline uses Qwen-2 Audio captions describing acoustic scenes, speech, and music to prompt ChatGPT to generate a correct answer and three plausible but incorrect choices.The correct choice is manually randomized among the four options.Source: C. Automatic Q/A Dataset Generation… · p. 3
  • Caption hallucinations, particularly about music, can yield questions about sounds or music not actually present in the audio.The authors reviewed the generated questions and answers and found their quality reasonable, while noting this issue.Source: C. Automatic Q/A Dataset Generation… · p. 3
  • AVQA-GPT uses generated questions for AVQA’s 40k training audios, while VGGS-GPT scales generation to 182k VGGSound audios but uses 54k samples for training due to GPU memory limits.AVQA’s audio is a subset of VGGSound’s.Source: C. Automatic Q/A Dataset Generation… · p. 3

IV. EXPERIMENTS

The experiments report results on MMAU and MMAR, with the MMAU results divided between its original and updated versions. The updated version revises questions and answers and refines some audio files.

IV. EXPERIMENTS

  • MMAU (v05.15.25) revises 25% of questions and answers and refines 5% of audio files.The authors report MMAU (Original) results in Section IV-B, updated-version results in Section IV-C, and MMAR results in Section IV-D.Source: §IV · pp. 3–4
  • The experiments cover both MMAU and MMAR benchmarks.Source: §IV · p. 3
  • Table II presents accuracy percentages on MMAU (v05.15.25), with Qwen2.5 scores marked as sourced from the benchmark homepage.Source: Table II · p. 4

A. Experimental Setup

Omni-R1 fine-tunes Qwen2.5-Omni with GRPO, and scaled automatically generated question–answer data yields the strongest MMAU results described here. The paper also reports the setup used for training.

A. Experimental Setup

  • Training used four A6000 GPUs, an effective batch size of 8, and 1,000 steps for AVQA and AVQA-GPT or 2,000 for VGGS-GPT.The learning rate was 1 × 10^-6, with temperature 1.2, four responses per GRPO step, and KL coefficient β = 0.04.Source: A. Experimental Setup · p. 4

B. Main Results on MMAU (Original)

  • Qwen2.5-Omni reaches a 65.6% average on MMAU Test-mini, or 65.9% in the authors’ reproduction, while SARI reports 67.1% using an LLM judge.On Test-full, Qwen2.5-Omni has the highest sound, speech, and overall average scores, while Audio-Flamingo 2 leads on music; SARI’s scoring method may limit direct comparison.Source: B. Main Results on MMAU (Original) · p. 4
  • 71.3% on Test-mini and 71.2% on Test-full are the best average scores reported for Omni-R1, which also leads on sound, music, and speech in both splits.Scaling AVQA-GPT to VGGSound clips produces VGGS-GPT; the result improves the average by 5.4% absolute on Test-mini and 2.8% absolute on Test-full over the base model.Source: B. Main Results on MMAU (Original) · p. 4

C. Main Results on MMAU (v05.15.25)

On MMAU v05.15.25, Omni-R1 improves over reproduced Qwen2.5-Omni results with GRPO fine-tuning. The strongest stated AVQA-GPT results are 76.8% on Test-mini and 75.3% on Test-full.

C. Main Results on MMAU (v05.15.25)

  • The reproduced Qwen2.5-Omni average is 71.5% on Test-mini and 71.8% on Test-full, versus 73.1% reported on the benchmark website for Test-full.The authors note that Test-full answers are hidden and scores are computed by an online server.Source: C. Main Results on MMAU (v05.15.25) · p. 4
  • 76.8% on Test-mini and 75.3% on Test-full are the stated AVQA-GPT average scores, exceeding reproduced Qwen2.5-Omni averages of 71.5% and 71.8%, respectively.The Test-full gain over the reproduced baseline is reported as 3.5% absolute; the passage also reports a 77.0% Test-mini score for an additional fine-tuning result.Source: C. Main Results on MMAU (v05.15.25) · p. 5

D. Main Results on MMAR

On MMAR, Omni-R1’s best reported model, fine-tuned on VGGS-GPT, achieves a new open-source state of the art. Scores improve across categories except Mix1, which has only 11 test examples.

D. Main Results on MMAR

  • 63.4% average on MMAR is 5.4% absolute above the base model and sets a new state of the art for open-source models.The base model’s reproduced average is 58%; AVQA fine-tuning reaches 61.2%.Source: D. Main Results on MMAR · p. 5
  • Fine-tuned models improve over the base model in every category except Mix1, where correct answers fall from 6 to 5 among 11 test samples.The model’s performance approaches that of Gemini and GPT-4o, and the reported result does not require long text reasoning steps.Source: D. Main Results on MMAR · p. 5

E. Fine-Tuning Audio LLMs without Audio

The authors test whether GRPO’s audio-benchmark gains reflect better text reasoning by evaluating models without audio and fine-tuning with text alone. Text-only fine-tuning can improve audio-input performance, though audio-based training performs better.

E. Fine-Tuning Audio LLMs without Audio

  • On AVQA, Qwen2-Audio’s average rises from 52.7% to 65.6% with GRPO, versus 65.9% to 68.6% for Qwen2.5-Omni.The authors use MMAU (Original) for these experiments and examine both audio-withheld inference and fine-tuning without audio.Source: E. Fine-Tuning Audio LLMs without Audio · p. 5
  • With audio withheld at inference, Qwen2-Audio scores 30.5% and Qwen2.5-Omni scores 49.3% on MMAU Test-mini.The authors note that Qwen2.5-Omni performs well without audio, indicating that many benchmark questions can be answered without the audio input.Source: E. Fine-Tuning Audio LLMs without Audio · p. 5
  • GRPO fine-tuning on AVQA raises audio-withheld scores from 30.5% to 44.6% for Qwen2-Audio and from 49.3% to 51.7% for Qwen2.5-Omni.The authors attribute the larger Qwen2-Audio gain to improved text-based reasoning and note that Qwen2.5-Omni’s baseline text performance is already high.Source: E. Fine-Tuning Audio LLMs without Audio · p. 6
  • Text-only AVQA fine-tuning raises Qwen2-Audio’s audio-input performance from 52.7% to 58.8%, but slightly decreases Qwen2.5-Omni’s performance.For Qwen2-Audio, performance without audio rises from 30.5% to 42.4%.Source: E. Fine-Tuning Audio LLMs without Audio · p. 6
  • Text-only AVQA fine-tuning improves Qwen2-Audio’s audio-withheld score from 30.5% to 42.4%, while its audio-input score rises from 52.7% to 58.8%.The authors expected text-only training might cause catastrophic forgetting of audio handling; they report a slight decrease for Qwen2.5-Omni instead.Source: E. Fine-Tuning Audio LLMs without Audio · p. 6
  • Text-only AVQA fine-tuning raises Qwen2-Audio’s audio-withheld score from 30.5% to 42.4%, but audio-based AVQA training performs better with and without audio.For Qwen2.5-Omni, text-only ARC-Easy reaches 68.2% with audio, compared with 68.6% for AVQA with audio.Source: E. Fine-Tuning Audio LLMs without Audio · p. 6
  • Text-only ARC-Easy fine-tuning yields 68.2% with audio for Qwen2.5-Omni, nearly matching the 68.6% from AVQA fine-tuning with audio.The authors interpret this as further evidence that GRPO fine-tuning mainly improves text-based reasoning, while noting AVQA with audio performs better.Source: E. Fine-Tuning Audio LLMs without Audio · p. 6
  • The authors conclude that Qwen2.5-Omni is the stronger text-based reasoner, that text reasoning explains much of Qwen2-Audio’s GRPO improvement, and that audio-based fine-tuning still works better.They also report that text-only question–answer fine-tuning is surprisingly effective.Source: E. Fine-Tuning Audio LLMs without Audio · p. 6

V. CONCLUSION

Omni-R1 achieves new state-of-the-art results on MMAU and MMAR, while experiments indicate that text-based reasoning and even text-only fine-tuning contribute to audio question answering gains. The authors suggest these approaches could reduce the cost of building audio-based agents.

  • Omni-R1 sets new MMAU SOTA across sounds, music, speech, and overall accuracy on Test-mini and Test-full, and new open-source SOTA on MMAR.Source: §V · p. 6
  • Text-only fine-tuning also improves audio question answering; the authors suggest exploring better text-only datasets and text/audio-text curricula.Source: §V · p. 6
  • Autogenerated audio QA datasets further boost performance, and these results could help reduce audio-agent development costs.The authors report that their best models used questions generated without human annotators.Source: §V · p. 6
Loading 2505.09439v3…