PaperlayerView exact revision on arXiv

Source-linked AI summary

AudioRouter: Data Efficient Audio Understanding via RL based Dual Reasoning

Liyang Chen, Hongkai Chen, Yujun Cai, Sifan Li, Qingwen Ye, Yiwei Wang

arXiv:2602.10439v1cs.SDcs.AIeess.AS

TL;DR

Fine-grained auditory perception remains unreliable in LALMs, while end-to-end training to internalize perceptual abilities requires substantial data. AudioRouter learns when and how to use external audio tools by optimizing a routing policy while keeping the reasoning model frozen. It improves performance on multiple standard audio reasoning benchmarks while requiring up to 600× less training data for tool use than mainstream end-to-end approaches.

  • Problem

    LALMs remain unreliable on fine-grained auditory perception, while end-to-end approaches to internalizing these abilities require costly annotated audio-text data.

    Source: §1 · p. 1

  • Method

    AudioRouter formulates direct and tool-augmented reasoning as alternative actions and optimizes a dedicated routing policy while keeping the audio reasoning model frozen.

    Source: §1 · p. 1, §3.1 · p. 3

  • Results

    Up to 600× less training data is required for learning tool usage, while AudioRouter improves overall performance on multiple standard audio reasoning benchmarks.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Learning to use external tools offers a practical and scalable alternative to end-to-end perceptual learning for audio language models.

    Source: §5 · p. 8

    Experiments focus on short-form, closed-set audio reasoning with a limited set of tools; long-form reasoning, open-ended questions, and more diverse tool capabilities remain future directions.

    Source: §6 · p. 9

Abstract

from arXiv · show

Large Audio Language Models (LALMs) have demonstrated strong capabilities in audio understanding and reasoning. However, their performance on fine grained auditory perception remains unreliable, and existing approaches largely rely on data intensive training to internalize perceptual abilities. We propose AudioRouter, a reinforcement learning framework that enables LALMs to improve audio understanding by learning when and how to use external audio tools. Rather than tightly coupling tool usage with audio reasoning, AudioRouter formulates tool use as an explicit decision making problem and optimizes a lightweight routing policy while keeping the underlying reasoning model frozen. Experimental results show that AudioRouter achieves substantial improvements on standard audio understanding benchmarks while requiring up to 600x less training data to learn tool usage compared with conventional training paradigms. These findings suggest that learning effective tool usage offers a data efficient and scalable alternative to internalizing perceptual abilities in LALMs.

1 Introduction

Fine-grained auditory perception remains unreliable in LALMs, while end-to-end training to internalize these skills is data-intensive. AudioRouter instead learns tool-use decisions with a separate routing policy, improving benchmark performance with substantially less training data.

1 Introduction

  • LALMs remain unreliable on low-level tasks such as pitch estimation, temporal structure analysis, and event counting, despite strong semantic and reasoning capabilities.End-to-end training relies on costly annotated audio-text pairs, and perceptual ability does not always scale proportionally with data volume.Source: §1 · p. 1
  • AudioRouter treats direct reasoning and tool-augmented reasoning as alternative actions, optimizing a dedicated Router while keeping the audio reasoning model frozen.This decouples tool-use decisions from audio reasoning and avoids modifying the model’s existing reasoning and perceptual representations.Source: §1 · p. 1
  • Tool-use systems can select tools through surface keyword overlap despite functional mismatch, or infer outputs beyond a tool’s capability boundaries.AudioRouter frames routing around functional validity and restricts inference to tool capability boundaries via relative reward.Source: §1 · p. 1, Figure 1 · p. 2
  • The Router learns from relative task outcomes with and without tools, avoiding manually annotated tool-use labels.It compares final task performance to determine whether tool invocation provides a tangible benefit.Source: §1 · pp. 1–2
  • Up to 600× less training data is required to learn tool usage, while overall performance improves on multiple standard audio reasoning benchmarks.In several settings, AudioRouter achieves state-of-the-art results.Source: §1 · p. 2

2 Related Work

Related work describes progress in LALMs and tool-augmented systems, alongside persistent limits in fine-grained acoustic reasoning. Reinforcement learning has supported reasoning and action selection, but learning-based management of audio tools remains underexplored.

2 Related Work

  • Recent LALMs integrate large audio encoders with instruction-following LLMs, yet continue to struggle with fine-grained acoustic tasks and rely heavily on curated datasets.They perform strongly on semantic or high-level tasks, while multi-task benchmarks report limits in fine-grained acoustic reasoning.Source: §2 · p. 2

Reinforcement Learning for LLM Reasoning

  • RL methods support language-model reasoning, planning, and action selection, but most reasoning frameworks focus on textual or visual domains.Relatively few address auditory tasks or managing perceptual tools, where action validity, tool selection, and trajectory consistency matter.Source: Reinforcement Learning for LLM Reasoning · pp. 2–3
  • Tool-augmented LLMs use external computation, knowledge, and perception resources, with multimodal examples including systems that orchestrate visual perception modules.Representative systems include Toolformer, ReAct, HuggingGPT, Gorilla, MM-ReAct, and ViperGPT.Source: Reinforcement Learning for LLM Reasoning · p. 3
  • AudioToolAgent and Audio-Maestro introduce tool-augmented LALMs, but rely largely on heuristic or scripted coordination.Robust learning-based frameworks, particularly those driven by reinforcement learning, remain underexplored for audio tools.Source: Reinforcement Learning for LLM Reasoning · p. 3

3 Methodology

AudioRouter formulates audio question answering as a choice between direct reasoning and tool-augmented reasoning, with a separate Router selecting the path while a frozen model produces answers. Its routing policy learns from relative correctness outcomes to use tools when they help and avoid redundant or harmful calls.

3.1 Problem Formulation

  • Audio question answering is framed as choosing the correct candidate through either direct reasoning or a specialized audio tool.The action space includes Direct and tool actions; a selected tool extracts structured perceptual evidence that the Reasoner can use.Source: §3.1 · p. 3

3.2 AudioRouter System

  • A lightweight routing policy dispatches among discrete actions while external audio tools and a frozen audio reasoner remain separate components.Restricting learning to the dispatch policy reduces its complexity and sample requirements relative to end-to-end optimization.Source: §3.2 · p. 3

3.2.1 Tool Router Model

  • The Router selects whether to reason directly or invoke a tool, with capability-aware routing intended to avoid unnecessary or harmful calls.Tool actions are constrained to predefined executable tools.Source: Figure 2 · p. 4, §3.2.1 · p. 4

3.2.2 Audio Reasoner Model

  • The audio reasoning model stays fixed, so differences between direct and tool-assisted predictions can be attributed to tool usage under the same reasoning capacity.The model predicts from the audio, question, candidate options, and optional tool output.Source: §3.2.2 · p. 4

3.2.3 Relative Outcome Reward

  • The Router compares direct and tool-augmented answer correctness, learning to use tools for positive gains while suppressing redundant or harmful calls.Tool actions receive +5.0 when they correct a direct error, −5.0 when they turn a correct direct answer wrong, and −0.1 when correctness matches; direct actions are scored on direct correctness.Source: §3.2.3 · p. 5

4 Experiment

AudioRouter trains a lightweight router while keeping the audio reasoner frozen, and improves results across MMAU-mini and MMAR with substantially less training data. The experiments compare backbone settings, routing strategies, and a targeted tool-use case.

4.1 Experimental Setup

  • The setup compares direct-reasoning baselines with AudioRouter using a frozen reasoner and a separately trained router, evaluated on MMAU-mini and MMAR.The router uses 1.5k filtered multiple-choice audio QA samples; the benchmarks span low-level perception and high-level multimodal reasoning.Source: §4.1 · pp. 5–6

4.2 Main Results

  • AudioRouter improves performance across both benchmarks and backbone settings, using 1.5k training samples versus 38.0k–971.3k for competing methods—a 25×–647× data-efficiency advantage.Figure 4 presents the data-scale and performance comparisons.Source: Figure 4 · p. 6, §4.2 · p. 6
  • On MMAU-mini, average accuracy rises from 66.4% to 67.4% with Qwen2-Audio and from 73.9% to 77.8% with Qwen2.5-Omni.The Qwen2-Audio gains are driven mainly by Sound and Music, with a marginal Speech decline; Qwen2.5-Omni improves across all three categories.Source: §4.2 · p. 6
  • On MMAR, average accuracy rises from 51.3% to 51.8% with Qwen2-Audio and from 63.0% to 64.4% with Qwen2.5-Omni.The Qwen2.5-Omni gains come largely from Sound and mixed-modality settings, despite a slight dip in So–Mu–Sp.Source: §4.2 · pp. 6–7

4.3 Effectiveness of AudioRouter

  • AudioRouter outperforms Omni-CLST by 3.9 points on MMAU-mini and 1.4 points on MMAR, and surpasses end-to-end Tool-use RL on both benchmarks.Unlike end-to-end Tool-use RL, AudioRouter trains only the routing policy while the audio reasoning backbone remains frozen.Source: §4.3 · p. 7

4.4 Router Evaluation

  • AudioRouter achieves the highest accuracy among the evaluated routing strategies on both MMAU-mini and MMAR.Random routing performs worst, while model-based routers perform better than random routing.Source: Table 5 · p. 8, §4.4 · p. 8

4.5 Case Study

  • In a music-genre case, genre_analysis returns tempo and pitch cues that help the reasoner reject Bossa Nova and identify a Nocturnal scene electronic style.The tool reports 129.2 BPM and 1520.58 Hz; the passage contrasts the tempo with a typical Bossa Nova range of 60–90 BPM.Source: §4.5 · p. 8

5 Conclusion

AudioRouter separates tool-use decisions from audio reasoning, allowing tool usage to be learned without modifying the underlying reasoning model. The paper reports improved audio reasoning with less training data and illustrates how tool measurements can inform an answer.

  • In the music example, tool analysis uses 129.20 BPM tempo, 1520.58 Hz pitch, and timbre features to support an electronic or club-oriented interpretation over Bossa Nova.The direct answer instead associates the relaxed accompaniment with Bossa Nova.Source: §5 · p. 8
  • Tool-use decisions are decoupled from audio reasoning, enabling effective, data-efficient learning without modifying the underlying reasoning model.Source: §5 · p. 8
  • AudioRouter improves overall audio reasoning while significantly reducing the training data needed to learn tool usage, offering a practical and scalable alternative to end-to-end perceptual learning.Source: §5 · p. 8

6 Limitations

The paper identifies limits in both the reward baseline and the scope of its experiments.

  • The relative outcome reward uses a fixed reasoning model as its comparison baseline.Source: §6 · p. 8
  • The Router’s learning signal is bounded by the capability of the reasoning model used as the baseline.Source: §6 · p. 8
  • Experiments cover short-form, closed-set audio reasoning with a limited set of tools; longform reasoning, open-ended questions, and more diverse tools remain future directions.Source: §6 · p. 9
Loading 2602.10439v1…