PaperlayerView exact revision on arXiv

Source-linked AI summary

Multichannel Audio Quality Assessment: Extending Pretrained Perceptual Models to Spatial Audio

Gouthaman KV, Shiv Gehlot, Vishnu Raj, Lars Villemoes, Arijit Biswas

arXiv:2609.37116v1cs.SDcs.AI

TL;DR

Subjective audio-quality testing is costly, and perceptual models trained for limited channel configurations do not directly apply to higher-channel spatial audio. The paper compares four ways to integrate multichannel information while reusing pretrained GMLv2, including learned latent aggregation and FGAtt. Across five 5.1-channel test sets, FGAtt achieves the strongest overall performance with a single frozen-backbone inference.

  • Problem

    Subjective quality tests are costly, while perceptual models designed for limited channel configurations do not directly support higher-channel audio.

    Source: §1 · p. 1

  • Method

    The study compares signal-, prediction-, latent-, and feature-level integration for pretrained GMLv2, including learned latent aggregation and FGAtt, which adaptively fuses spatial-group features before perceptual processing.

    Source: §1 · p. 1, §2.1 · p. 2

  • Results

    Across five 5.1-channel test sets, FGAtt achieves the strongest overall performance using a single frozen-backbone inference.

    Source: §4 · p. 4

  • Takeaways & Limitations

    The results show that integration stage matters and that feature-level adaptation supports reuse of pretrained perceptual knowledge for 5.1-channel audio.

    Source: §4 · p. 4

Abstract

from arXiv · show

Accurate perceptual quality assessment is essential for evaluating and optimizing spatial audio, where perceived quality depends on both signal fidelity and inter-channel spatial relationships. However, subjective evaluation is costly, while existing perceptual models are often trained for limited channel configurations and cannot be directly applied to higher-channel-count audio. This raises the question: how can pretrained perceptual knowledge be effectively reused for multichannel spatial audio? Using 5.1-channel audio, we study four levels of multichannel integration: signal, prediction, latent, and feature and propose two learned approaches: latent-level aggregation of spatial-group representations and the feature-level Feature-Band Group Attention (FGAtt), which adaptively fuses spatial groups at the feature level before perceptual processing. Across five 5.1-channel test sets, FGAtt achieves the strongest over- all performance, demonstrating the effectiveness of feature-level adaptation for reusing pretrained perceptual knowledge

1. INTRODUCTION

Objective metrics are needed because subjective audio-quality tests are costly, but perceptual models built for limited channel configurations do not directly support higher-channel spatial audio. The paper compares integration strategies for reusing pretrained perceptual knowledge in 5.1 audio and reports the strongest overall performance from FGAtt.

1. INTRODUCTION

  • Objective perceptual assessment matters for audio coding, enhancement, restoration, and spatial-audio systems, while costly and less scalable subjective tests motivate objective metrics.Learning-based models such as InSE-NET and GML learn representations from subjective data and agree more closely with human judgments than conventional signal-level metrics.Source: §1 · p. 1
  • Higher-channel audio challenges perceptual models designed for fixed channel configurations, including GMLv2, which supports mono, stereo, and binaural audio but not 5.1 or 7.1.4 directly.Downmixing or binauralization can lose channel-specific information and inter-channel relationships; prediction-level aggregation cannot model those interactions and adds per-channel computation.Source: §1 · p. 1
  • The study compares signal-, prediction-, latent-, and feature-level integration while retaining the pretrained GMLv2 backbone, and proposes learned latent aggregation and feature-level FGAtt.FGAtt adaptively fuses multichannel Gammatone features using input-dependent, feature-band-specific spatial-group weights before perceptual processing.Source: §1 · p. 1
  • Across five 5.1-channel test sets, FGAtt provides more effective reuse of pretrained perceptual knowledge than signal-domain reduction, prediction-level aggregation, and latent-level aggregation.The findings emphasize both the integration level and granularity of multichannel information.Source: §1 · p. 1

2. METHODOLOGY

The methodology compares four levels of multichannel integration around a frozen GMLv2 backbone. It introduces learned latent aggregation and feature-level FGAtt, which assigns input- and band-dependent weights to spatial groups before perceptual processing.

2.1. Problem Formulation

  • The framework retains frozen GMLv2 while comparing fixed or learned integration strategies that incorporate multichannel information at different points in processing.The strategies span signal, prediction, latent, and feature levels.Source: §2.1 · pp. 1–2

2.2. Multichannel Adaptation

  • The methods exclude the LFE channel and represent the front pair, surround pair, and center as three spatial groups in GMLv2-compatible four-component inputs.Each paired group uses its two channels, their average, and their difference; the center is duplicated using the backbone’s mono-input procedure.Source: §2.2 · p. 2
  • Signal-level downmixing or binauralization uses fixed transformations, while prediction-level strategies average scores from independent mono or two-channel processing.Fixed signal transformations may discard channel-specific spatial information; independent predictions cannot model inter-channel interactions and require repeated backbone evaluations.Source: §2.2 · p. 2
  • Latent-level aggregation independently processes each spatial group, then learns attention weights to combine group representations before GMLv2’s frozen prediction head.This retains perceptual processing within each group but requires a separate backbone inference per group.Source: §2.2 · p. 2
  • FGAtt learns input-dependent, feature-band-specific weights to fuse spatial-group features before the frozen GMLv2 backbone processes them jointly.A shared mapping scores groups at each band; temporal pooling is used only to estimate attention, while the original temporal features are preserved for fusion.Source: §2.2 · pp. 2–3

3. EXPERIMENTS AND RESULTS

The experiments evaluate multichannel integration strategies on five independent 5.1-channel test sets. FGAtt achieves the strongest overall correlations, while the results also show that performance varies by test set and attention granularity.

3.1. Experimental Setup

  • The five independent evaluation sets contain 428 samples spanning diverse codecs, bitrates, content types, and subjective quality levels.The training corpus contains approximately 26 hours of 5.1-channel audio and 21K samples with subjective scores from 75 MUSHRA tests.Source: §3.1 · p. 3
  • The learned aggregation methods are trained on the 5.1-channel corpus with the GMLv2 backbone frozen and the aggregation modules optimized using AdamW.The learning rate is 0.001; signal-level experiments use ITU downmixing and the Binamix binauralizer.Source: §3.1 · p. 3

3.2. Results and Discussion

  • Table 1 reports Pearson and Spearman correlations across five 5.1-channel test sets, with the LFE channel excluded.The table includes GMLv2-based integration strategies and mono or two-channel perceptual-metric baselines.Source: §3.2 · p. 3, Table 1 · p. 4
  • With ViSQOL, ITU Downmix achieves overall PCC/SCC of 0.8573/0.7455, versus 0.8395/0.8066 for Binauralizer; Binauralizer has higher SCC across all five evaluation sets.For PEAQ, the corresponding overall PCC/SCC scores are 0.4667/0.5944 for ITU Downmix and 0.4051/0.5112 for Binauralizer.Source: §3.2 · p. 3
  • Group prediction-level aggregation improves overall PCC/SCC to 0.8131/0.7731 from 0.7464/0.5249 for mono prediction-level aggregation.The approaches require separate backbone inference for each channel or spatial group; ITU Downmix scores 0.8623/0.7839, compared with 0.8045/0.7564 for Binauralizer.Source: §3.2 · p. 3
  • FGAtt achieves the strongest overall PCC/SCC, 0.9063/0.8303, and outperforms latent aggregation across all five test sets.It fuses spatial-group representations before perceptual processing, allowing a single inference through the frozen GMLv2 backbone.Source: §3.2 · pp. 3–4, Table 1 · p. 4, Figure 1 · p. 4
  • On T1–T3, ITU Downmix performs strongly; on T4–T5, FGAtt has higher correlations and more consistent performance across the five sets.The T4/T5 PCC/SCC values are 0.9142/0.7889 and 0.9415/0.8616 for FGAtt, versus 0.8493/0.7844 and 0.7976/0.6891 for ITU Downmix.Source: §3.2 · p. 4
  • Feature-band-dependent group attention yields 0.9063/0.8303 PCC/SCC, versus 0.8912/0.8128 without band-specific weights and 0.8803/0.8045 with component-level attention.The ablation supports preserving the internal GML-compatible structure while varying spatial-group weights across feature bands.Source: §3.2 · p. 4

4. CONCLUSION

The conclusion compares four integration levels for extending pretrained perceptual assessment to 5.1-channel audio. FGAtt has the strongest overall performance across five test sets using a single frozen-backbone inference.

4. CONCLUSION

  • FGAtt adaptively fuses spatial-group features before perceptual processing and achieves the strongest overall performance across five 5.1-channel test sets using one frozen-backbone inference.The study compares signal-, prediction-, latent-, and feature-level integration, including learned latent aggregation and FGAtt.Source: §4 · p. 4
  • The results show that the integration stage is critical to effective reuse of pretrained perceptual knowledge for 5.1-channel audio.The framework can extend to richer formats through appropriate channel grouping and backbone-compatible representations.Source: §4 · p. 4
Loading 2609.37116v1…