PaperlayerView exact revision on arXiv

Source-linked AI summary

Spatial Audio Coding Through Relative Room Impulse Response Estimation

Nour Bouayed, Adrien Llave, Jérôme Daniel, Pascal Scalart

arXiv:2609.08542v1eess.AS

TL;DR

High-resolution HOA needs compression for bandwidth-limited use, while IVAS performance tends to drop on reverberant audio at higher compression ratios. This paper estimates and codes a ReSRIR relative to a beamformed HOA reference, reporting comparable to slightly better performance at lower bitrates.

  • Problem

    High-resolution HOA requires many channels and compression, while IVAS performance tends to drop on reverberant audio at higher compression ratios.

    Source: §I · p. 1

  • Method

    The method blindly estimates ReSRIR using a beamformed HOA reference and exploits its structure and sparsity for coding.

    Source: §I · p. 2

  • Results

    The method achieves comparable to slightly better performance at lower bitrates; on validation data, its QASTANet score improves by 0.1 over IVAS 32 kbps.

    Source: 1) Comparison to baselines · p. 5, §IV · p. 7

  • Takeaways & Limitations

    The study supports that careful modeling and coding of spatial parameters can improve spatial rendering quality under bitrate constraints.

    Source: §IV · p. 7

    The evaluation is limited to single-source scenes; future work will extend it to more diverse acoustic conditions and content and include comprehensive subjective listening tests.

    Source: §IV · p. 7

Abstract

from arXiv · show

Immersive virtual listening relies on spatial audio technologies such as Higher-Order Ambisonics (HOA), which represent sound scenes as multichannel signals. As the desired spatial resolution increases, so does the number of channels, making efficient compression essential for transmission over bandwidth-limited networks. Moreover, to facilitate deployment by network operators, the target bitrate for immersive audio coding should ideally remain close to the 25 kbps currently allocated to VoLTE audio services. State-of-the-art parametric codecs, such as the recently standardized Immersive Voice and Audio Services (IVAS) codec, achieve compression by transmitting spatial metadata together with a reduced number of transport channels. However, recent studies have shown that IVAS performance degrades on reverberant content, particularly at low bitrates, a limitation that suggests its inability to accurately model room acoustics. In this paper, we propose a novel HOA coding scheme based on the explicit and blind estimation of the Relative Spatial Room Impulse Response (ReSRIR), using a beamformed version of the HOA signal as a reference signal. By exploiting the structure and sparsity of the estimated ReSRIR, we derive an efficient parametric representation for immersive audio coding. Experimental evaluations show that the proposed method achieves higher compression than IVAS in the single-transport-channel regime, while maintaining comparable to slightly better quality.

I. INTRODUCTION

HOA provides flexible 3D sound representation, but high spatial resolution requires many channels and makes compression important for bandwidth-limited networks. The paper proposes estimating and coding a relative room response to address reverberant-audio limits in existing parametric coding.

I. INTRODUCTION

  • At least 16 channels are needed for good spatial localization with 3rd-order Ambisonics, increasing data rates and making compression necessary for bandwidth-constrained networks.HOA is also described as a flexible, output system-agnostic format suited to networked applications.Source: §I · p. 1
  • IVAS performance tends to drop on reverberant audio at higher compression ratios, suggesting limited ability to model the room’s acoustic fingerprint.The room response is represented by Spatial Room Impulse Responses mapping a dry source to the reverberant multichannel scene.Source: §I · p. 1
  • IVAS retains only real predictive coefficients, yielding a symmetric, anti-causal ReSRIR; its decorrelation filters better reproduce late noise-like reverberation than early reflections.A mismatch with the true room response can degrade quality, and early reflections strongly influence perceived quality and spatial impression.Source: §I · p. 2
  • Neural spatial codecs include approaches for microphone arrays and Ambisonics, but the cited methods are not evaluated because open-source code and model weights are unavailable.The cited array method’s beamforming objective does not fully align with preserving perceived sound-scene quality.Source: §I · p. 2
  • The proposed HOA coding scheme explicitly and blindly estimates ReSRIR using a beamformed HOA signal as reference, then exploits its structure and sparsity for compression.The authors target quality comparable to IVAS in reverberant conditions.Source: §I · p. 2

II. METHOD

The signal model represents a single static source in a noisy reverberant environment through multichannel HOA responses. It separates dominant early directional paths from late diffuse reverberation and relates the spatial image to the source through an SRIR.

A. Signal model

  • For a single static source in a noisy reverberant environment, the HOA signal comprises a spatial image and ambient noise.The HOA channels are spherical-harmonic expansion coefficients, with C = (O + 1)^2 in 3D.Source: A. Signal model · p. 2
  • An SRIR stacks the responses of HOA components, while a ReSRIR is defined relative to a pseudo-source because the clean source is unavailable.Each HOA response can be represented as a finite-length FIR filter, and the spatial image is the convolution of the clean source with the SRIR.Source: Terminology (RIR, SRIR, ReSRIR) · p. 2, A. Signal model · p. 2
  • The SRIR separates into an early directional component of direct path and reflections, plus a late diffuse component for dense reverberation.The early component is modeled by a truncated plane-wave expansion; the additive late term captures remaining reverberation.Source: Terminology (RIR, SRIR, ReSRIR) · pp. 2–3
  • When the frame length is sufficiently large relative to RIR length, the MTF approximation converts time-domain convolution into a multiplicative STFT-domain model.The conventional STFT model otherwise entails interframe and inter-band convolution.Source: Terminology (RIR, SRIR, ReSRIR) · p. 3
  • When the clean source is inaccessible, the multichannel HOA signal can be modeled using an RTF applied to a known reference channel, such as a beamformed HOA signal.A reference can also be one HOA component or another signal correlated with the source.Source: Terminology (RIR, SRIR, ReSRIR) · p. 3

B. Overview of the proposed method

The proposed coding pipeline estimates a ReSRIR relative to a beamformed HOA reference, then parameterizes its sparse and residual components. The decoder reconstructs the ReSRIR and convolves it with the independently coded reference signal.

B. Overview of the proposed method

  • The encoder estimates a ReSRIR relative to a beamformed HOA signal, then separates dominant propagation paths from residual diffuse energy.The decomposition is parameterized and quantized for encoding.Source: B. Overview of the proposed method · p. 3
  • At decoding, the ReSRIR is resynthesized from its parameters and convolved with the reference signal, independently coded using a core monochannel audio codec.Source: B. Overview of the proposed method · p. 3

C. ReSRIR estimation using GTVV

The method estimates a frequency-domain RTF using a beamformed reference, then converts it to a time-domain ReSRIR. The resulting ReSRIR is sparse and quasi-causal, and is decomposed into sparse and residual components.

C. ReSRIR estimation using GTVV

  • The Generalized Frequency-domain Velocity Vector replaces the omnidirectional reference with a beamformer output, and its time-domain counterpart is the ReSRIR.The method uses the MTF approximation to estimate the frequency-domain RTF before obtaining its time-domain representation.Source: C. ReSRIR estimation using GTVV · p. 3
  • The self-steering algorithm jointly estimates the ReSRIR and source direction, iteratively refining both using a progressively more selective beamformer.It begins with the omnidirectional HOA component as reference; five iterations were sufficient for convergence in the experiments.Source: C. ReSRIR estimation using GTVV · pp. 3–4
  • The estimated ReSRIR is sparse and quasi-causal, motivating its decomposition into a dominant sparse component and a residual.Figure 3 shows the causal portion split into sparse and residual components; Figure 4’s magnitude-histogram mismatch with a fitted Gaussian also indicates sparsity.Source: C. ReSRIR estimation using GTVV · p. 4, Figure 3 · p. 4, Figure 4 · p. 4

D. ReSRIR decomposition and parametrization

The estimated ReSRIR is represented as dominant peaks plus a residual. The sparse component uses multichannel wave packets at peak delays, while the residual is modeled through time-varying channel variances and synthesized with shaped noise.

D. ReSRIR decomposition and parametrization

  • Reflection peaks share time positions across HOA components and are represented by multichannel wave packets spanning multiple samples.Peak delays are local maxima of the delay-magnitude curve; surface absorption motivates wave packets rather than single-sample impulses.Source: D. ReSRIR decomposition and… · p. 4
  • The sparse component can use either projected-and-expanded wave packets or plane-wave decomposition, which preserves reflection direction but not spectral modifications.The projection/expansion strategy uses sequential linear layers; the plane-wave strategy approximates each packet with a correlated spherical-harmonic encoding vector.Source: D. ReSRIR decomposition and… · p. 4
  • The residual is parameterized with short-time covariances that are nearly diagonal, allowing per-channel delay-dependent variances to represent its envelope.At decoding, filtered noise is shaped over time by the upsampled envelope to reconstruct an approximation of the residual.Source: D. ReSRIR decomposition and… · p. 4

E. Parameters quantization

Learned quantizers compress the ReSRIR parameters, with separate treatments for differential delays, wave-packet amplitudes, and residual-envelope vectors.

E. Parameters quantization

F. Hyperparameters and training

The implementation uses three wave packets per estimated ReSRIR and quantizes projected packet latents, differential delays, and residual envelopes. Experiments use synthesized third-order Ambisonics speech signals derived from measured room responses.

F. Hyperparameters and training

  • The configuration uses 3 wave packets per ReSRIR, 42-sample packet support, a 128-dimensional projected latent, and 7 downsampled envelope indices.The 2048-sample estimation frames are 42 ms; quantizer resolutions are 5, 11, and 6 bits for differential delays, projected packet latents, and residual-envelope vectors, respectively.Source: 1) Architecture · p. 5
  • Spatial-parameter coding uses 85 bits per frame, approximately 2 kbps.Source: 1) Architecture · p. 5
  • Experiments use 3rd-order Ambisonics at 48 kHz, with measured SRIRs truncated to 40 ms and convolved with random 3-second DNS-Challenge speech excerpts.The SRIRs come from the Aalto, IKSLabIR, and MOTUS databases, and scenes are randomly rotated in 3D.Source: 2) Training data · p. 5

2) Training data:

Training separates reconstruction learning from quantizer learning: first optimize the non-quantization modules, then freeze them and train quantization codebooks.

3) Training method:

  • With quantization bypassed, the first stage optimizes the remaining modules using wave-packet projection/expansion and residual spectral reconstruction MSE losses.Source: 3) Training method · p. 5
  • The second stage freezes previously trained modules and trains only quantization layers using MSE between input vectors and their quantized versions.Source: 3) Training method · p. 5
  • Training uses Adam at an initial learning rate of 5 × 10−3, batch size 32, 700 first-stage steps, and 1500 second-stage steps.Source: 3) Training method · p. 5

III. EVALUATION

The evaluation uses held-out speech and room-response data, compares against IVAS under a matched single-transport-channel setting, and primarily assesses spatial quality with QASTANet.

1) Data:

  • Test sets contain 50 speech signals drawn from a different DNS-Challenge portion and SRIRs from RSoANU and 50-Lebedev.Source: 1) Data · p. 5

2) Baselines:

  • IVAS at 32 kbps is the main baseline because it uses one transport channel there; the reference signal uses EVS at 24.4 kbps, and IVAS 64 kbps is an additional reference.Source: 2) Baselines · p. 5

3) Criterion:

  • QASTANet, a deep-learning metric, is used mainly to evaluate overall spatial audio quality.Source: 3) Criterion · p. 5

B. Results

The proposed codec achieves comparable or better quality than IVAS at a lower bitrate, while ablations favor wave-packet modeling of the sparse component and diagonal residual covariance.

1) Comparison to baselines:

2) Ablation study:

  • ReSRIR (wp + σ) is superior and close to non-approximated estimated ReSRIR; full residual covariance does not improve and can degrade performance versus diagonal covariance.Source: Figure 6 · p. 6, 2) Ablation study · p. 6
  • Replacing the sparse wave-packet component with Dirac-like impulses significantly degrades performance and increases spectral distortions because it cannot model direct-path versus early-reflection spectral mismatch.Source: 2) Ablation study · p. 6

IV. CONCLUSION

The paper presents an efficient HOA coding method based on blind estimation and parameterization of the relative room impulse response. It reports comparable to slightly better performance at lower bitrates, while noting single-source scope and further evaluation needs.

IV. CONCLUSION

  • The proposed blind relative-room-response estimation and parameterization approach achieves comparable to slightly better performance at lower bitrates.Source: §IV · pp. 6–7
  • The evaluation is limited to single-source scenes; future work will test more diverse acoustic conditions and content and include comprehensive subjective listening tests.Source: §IV · p. 7
Loading 2609.08542v1…