Source-linked AI summary
Direction-preserving Spatial Audio Signal Enhancement in The Spherical Harmonic Domain
Huawei Zhang, Jihui Aimee Zhang, Huiyuan June Sun, Prasanga Samarasinghe
TL;DR
Existing SASE algorithms often inadequately preserve residual-interference spatial cues, which matter for listening experience and speech intelligibility. The paper proposes an SH-domain, multi-output beamforming approach that preserves cues of both desired and residual-interference soundfields. Simulations report comparable estimation and enhancement performance to existing SASE while better preserving residual-interference cues within an ROI.
Problem
Existing SASE algorithms often inadequately preserve residual-interference spatial cues, despite their relevance to listening experience and speech intelligibility.
Source: §1 · p. 1Method
Results
Simulations show performance comparable to existing SASE in estimation error, speech-distortion-ratio, and noise reduction, with better preservation of residual-interference spatial cues.
Source: §5 · p. 8Takeaways & Limitations
The simulations support preserving both desired and residual-interference spatial cues within the ROI while reducing interference and noise.
Source: §1 · p. 2The simulations assume accurate ReHCs for both soundfields and estimate the interference-plus-noise PSD from about 0.5 seconds of recordings with the desired source inactive.
Source: §4.1 · p. 5
Abstract
from arXiv · showhide
Spatial audio signal enhancement (SASE) aims to separate a desired soundfield from interference and noise while preserving its spatial cues. However, existing algorithms pay limited attention to preserving the spatial cues of the residual interference, which reduces their effectiveness in practical applications. In this paper, we propose a direction-preserving SASE algorithm based on multi-output beamforming. The proposed algorithm extracts the desired soundfield while preserving the spatial cues of both the desired soundfield and the residual interference soundfield. The simulation study shows that the proposed algorithm achieves performance comparable to the existing SASE algorithm in terms of estimation error, while it better preserves the spatial cues of the residual interference within a region of interest, in a noisy and reverberant environment.
1. INTRODUCTION
Existing SASE algorithms often distort residual-interference spatial cues, although those cues matter for listening experience and speech intelligibility. The paper proposes an SH-domain, ReHC-based multi-output beamformer to preserve cues from both desired and residual-interference soundfields within a region of interest.
1. INTRODUCTION
- SASE extracts a desired soundfield from mixed microphone recordings while preserving desired spatial cues, supporting applications including soundfield decomposition and reproduction [8].Source: §1 · p. 1
- Residual-interference spatial cues matter for immersive listening and speech intelligibility, but existing SASE algorithms often distort them.Such distortions can degrade spatial perception and the overall listening experience.Source: §1 · p. 1
- Earlier direction-preserving SASE approaches either require prior knowledge of the desired-field PSD matrix or compromise interference reduction.Source: §1 · p. 1
- Direction-preserving binaural beamformers target two points rather than a region of interest, limiting their spatial-audio applicability.Source: §1 · p. 2
- A prior ReHC-based SASE algorithm preserves desired-field cues, but does not adequately preserve residual-interference cues [9].Source: §1 · p. 2
- The proposed SH-domain algorithm uses ReHC-based multi-output beamforming to preserve desired and residual-interference spatial cues within a ROI.It derives additional constraints from interference ReHCs and does not require prior knowledge of the desired soundfield PSD matrix.Source: §1 · p. 2
2. PROBLEM FORMULATION
The formulation represents a mixed soundfield using spherical-harmonic coefficients and separates its desired, interference, and diffuse-noise components. The objective is to estimate the desired coefficients while keeping residual-interference cues similar to those of the original interference field.
2. PROBLEM FORMULATION
- At an arbitrary position, the mixed soundfield is decomposed into SH coefficients using order and mode indices, a radial Bessel function, and an angular SH function.The maximum order depends on wavenumber; the array and ROI share radius R0.Source: §2 · p. 2
- The measured microphone signals are transformed into mixed SH coefficients representing desired, interference, and diffuse-noise fields.Source: §2 · p. 2
- The system setup contains desired and interference sources, diffuse noise, and a spherical microphone array centered on the spherical ROI.Source: §2 · p. 2, Figure 1 · p. 3
- The objective is to estimate desired SH coefficients while ensuring residual-interference cues remain similar to those of the original interference field.Source: §2 · p. 3
3. DIRECTION-PRESERVING SPATIAL AUDIO SIGNAL ENHANCEMENT
The proposed SH-domain SASE method uses multi-output beamforming and ReHC-based spatial constraints to extract the desired soundfield while preserving spatial cues of both it and the residual interference.
- The algorithm is formulated as multi-output MVDR beamforming to extract desired SH coefficients and preserve desired and residual-interference spatial cues.Source: §3 · p. 3
- ReHC-derived constraints for the desired soundfield are intended to extract it with its spatial cues.Source: §3 · p. 3
- Additional constraints derived from interference ReHCs are intended to preserve the residual interference’s spatial cues.Source: §3 · p. 3
- The beamformer cost function supports desired-soundfield extraction, interference and diffuse-noise reduction, and preservation of both soundfields’ spatial cues.The estimated desired SH coefficients are transformed by ISHT to obtain pressures over the ROI.Source: §3 · pp. 4–5
4. SIMULATION AND RESULTS
The simulation compares the proposed algorithm with existing and modified SASE baselines in a noisy, reverberant room. Across the evaluation, it performs comparably to existing SASE while better preserving residual-interference spatial cues under tested conditions.
4. SIMULATION AND RESULTS
- The comparison includes existing SASE and a modified baseline that partially adds original mixed SH coefficients to processed output to preserve residual-interference cues.The modified baseline builds on prior work and SASE [9].Source: §4 · p. 5
4.2. Evaluation
- The proposed algorithm matches existing SASE on Error, SDR, and NR; the modified baseline has the lowest NR and deteriorated Error.Figure 2 reports averages inside the spherical ROI.Source: Figure 2 · p. 6, §4.2 · p. 6
4.1. Setup
- Evaluation averages Error, SDR, and NR across frequencies and 343 uniformly distributed ROI observation points over 200 time frames.The setup uses LibriSpeech speech sources, a 32-microphone spherical array, diffuse and sensor noise, and a reverberant room with T60 = 0.2 s.Source: §4.1 · p. 5, §4.2 · p. 6
4.2. Evaluation
- Across SDNR conditions from 10 to 30 dB, the proposed algorithm has the best average Error and NR, with SDR comparable to both baselines.The modified algorithm has the lowest NR and highest Error despite superior SDR.Source: §4.2 · p. 6, Table 1 · p. 7
- At 10 dB SDNR, proposed SASE achieves 0.85 interference-soundfield similarity, comparable to modified SASE; at 30 dB, proposed similarity decreases to 0.75, below modified yet above existing SASE (about 0.70).All algorithms achieve desired-soundfield similarity of 0.99 across SDNR levels.Source: §4.2 · p. 7
5. CONCLUSION
The paper concludes that its spherical-harmonic SASE algorithm captures the desired soundfield while preserving spatial cues of both desired and residual-interference soundfields. Simulations show comparable standard enhancement performance and better residual-interference cue preservation.
5. CONCLUSION
- The algorithm is designed to capture the desired soundfield and preserve spatial cues of both desired and residual-interference soundfields.Source: §5 · p. 7
- The proposed algorithm achieves performance comparable to existing SASE in Error, speech-distortion-ratio, and noise reduction.Source: §5 · p. 8
- The paper identifies designing a specific metric for evaluating direction-preserving SASE as future work.Source: §5 · p. 8