Source-linked AI summary
Geometry-Conditioned Target Speaker Extraction based on spatial features incorporating acoustic propagation principles
Jiatong Li, Simon Doclo
TL;DR
GCSSF addresses the fixed-array-geometry dependency of standard SSF, whose geometry-conditioned variant uses abstract positional embeddings. This paper conditions GCSSF with steering-vector and GCC-PHAT spatial features instead, and reports better extraction performance across circular and random arrays, with an advantage for DOA errors up to 20°.
Problem
Method
The paper conditions GCSSF with a target steering vector for inter-microphone phase differences and a GCC-PHAT spatial mask for target presence per time-frequency bin.
Source: §3 · p. 2, §3.1 · p. 2, §3.2 · p. 2Results
The proposed systems consistently outperform SSF and GCSSF-MPE on random and circular arrays, with a clear advantage for target DOA errors up to 20°.
Source: §5 · p. 4, §4.3 · p. 4Takeaways & Limitations
Explicit propagation-derived spatial features provide a geometry-conditioning alternative that outperforms abstract positional encodings in the evaluated arrays.
Source: §5 · p. 4For target DOA errors above 20°, the proposed systems’ PESQ scores fall below the baseline systems’ scores.
Source: §4.3 · p. 4
Abstract
from arXiv · showhide
Recently, spatially selective non-linear filter (SSF) was proposed for target speaker extraction, utilizing the direction-of-arrival (DOA) of the target speaker as a spatial cue. To overcome its dependency on a fixed microphone array geometry, geometry-conditioned SSF (GCSSF) was proposed, which maps the target DOA and the microphone positions into abstract high-dimensional embeddings using positional encoding. As an alternative to these high-dimensional embeddings, in this paper we propose utilizing two spatial features incorporating acoustic propagation principles for geometry conditioning: 1) the steering vector of the target speaker, which captures inter-microphone phase differences, and 2) a spatial mask based on GCC-PHAT, which quantifies presence of the target speaker per time-frequency bin. These features are used as input to a geometry-conditioning encoder and integrated via a FiLM layer into the GCSSF system. Experimental results on circular and random arrays demonstrate that conditioning with spatial features improves target extraction performance compared to abstract high-dimensional embeddings, maintaining a clear performance advantage across a wide range of target DOA errors.
1. INTRODUCTION
Target speaker extraction must distinguish a desired voice from interfering speakers and noise. This work replaces abstract geometry embeddings with propagation-based spatial features and reports improved extraction performance.
1. INTRODUCTION
- Target-speaker extraction addresses separating a desired speaker from mixtures containing other speakers and background noise.Proposed cues include enrollment utterances, visual information, and spatial features.Source: §1 · p. 1
- GCSSF-MPE maps target DOA and microphone positions into abstract high-dimensional sinusoidal embeddings, allowing arbitrary DOA-steered extraction beyond fixed-geometry SSF.SSF depends on the fixed microphone array geometry used during training.Source: §1 · p. 1
- The proposed spatial-feature GCSSF systems outperform SSF and GCSSF-MPE across evaluated metrics, with a performance advantage for target DOA errors up to 20°.The target steering vector yields the best performance.Source: §1 · p. 1
- The proposed conditioning features are a target steering vector encoding inter-microphone phase shifts and a GCC-PHAT spatial mask quantifying target presence per time-frequency bin.The features enter a geometry-conditioning encoder and are integrated into GCSSF through a FiLM layer.Source: §1 · p. 1, Figure 1 · p. 1
2. GEOMETRY-CONDITIONED SPATIALLY SELECTIVE NON-LINEAR FILTER
GCSSF estimates a complex mask to recover the reverberant target component from multichannel noisy speech. Its geometry-conditioning branch uses microphone and target-DOA positional encodings, but these do not explicitly encode acoustic propagation features.
2. GEOMETRY-CONDITIONED SPATIALLY SELECTIVE NON-LINEAR FILTER
- The input comprises microphone STFT signals stacked into a multichannel noisy-speech vector, with target speech and interference-plus-noise vectors defined similarly.The scenario uses an arbitrary planar array of omnidirectional microphones in a reverberant room.Source: §2 · pp. 1–2
- GCSSF estimates a complex mask and applies it to the noisy reference-microphone signal to recover the reverberant target speech component.The first microphone is selected as the reference without loss of generality.Source: §2 · p. 2
- The extraction branch processes spatial, spectral, and temporal information with frequency-domain and time-domain LSTMs and a linear layer; target DOA also conditions the frequency-domain LSTM.A FiLM layer modulates intermediate extraction-branch representations using the geometry-conditioning encoder output.Source: §2 · p. 2
- The geometry-conditioning branch encodes microphone coordinates and target DOA using microphone positional encodings concatenated across microphones and the target direction.The feature is reshaped to [B,M + 1,Q] before entering the branch.Source: §2 · p. 2
- Positional encodings provide spatial coordinates but not explicit propagation-derived features, so the network may need to infer relationships among phase differences, target DOA, and array geometry.The passage notes this may potentially limit extraction performance.Source: §2 · p. 2
3. EXPLICIT SPATIAL FEATURES FOR GEOMETRY CONDITIONING
The section proposes two acoustic-propagation-based spatial features for geometry conditioning: a target steering vector and a GCC-PHAT spatial mask. The steering vector encodes target phase differences across microphones, while the mask indicates target presence per time-frequency bin.
3. EXPLICIT SPATIAL FEATURES FOR GEOMETRY CONDITIONING
- The proposed geometry-conditioning features are the target steering vector and a GCC-PHAT spatial mask, replacing abstract positional encodings with cues based on acoustic propagation.Source: §3 · p. 2
3.1. Target Steering Vector
- Under a far-field model, the target steering vector uses array geometry and target DOA to represent frequency-dependent phase differences across microphones.Its conjugate compensates microphone phases to align signals to the target DOA; stacking it across frequency bins forms PTSV, with shape [B,M,K].Source: §3.1 · p. 2
3.2. GCC-PHAT Spatial Mask
- The GCC-PHAT spatial mask averages real pairwise phase correlations after microphone signals are aligned toward the target DOA.When the target dominates a bin, aligned channels are in phase and the mask approaches 1; interference, reverberation, or noise degrades alignment and decreases it.Source: §3.2 · p. 2
- PMask aggregates the GCC-PHAT mask across frequency bins at each time step and processes frames independently for online processing.Its input tensor has shape [BL,1,K].Source: §3.2 · p. 3
4. EXPERIMENTS
The experiments compare explicit spatial-feature GCSSF systems with SSF and GCSSF-MPE across random and circular arrays. The proposed systems perform better overall, with their advantage over the baselines persisting for target DOA errors up to 20°.
4.1. Datasets
- The evaluation uses simulated reverberant mixtures with two to six speakers and 5-microphone circular or random arrays, trained on random arrays and tested on both geometries.The data use 16 kHz WSJ0 speech, with 30 hours for training and 1 hour per geometry for evaluation; added diffuse noise yields a 15 dB signal-to-noise ratio.Source: §4.1 · p. 3
4.2. Network and Training
- All systems use the GCSSF-MPE architecture framework, with encoder output channels adjusted to match the geometry-conditioning feature dimensions and keep GCSSF parameter counts comparable.Training uses a joint time- and frequency-domain loss, random DOA and microphone-position perturbations, and 500 epochs.Source: §4.2 · p. 3
4.3. Experimental Results
- GCSSF-TSV achieves the best overall performance; both proposed systems outperform SSF and GCSSF-MPE across metrics on random and circular arrays.On random arrays, GCSSF-TSV improves over GCSSF-MPE by 1.66 dB SI-SDR, 0.31 PESQ, 0.06 ESTOI, and 0.12 DNSMOS.Source: §4.3 · p. 4, Table 1 · p. 4
- For target DOA errors up to 20°, GCSSF-TSV and GCSSF-Mask maintain a clear performance advantage over the baselines; beyond 20°, their PESQ scores are lower.Large DOA errors introduce phase mismatches and incorrect signal alignments in the proposed features.Source: §4.3 · p. 4, Figure 4 · p. 4
5. CONCLUSIONS
The paper replaces abstract positional encodings in GCSSF with target steering-vector and GCC-PHAT spatial-mask features. These systems outperform SSF and GCSSF-MPE on both array types, and retain an advantage for DOA errors up to 20°.
5. CONCLUSIONS
- The target steering vector and GCC-PHAT spatial mask condition GCSSF using features derived from acoustic propagation principles.They directly connect multichannel phase differences, target DOA, and array geometry.Source: §5 · p. 4
- Both proposed systems consistently outperform SSF and GCSSF-MPE on random and circular arrays.GCSSF-TSV achieves the best overall performance.Source: §5 · p. 4
- The proposed systems maintain better extraction performance for target DOA errors up to 20°.The conclusion identifies large DOA errors as a sensitivity of the explicit spatial features.Source: §5 · p. 4