Task-oriented neural FOA encoding for SELD from irregular microphone arrays

1. INTRODUCTION

• Irregular microphone arrays challenge SELD because existing systems often assume a specific spatial format or well-calibrated geometry.SELD jointly performs sound event detection and direction-of-arrival estimation.Source: §1 · p. 1
• FOA offers a fixed-dimensional spherical-harmonic representation, but sparse or nonspherical arrays can cause aliasing, ill-conditioned encoding, and array-model mismatch.Conventional analytical and least-squares encoders work well for suitably sampled regular arrays.Source: §1 · p. 1
• Neural Ambisonics encoders and residual methods can accommodate array geometry, but reconstruction-focused objectives may not preserve event-discriminative and directional information for multisource SELD.Direct SELD supervision may also be insufficient because estimated and theoretical FOA occupy different representation domains.Source: §1 · p. 1
• The proposed two-stage framework jointly optimizes a residual encoder and SELD network, then distills event and localization knowledge from theoretical FOA using frame-level permutation-invariant training.Experiments on synthetic and real recordings show teacher guidance consistently improves SELD performance over the unguided neural encoder and analytical encoding baselines.Source: §1 · p. 1

2. FOA REPRESENTATION AND CONVENTIONAL ENCODING

• Theoretical FOA follows the ACN/SN3D convention with channels ordered (W, Y, Z, X), and is constructed from source signals and directions of arrival.Because it avoids array-induced encoding errors, it serves as the reference and privileged teacher input.Source: §2 · pp. 1–2
• Conventional FOA encoding applies spherical modal-domain inversion to microphone-array STFT coefficients.For nonspherical arrays, the geometry is approximated by a virtual sphere with mean radius while preserving microphone directions.Source: §2 · p. 2
• The spherical approximation can introduce model mismatch, and its encoder provides the initialization for the proposed residual encoder.The encoding uses the first-order SH sampling matrix and corresponding mode-strength matrix.Source: §2 · p. 2

3.1. Residual FOA-Form Encoder

• The residual encoder adds a signal-dependent correction to conventional FOA encoding to produce a four-channel representation that can compensate for array-response mismatch and unmodeled magnitude and phase distortions.Its output follows FOA’s four-channel (W, Y, Z, X) organization, but is optimized for downstream SELD rather than exact theoretical FOA coefficients.Source: §3.1 · p. 2

3.2. Teacher–Student SELD Learning

• The teacher and student features share a seven-channel SELD representation comprising four channel-wise log-Mel spectrograms and three normalized active-intensity components.The features are extracted from theoretical FOA for the teacher and from the learned FOA-compatible representation for the student.Source: §3.2 · p. 2
• A frozen teacher processes theoretical FOA while the student processes FOA-compatible features derived from noisy microphone signals, transferring event and localization knowledge rather than imposing a signal-level reconstruction target.The teacher and student use the same track-wise architecture with independent parameters, and the student and encoder are jointly optimized.Source: §3.2 · p. 2

3.3. Training Objective

• Ground-truth supervision uses frame-level tPIT with binary cross-entropy for SED and mean squared error for DOA, while distillation applies frame-level tPIT between student and teacher outputs.The DOA distillation loss is confidence-weighted, and the teacher-track permutation minimizing the weighted losses is selected independently for each frame.Source: §3.3 · p. 2

4.1. Datasets

• Evaluation uses simulated scenes with one to three sources on tetrahedral and 12-channel Benchmark arrays, plus stationary-source LOCATA recordings for single- and multi-source scenarios.The simulated sets contain 20,000 training, 2,000 validation, and 2,000 test scenes; real evaluation includes all 16 Task 1 recordings and 11 multi-source Task 2 recordings.Source: §4.1 · p. 3

4.2. Implementation Details

• The teacher is pretrained on theoretical FOA and frozen, while the residual encoder and student SELD network are jointly trained.The implementation uses 128-frame input segments and separate models for the two array configurations.Source: §4.2 · p. 4

4.3. Evaluation on Synthetic Data

• On synthetic arrays, KD further improves Neural FOA on both configurations, with clear gains in event detection and localization accuracy.The distilled model has the best overall SELD performance among practical methods, although it does not consistently achieve the highest localization recall.Source: §4.3 · p. 4
• ASM has the best overall signal-level reconstruction quality, but KD achieves the best practical SELD performance despite higher NMSE and no consistent IV-MSE improvement.Without distillation, the residual encoder improves overall NMSE over Conventional FOA on both arrays; reconstruction fidelity alone does not determine downstream utility.Source: §4.3 · p. 4

• A neural residual encoder refines conventional FOA encoding, and a teacher–student scheme transfers event and directional knowledge from theoretical FOA through frame-level permutation-invariant distillation.Source: §5 · p. 4
• On noisy synthetic scenes, teacher guidance consistently improves SELD and substantially reduces localization error on tetrahedral and Benchmark arrays.Source: §5 · p. 4
• On real LOCATA recordings, the proposed approach improves SELD over analytical encoding baselines in single- and multi-source scenarios.Source: §5 · p. 4
• The lowest FOA reconstruction error does not necessarily yield the best downstream SELD performance, distinguishing coefficient fidelity from task-oriented spatial representation quality.Source: §5 · p. 4