Source-linked AI summary
Broadband DOA estimation using Convolutional neural networks trained with noise signals
Soumitro Chakrabarty, Emanuël. A. P. Habets
TL;DR
Broadband DOA estimation is difficult under noise and reverberation, motivating a CNN that learns directly from microphone-array STFT phase. Trained on synthesized noise, the system generalizes to speech and is evaluated for robustness to noise, microphone-position perturbations, and acoustic mismatch.
Problem
Accurate broadband DOA estimation is challenging in acoustic environments with noise and reverberation.
Method
A CNN directly receives STFT phase components and learns DOA-relevant features during supervised training with synthesized noise signals.
Results
Experiments show that the noise-trained CNN generalizes to speech and localizes sources under noise, microphone-position perturbations, different acoustic conditions, and real environments.
Takeaways & Limitations
Training with synthesized noise simplifies dataset preparation while supporting broadband DOA localization across the evaluated simulated and real acoustic conditions.
Takeaways & Limitations
The robustness explanation relies on learned local phase structures and weight sharing as proposed mechanisms for handling noise and microphone-position perturbations.
Abstract
from arXiv · showhide
A convolution neural network (CNN) based classification method for broadband DOA estimation is proposed, where the phase component of the short-time Fourier transform coefficients of the received microphone signals are directly fed into the CNN and the features required for DOA estimation are learnt during training. Since only the phase component of the input is used, the CNN can be trained with synthesized noise signals, thereby making the preparation of the training data set easier compared to using speech signals. Through experimental evaluation, the ability of the proposed noise trained CNN framework to generalize to speech sources is demonstrated. In addition, the robustness of the system to noise, small perturbations in microphone positions, as well as its ability to adapt to different acoustic conditions is investigated using experiments with simulated and real data.
1. INTRODUCTION
Broadband DOA estimation is needed in many acoustic applications but remains difficult under noise and reverberation. The paper proposes a CNN that learns DOA-relevant features directly from STFT phase and can be trained with synthesized noise.
- Broadband DOA estimation supports hands-free communication, teleconferencing, and distant speech recognition applications.
- Accurate DOA estimation is challenging when acoustic recordings contain noise and reverberation.
- Existing broadband methods include MUSIC, GCC-based TDOA, SRP-PHAT, MCCC, and maximum-likelihood approaches.
- DNN-based DOA methods often require explicit feature extraction, adding computational cost and potentially retaining traditional-method problems.
- The proposed CNN directly uses STFT phase, learns DOA features during training, and can be trained with synthesized noise instead of speech.
2. DOA ESTIMATION AS A CLASSIFICATION PROBLEM
The paper formulates broadband DOA estimation as supervised classification over discrete angular classes. A trained classifier maps fixed-dimensional features to posterior probabilities, and the highest-probability class determines the estimate.
- The CNN framework learns a mapping from observed microphone-array signals to the DOA of the arriving sound wave.
- DOA estimation is performed independently for each time frame of the signals’ STFT representation.
- The task is an I-class classification problem in which each class represents one possible discretized DOA value.
- For a ULA covering 0°–180° at 2° resolution, the classification problem contains I = 91 DOA classes.
- During testing, the classifier produces posterior probabilities for the DOA classes from each input feature vector.
3. CNN BASED DOA ESTIMATION
The proposed system feeds STFT phase maps directly into a CNN that learns local microphone-frequency correlations and classifies each frame’s DOA. Convolution, activation, and fully connected layers produce posterior probabilities without pooling.
- 3.1. Input feature representation: The received signals are transformed into complex STFT coefficients whose magnitude and phase components are identified.
- 3.1. Input feature representation: The system uses STFT phase directly as input so training learns the features relevant to DOA estimation.
- 3.1. Input feature representation: Each frame becomes an M × K phase map, arranging phase values across microphones and frequency bins.For M = 4 microphones and Nf = 256, the phase map is 4 × 129.
- 3.2. Convolutional neural networks: CNN convolution filters process local input regions, generate feature maps, and share weights across positions.
- 3.2. Convolutional neural networks: The network combines convolution, activation, and fully connected layers to classify the complete input, while pooling reduces feature-map resolution.
- 3.3. DOA estimation with CNNs: The proposed CNN applies 2 × 2 filters to neighboring microphones and local frequency regions, then combines learned structures through fully connected layers.
- 3.3. DOA estimation with CNNs: Local filters and weight sharing are proposed to improve robustness to noise, local distortions, and small microphone-position perturbations.
- 3.3. DOA estimation with CNNs: The architecture uses ReLU activations and omits pooling because experiments found that pooling slightly decreased performance.
4. TRAINING WITH NOISE
Because the input excludes the magnitude spectrum, the CNN can be trained on synthesized spectrally white noise rather than speech recordings. This simplifies both data preparation and ground-truth labeling.
- The proposed network is trained with spectrally white noise sources placed at different angles and distances from the microphone array.
- Using only STFT phase makes synthesized training signals possible because the magnitude spectrum is not used.
- Noise-based training removes the need for speech databases and simplifies ground-truth label preparation.
- Unlike speech training, synthesized noise avoids relying on voice activity detection to identify silent frames and prevents label inconsistencies from VAD errors.
5. EXPERIMENTAL RESULTS
Experiments evaluate frame-level DOA accuracy for simulated and real speech under noise, mismatched acoustics, microphone-position perturbations, and varying reverberation and distance. The CNN generalizes from synthesized noise to speech and generally outperforms SRP-PHAT, while performance declines somewhat under unmatched conditions.
- Evaluation setup: Frame-level accuracy measures the proportion of active speech frames whose estimated DOA matches the true DOA.The evaluation uses this metric because the proposed method performs DOA classification for each STFT time frame.
- Evaluation setup: The simulated evaluation uses a four-microphone ULA, 5° DOA resolution, 129 frequency bins, and approximately 5.6 million training frames across 37 classes.Training data combine reverberated spectrally white noise with spatially uncorrelated Gaussian noise at randomly selected levels from 0 to 20 dB.
- Generalization to speech and noise: The noise-trained CNN generalizes to speech in matched acoustic conditions and achieves much higher frame-level accuracy than SRP-PHAT under additive white noise.The matched-condition test uses 500 four-second TIMIT speech samples convolved with a simulated room impulse response.
- Different acoustic conditions: Under unmatched room size, reverberation, and source-array-distance conditions, the CNN accurately localizes the source for most frames, though performance is slightly worse than in matched conditions.It remains considerably better than SRP-PHAT, which fails to provide accurate estimates in the presence of reverberation and noise.
- Microphone-position perturbations: With 5 mm and 3 mm opposite-direction microphone shifts, the CNN is more robust to array perturbations than SRP-PHAT.The paper attributes this to CNN weight sharing, which makes the model robust to local distortions in the input phase map, whereas SRP-PHAT requires exact array geometry.
- Real acoustic conditions: On measured real-room data, the CNN adapts to real acoustic scenarios despite training on simulated data and noise signals, clearly outperforming SRP-PHAT at a 2 m source distance.At closer distances and lower reverberation times, SRP-PHAT performs better, which the paper attributes to the spatial selectivity provided by eight microphones.
6. CONCLUSION
The proposed CNN-based broadband DOA estimator can be trained with noise signals while generalizing to speech sources. Experiments showed robustness to noise, microphone-position perturbations, differing acoustic conditions, and real environments.
- The CNN-based broadband DOA estimator can be trained with noise signals and generalize to speech sources.
- Experiments demonstrated robustness to noise and small perturbations in microphone positions.
- The method localized sources under acoustic conditions different from the training data and in real acoustic environments.