Source-linked AI summary
QK-GCC: Learnable Query-Key Spectral Matching for Robust Time Delay Estimation
Jinkai Zhang, Weiye Chen, Yue Huang, Xiaotong Tu, Xinghao Ding
TL;DR
TDE supports microphone-array localization, but handcrafted GCC spectral matching and frequency weighting are vulnerable to noise and reverberation, while neural GCC methods leave cross-channel matching handcrafted. QK-GCC learns local Query-Key spectral matching between microphone signals, and the paper reports better performance than GCC-PHAT and learning-based GCC variants under noise, reverberation, and unseen source types while remaining lightweight.
Problem
GCC’s predefined frequency weighting and handcrafted cross-channel matching remain vulnerable to noise and reverberation, despite neural improvements elsewhere in the GCC pipeline.
Source: §1 · p. 1Method
QK-GCC replaces handcrafted weighted spectral matching with local Query-Key matching of magnitude-phase frequency tokens from two microphone signals.
Source: §1 · p. 1Results
Takeaways & Limitations
The learned Query-Key inner product provides learnable counterparts to GCC’s frequency weighting and cross-channel matching components.
Source: §1 · p. 1, §2.2 · p. 2Validation on real recordings and extension to multi-microphone arrays are left for future work.
Source: §4 · p. 5
Abstract
from arXiv · showhide
Time delay estimation (TDE) is a fundamental component of microphone-array sound source localization. Generalized cross-correlation (GCC) is widely used because it is efficient and interpretable, but its handcrafted spectral matching and predefined frequency weighting are vulnerable to noise and reverberation. Existing neural GCC variants mainly improve robustness by enhancing input signals or modeling GCC responses, while the cross-channel spectral matching step itself remains handcrafted. We propose QK-GCC, a learnable GCC-like framework that replaces handcrafted weighted spectral matching in GCC with Query-Key matching between two microphone signals. The two microphone signals are encoded as magnitude-phase frequency tokens and mapped to Query and Key representations, respectively, enabling frequency reliability learning and local spectral evidence aggregation for delay estimation. Experiments in simulated reverberant rooms across diverse SNR and reverberation conditions show that QK-GCC improves TDE accuracy over GCC-PHAT and learning-based GCC variants, while remaining lightweight and generalizing to unseen source types. The code is available at https://github.com/zhangjinkai33-ui/QK-GCC.
1. INTRODUCTION
QK-GCC replaces handcrafted cross-channel spectral matching with a learnable local Query-Key operator, targeting frequency-reliability and neighboring-evidence limitations in GCC-style TDE. The paper reports improved delay estimation under noise, reverberation, and unseen source types while remaining lightweight.
1. INTRODUCTION
- Accurate relative-delay estimates provide essential spatial cues for downstream direction-of-arrival estimation and source tracking.TDE is a core component of microphone-array sound source localization.Source: §1 · p. 1
- Handcrafted GCC matching fixes frequency reliability and strict bin-to-bin aggregation, limiting adaptation to corrupted bins and nearby spectral evidence.Prior neural GCC methods mainly process signals before GCC or GCC responses afterward, leaving cross-channel matching handcrafted.Source: §1 · p. 1
- QK-GCC replaces fixed weighted spectral matching with local Query-Key matching between magnitude-phase frequency tokens from two microphone signals.The learned operator retains GCC-style cross-channel spectral matching while aggregating evidence within local frequency neighborhoods.Source: §1 · p. 1
- Its Query-Key similarity separates into learned frequency reliability and angular similarity, providing learnable counterparts to GCC weighting and cross-channel matching.This decomposition is presented as a core contribution of the framework.Source: §1 · p. 1
- QK-GCC improves delay estimation under noise, reverberation, and unseen source types while remaining lightweight.Its learned reliability score also aligns with the average inter-channel coherence profile.Source: §1 · p. 1
2. METHOD
QK-GCC replaces GCC’s fixed frequency weighting and phase-matching rule with learnable Query-Key similarity between microphone spectra. It aggregates local frequency evidence into delay logits and trains delay estimation as classification over a candidate delay grid.
2.1. Revisiting GCC as Weighted Spectral Matching
- GCC estimates delay by searching for the peak of a correlation function built from a weighted cross-spectrum, while QK-GCC replaces its fixed matching rule with learnable Query-Key similarity.GCC’s spectral evidence combines predefined weighting with cross-channel phase matching.Source: §2.1 · p. 2
2.2. QK-GCC: Learnable Spectral Matching
- Each signal’s frequency-bin magnitude and phase features are mapped into Query and Key representations whose inner product provides learnable frequency-wise evidence.The norm product acts like GCC’s weighting factor, while angular similarity plays a role analogous to cross-channel matching.Source: §2.2 · p. 2
2.3. Local Frequency Correlation and Delay Head
- QK-GCC matches neighboring frequency bins within a local bandwidth, then combines center, symmetric, and antisymmetric evidence to produce delay logits.A lightweight delay head uses convolutional blocks, frequency pooling, and an MLP; the default local bandwidth is b = 8.Source: §2.3 · p. 2
2.4. Training Objective
- Frame-level TDE is trained as classification over a candidate delay grid using cross-entropy, with inference selecting the delay of maximum posterior probability.Source: §2.4 · p. 2
3. EXPERIMENTS
In simulated-room experiments, QK-GCC leads the learning-based GCC comparisons and maintains its advantage under harder acoustics and unseen source types. The analyses also examine its learned spectral matching and the contribution of its design choices.
3.1. Experimental Setup
- Two-channel simulated LibriSpeech TDE uses speaker-disjoint splits, varied SNR and T60, and matched baseline retraining without test-room or interference-specific fine-tuning.Testing also replaces speech with MUSAN music and noise for zero-shot evaluation.Source: §3.1 · pp. 2–3
3.2. Evaluation Metrics
- MAE and Acc@10cm are treated as primary localization-quality indicators, while RMSE reflects sensitivity to large errors.Acc@10cm measures the share of predictions whose propagation-distance error is within 10 cm.Source: §3.2 · p. 3
3.3. Main Results
- QK-GCC achieves the lowest errors and highest Acc@10cm among learning-based methods, with the fewest parameters and markedly lower cost than NGCC-PHAT.Table 1 reports overall TDE performance over three seeds as mean (std).Source: Table 1 · p. 3, §3.3 · p. 3
3.4. Robustness under Noise and Reverberation
- Fig. 2 shows QK-GCC’s advantage widening as SNR falls and its performance degrading more slowly as reverberation increases.The panels show Acc@10cm across SNR averaged over T60 and reverberation time averaged over SNR.Source: §3.4 · p. 3, Figure 2 · p. 4
3.5. Zero-shot Generalization to Unseen Source Types
- On unseen MUSAN music and noise, QK-GCC retains its lead without adaptation.The zero-shot evaluation uses the same acoustic simulation with source signals replaced.Source: §3.5 · p. 4, Table 2 · p. 4
3.6. Interpretability Analysis
- The learned norm weight follows average inter-channel coherence (Pearson r = 0.90), while local matching uses neighboring offsets and both symmetric and antisymmetric components contribute.Fig. 3 presents learned norm weighting versus coherence, per-head energy across frequency offsets, and component shares.Source: §3.6 · p. 4, Figure 3 · p. 4
3.7. Ablation Study
- One-seed ablations favor magnitude cues, offset decomposition, and moderate local bandwidth over their tested alternatives.A larger self-attention model reaches only 52.35 Acc@10cm; removing magnitude cues lowers Acc@10cm from 66.90 to 62.48, and removing symmetric and antisymmetric components lowers it to 64.08. Bandwidth b=0 gives 60.99, b=16 gives 66.20 versus b=8 at 66.90, and b=32 gives 63.85.Source: §3.7 · p. 4, Table 3 · p. 4
4. CONCLUSION
The conclusion presents QK-GCC as a learnable replacement for handcrafted GCC spectral matching, reporting better performance under noise, reverberation, and unseen source types while remaining lightweight and interpretable. Real-recording validation and multi-microphone extension remain future work.
4. CONCLUSION
- QK-GCC outperforms GCC-PHAT and learning-based GCC variants under noise, reverberation, and unseen source types while remaining lightweight and interpretable.Validation on real recordings and extension to multi-microphone arrays are left for future work.Source: §4 · pp. 4–5
- QK-GCC replaces handcrafted weighted spectral matching with local Query-Key similarity in a learnable GCC-like operator.Its inner product decomposes into a learned frequency-reliability score and an angular similarity term, corresponding to GCC's weighting and cross-channel matching components.Source: §4 · p. 4