PaperlayerView exact revision on arXiv

Source-linked AI summary

The LOCATA Challenge: Acoustic Source Localization and Tracking

Christine Evers, Heinrich Loellmann, Heinrich Mellmann, Alexander Schmidt, Hendrik Barfuss, Patrick Naylor, Walter Kellermann

arXiv:1909.01008v3eess.AScs.SDeess.SP

TL;DR

Acoustic localization and tracking must operate despite reverberation, noise, inactivity, false estimates, and changing source-sensor geometries. The paper reviews relevant methods and evaluates LOCATA’s open benchmarking framework, finding task- and array-dependent performance patterns and improved dynamic localization when tracking is incorporated.

  • Problem

    Realistic and dynamic acoustic scenes cause localization and tracking errors, missed estimates, and false estimates, while existing open datasets were insufficient for broad objective benchmarking.

  • Method

    The paper reviews localization and tracking algorithms and evaluates challenge submissions using LOCATA’s open corpus and evaluation framework across six static and dynamic tasks.

  • Results

    Performance depended on the array and scenario: static-task azimuth accuracies reached approximately 1.0°, while dynamic-task accuracies reached up to 1.8° with the DICIT array and 3.1° with another evaluated configuration.

  • Takeaways & Limitations

    Tracking filters benefit dynamic localization by exploiting source-dynamics models, while array geometry and microphone count influence which approaches perform best.

  • Takeaways & Limitations

    LOCATA used only one acoustic environment because spatial ground-truth localization constrained the data-challenge setup.

Abstract

from arXiv · show

The ability to localize and track acoustic events is a fundamental prerequisite for equipping machines with the ability to be aware of and engage with humans in their surrounding environment. However, in realistic scenarios, audio signals are adversely affected by reverberation, noise, interference, and periods of speech inactivity. In dynamic scenarios, where the sources and microphone platforms may be moving, the signals are additionally affected by variations in the source-sensor geometries. In practice, approaches to sound source localization and tracking are often impeded by missing estimates of active sources, estimation errors, as well as false estimates. The aim of the LOCAlization and TrAcking (LOCATA) Challenge is an open-access framework for the objective evaluation and benchmarking of broad classes of algorithms for sound source localization and tracking. This article provides a review of relevant localization and tracking algorithms and, within the context of the existing literature, a detailed evaluation and dissemination of the LOCATA submissions. The evaluation highlights achievements in the field, open challenges, and identifies potential future directions.

I. INTRODUCTION

LOCATA addresses the need for reproducible evaluation of acoustic source localization and tracking in realistic, dynamic environments. It combines an open-access corpus, evaluation software, literature context, and benchmark submissions.

  • Motivation: Realistic environments introduce reverberation, noise, interference, inactivity, and changing source-sensor geometries that degrade localization and tracking.These conditions produce decreased accuracy, missed detections, false detections, and a need for fast updates.
  • Challenge framework: LOCATA provides an open-access framework for objective benchmarking across static and dynamic acoustic scenarios.The framework targets reproducible and comparable research using real-world recordings rather than simulated data alone.
  • Challenge framework: The corpus includes four microphone arrays, static and dynamic recordings, ground-truth source and sensor poses, voice-activity labels, and close-talking reference signals.The recordings emphasize human speech in realistic reverberant environments with ambient noise and diverse sensor configurations.
  • Contributions: The paper contributes a literature review and detailed discussion of LOCATA benchmark results, achievements, open challenges, and future directions.The review is framed around the challenge tasks and submitted approaches.
  • Research gap: Existing open-access datasets were limited by static scenarios, restricted source or sensor motion, or audio-visual annotations that did not provide full 3D acoustic ground truth.LOCATA was designed to address these limitations for comparisons across varied scenarios.

IV. LOCATA DATA CORPUS

The LOCATA corpus combines diverse microphone arrays, speech recordings, optical motion capture, and synchronized positional data in a reverberant laboratory environment. It supports localization across static and moving source-array configurations.

  • Recording environment: Recordings were made in a 7.1 × 9.8 × 3 m3 laboratory with a reverberation time of about 0.55 s.A 4 × 6 m2 area within the room was used for recordings.
  • Microphone arrays: The corpus uses a 12-microphone pseudo-spherical robot head, a 32-microphone spherical Eigenmike, a 15-microphone planar DICIT array, and binaural hearing aids.The arrays span spherical, planar, and wearable configurations with different apertures and microphone spacings.
  • Acoustic sources: Static-source tasks use loudspeakers replaying VCTK utterances, while dynamic tasks use five human talkers reading randomly selected VCTK sentences.The dynamic talkers wore close-talking microphones near their mouths.
  • Ground truth: OptiTrack recorded source and array motion using ten synchronized infrared cameras at 120 Hz with approximately 1 mm marker-position accuracy.Markers were attached to each source and microphone array to identify and track their positions.
  • Synchronization and coordinates: Audio and positional streams were recorded in parallel and synchronized afterward using timestamps.The corpus also provides translations and rotations for transforming source coordinates between global and array-local reference frames.

D. Voice Activity Labels

The paper describes voice activity handling and the localization methods used in LOCATA-related submissions, from time-delay estimation to tracking and reverberation-aware approaches. These methods address false, missing, and spurious estimates in dynamic speech recordings.

  • Voice Activity Labels: Voice-activity labels were manually derived from source or close-talking signals and adjusted for propagation and processing delays.Ground-truth labels were available for development data but excluded from evaluation data.
  • Localization and tracking: Localization systems process microphone signals in batches or sliding windows and estimate source positions directly or through spatial cues such as TDoAs.Tracking then uses past observations and source-dynamics models to smooth source trajectories.
  • Time Delay Estimation:: GCC-PHAT estimates time delays by weighting cross-correlation in the DTFT domain to pre-whiten correlated speech signals.This approach helps address spurious cross-correlation peaks caused by speech periodicity.
  • Time Delay Estimation:: Reverberation and simultaneous sources can create spurious or ambiguous GCC peaks because the underlying models assume free-field propagation.The paper discusses explicit reverberant-channel modeling through acoustic impulse-response estimation as an alternative.
  • Localization and tracking: Localization converts TDoA estimates into source directions or positions using lookup tables, least-squares triangulation, or interhyperboloidal spatial intersections.LOCATA submissions combined such localization methods with particle filters, variational EM, or wrapped Kalman filtering for tracking.

2) Binaural Localization:

Binaural localization exploits interaural cues, while submitted LOCATA systems combine these cues with time-delay estimation and tracking filters. However, front-back ambiguity remains a fundamental limitation without additional spatial information or motion.

  • Binaural cues: Binaural localization uses ILDs, IPDs, and ITDs to encode relative source position at the listener’s ears.ITDs are equivalent to TDoAs in this context.
  • Binaural limitations: Sources on the cone of confusion produce binaural cues that cannot distinguish frontal from rear-hemisphere locations.Head or source motion can help resolve this ambiguity, with ITD changes more informative than ILD changes.
  • LOCATA submissions: LOCATA submission ID 8 combines time-delay estimation with IPDs and a wrapped Kalman filter for single-source Tasks 1, 3, and 5.
  • LOCATA submissions: Submission ID 7 combines diagonal-unloading beamforming with a Kalman filter using a seven-microphone DICIT subarray for single-source Tasks 1, 3, and 5.

4) Spherical Microphone Arrays:

Spherical microphone arrays represent three-dimensional soundfields through spherical harmonics, enabling localization methods such as MUSIC and pseudo-intensity processing. LOCATA submissions combine these representations with tracking and multi-source estimation methods.

  • Array representation: Spherical microphone arrays sample the soundfield in three dimensions using microphones distributed over a typically rigid spherical baffle.Their geometry supports efficient computation through orthonormal wavefield decomposition and spherical harmonics.
  • Array representation: A spherical-array response is represented with spherical harmonics, whose weights depend on the array configuration and whose basis functions are indexed by order and degree.
  • Localization methods: Existing localization methods extend to spherical-harmonic signals, including MVDR near-field localization and pseudo-intensity approaches based on dipole beamforming.
  • LOCATA submissions: LOCATA submissions use Eigenmike spherical-harmonic processing with particle filtering, MUSIC extensions, or subspace pseudo-intensity methods across Tasks 1–4.IDs 10, 12, 15, and 16 cover different task combinations and processing variants.
  • Subspace localization: MUSIC estimates source locations from the signal subspace of a correlation matrix, assuming uncorrelated plane-wave signals and an orthogonal noise subspace.
  • Subspace limitations: Correlated sources and reverberation make MUSIC difficult because correlation-matrix rank deficiency prevents effective separation of signal and noise subspaces.A direct-path dominance test can retain rank-1 time-frequency bins to reduce reverberation effects.
  • LOCATA submissions: LOCATA submissions apply MUSIC, direct-path dominance, GEVD, and subspace pseudo-intensity variants for localization and tracking across static and dynamic tasks.

2) Supervised Learning and Neural Networks:

Data-driven localization learns mappings from acoustic cues to source positions, while tracking methods recursively combine localization observations with dynamical models. LOCATA submissions instantiate these approaches with classifiers, neural regression, particle filters, Kalman filters, and related methods.

  • Learning-based localization: Supervised and semi-supervised approaches learn source locations from annotated binaural cues or RTF features using regression and manifold regularization.
  • Learning-based localization: Deep localization methods use hand-crafted cues such as GCC, spatial-coherence eigenvectors, ILDs, cross-correlation functions, or TDoAs as inputs.TDoAs are used in some approaches to reduce reverberation effects.
  • LOCATA submissions: LOCATA submissions include an amplitude-modulation-spectrum classifier and a DNN regression model for single-source localization.ID 1 targets Task 1 with hearing-aid signals, while ID 9 estimates source DoA from four DICIT microphones.
  • Single-source tracking: Bayesian tracking estimates the current source-state posterior from observations such as positions, TDoAs, or DoAs, with state variables potentially including velocity and acceleration.
  • Single-source tracking: For linear Gaussian state spaces, sequential Bayesian filtering reduces to a Kalman filter, whereas nonlinear or non-Gaussian acoustic models motivate particle filtering.
  • Single-source tracking: Particle filters approximate intractable posteriors by importance sampling particles and weighting them using the observation likelihood and state-transition model.
  • Tracking limitations: Tracking accuracy depends strongly on localization quality, while TDoA-based tracking additionally depends on accurate calibration and synchronization.DoA observations can relax those dependencies, though the resulting state-space model may be non-Gaussian.
  • LOCATA submissions: LOCATA single-source submissions combine TDE, beamforming, IPDs, ambisonics, or MUSIC localization with particle, Kalman, or wrapped Kalman tracking.

2) Multi-Source Tracking:

Multi-source tracking must estimate both source states and the changing number of sources while handling uncertain observation-to-track associations. LOCATA submissions address this with PHD filtering and variational EM approaches.

  • Problem formulation: Unlike classical Bayesian tracking, multi-source tracking must represent uncertainty in both source positions and the number of active sources.
  • Data association: Heuristic data association uses prediction-based gates or collars to discard unlikely observation-to-track pairings before assigning observations.
  • Data association: Variational EM avoids explicit data association by modeling observation-to-track assignments as discrete latent variables.A von Mises extension supports robust estimation of DoA trajectories.
  • Random finite sets: Random Finite Sets model time-varying source states and unknown source counts, while the PHD filter approximates their otherwise intractable realization density.This formulation also supports track initiation and termination with false or missing observations.
  • Tracking ambiguities: Fig. 3 depicts tracking ambiguities by assigning colors to unique track IDs.
  • PHD tracking: PHD tracking has been combined with particle filtering for nonlinear TDoA-based position tracking and with von Mises mixtures for bearing-only tracking.
  • LOCATA submissions: LOCATA submissions use MUSIC DoAs with a PHD filter and direct-path RTF localization with variational EM for multi-source tracking.

VI. EVALUATION MEASURES

The evaluation framework addresses localization and tracking errors caused by missed, false, fragmented, and swapped estimates in realistic acoustic scenarios. It defines measures around source activity periods, recordings, and time-stamped source estimates.

  • Challenges: Realistic reverberation, noise, inactivity, and dynamic source-sensor geometries produce estimation errors, missed detections, false detections, track divergence, and track swaps.Tracking additionally requires handling initialization, termination, and short-term missing estimates.
  • Evaluation dimensions: The measures cover estimation accuracy, ambiguity, track completeness, continuity, and timeliness.These dimensions distinguish positional error, false estimates, missing detections, fragmentation or swaps, and response delay.
  • Evaluation setup: Each recording is modeled over duration Trec with up to Nmax sources, whose activity is divided into annotated periods with start and end times.The nomenclature supports per-recording evaluation of source activity and estimates.
  • Evaluation setup: Participants submit azimuth estimates at prescribed timestamps, and each estimate carries an integer ID assigning it to an active source.The timestamps correspond to the optical tracking rate used during recording.

B. Individual Evaluation Measures

Individual measures are built from gating and source-to-estimate association, which separate valid, missing, and false estimates before quantifying angular and tracking behavior. The framework reports complementary measures for accuracy, ambiguity, completeness, continuity, and timeliness.

  • Assignment cases: For a single source, the evaluation distinguishes inactivity, valid estimates, missing estimates, and false estimates caused by reflections or other acoustic artifacts.This motivates measuring missing and false estimates separately from directional accuracy.
  • Gating and association: Gating removes source-estimate assignments whose errors exceed a threshold; removed estimates count as false, while sources without gated detections count as missed.For LOCATA, a 30° azimuth threshold was selected to identify systematic false estimates while limiting effects on most single-source submissions.
  • Gating and association: In multisource cases, the Munkres algorithm assigns sources and estimates by minimizing total angular error, with each source and track used at most once.Unassigned estimates are false, and unassigned sources are missing.
  • Directional accuracy: Angular errors are computed separately for azimuth and elevation for each assigned source-track pair at every voice-activity timestamp.Ground-truth and estimated azimuth and elevation define the corresponding directional errors.
  • Performance measures: The framework aggregates valid, false, missing, broken, and swapped tracks at each time step and summarizes measures across recordings using means and standard deviations.These counts support the reported completeness, continuity, ambiguity, and timeliness measures.
  • Performance measures: Probability of detection measures valid-track time, FAR measures false estimates per second, TL measures onset-to-detection delay, and TFR measures fragmentations per second.Track fragmentation combines broken tracks and track swaps.

C. Combined Evaluation Measure

OSPA combines source-count errors and angular estimation errors into a single time-stamped distance, complementing the individual evaluation measures. LOCATA summarizes OSPA predominantly through means and variances across timestamps and recordings because exhaustive analysis is impractical.

  • OSPA metric: OSPA combines estimated-source cardinality error with angular accuracy across active sources using an optimal subpattern assignment.It therefore jointly reflects estimation accuracy, track completeness, and timeliness.
  • OSPA metric: The metric compares estimated and ground-truth source sets, using order p, cutoff c, angular distance, and permutations to determine the assignment cost.The angular distance is capped by c, and LOCATA sets c = 30°.
  • OSPA interpretation: OSPA is agnostic to estimate-to-source association because it searches assignments, while its cardinality term penalizes differences between estimated and true source counts.The order parameter p controls the relative weighting of angular and cardinality errors.
  • Analysis scope: Because the LOCATA corpus is limited in size, the results analysis uses mean and variance of OSPA across timestamps and recordings rather than exhaustive per-case analysis.This scope boundary limits the reported combined-measure analysis to aggregate summaries.

VII. EVALUATION RESULTS

LOCATA evaluates submissions across single-source tasks using average azimuth error during voice activity, comparing localization-only and localization-plus-tracking systems across arrays. Results show strong static-task accuracy, degradation for moving sources, and substantial dependence on array and algorithm configuration.

  • Task 1 results: Task 1 achieves average azimuth accuracies around 1° for the static single-source setting, while hearing-aid arrays produce errors of 8.5° and 8.7°.The hearing-aid results are not directly comparable because those recordings were collected separately.
  • Algorithm and array comparisons: On the Eigenmike, MUSIC Submission 12 achieves 1.1° error, outperforming SRP-based Submissions 6 and 7 at 6.4° and 7.0°.The paper relates this difference to the finer spatial-spectrum resolution of MUSIC compared with pseudo-intensity approaches.
  • Algorithm and array comparisons: Pseudo-intensity Submissions 10 and 15 achieve 8.9° and 8.1° on the Eigenmike, approximately 7° worse than MUSIC Submission 12.The paper attributes the possible degradation to coarse spatial-spectrum approximations, while noting implementation uncertainty.
  • Algorithm and array comparisons: SRP-based Submissions 6 and 7 obtain comparable accuracy without and with tracking, respectively, across robot-head and Eigenmike arrays.Submission 6 reports 1.5° and 6.4°, while Submission 7 reports 1.8° and 7.0° for the two arrays.
  • Task 1 array comparison: For DICIT in Task 1, Submission 3 achieves 1.0° average azimuth error, compared with 2.2° for Submission 7 and 9.1° for Submission 9.The paper connects these differences to the microphone-pair spacing and subarray configurations used by the submissions.
  • Task 1 results: Static Task 1 shows comparable performance between localization-only submissions and systems combining localization with tracking.Static sources permit long data blocks and temporal averaging without requiring a dynamical model.
  • Task 3 results: Average azimuth error increases from 4.3° in Task 1 to 5.5° in moving-source Task 3 across submissions evaluated in both tasks.Submission 7 reaches 3.1° on Task 3 with the robot head, a 1.3° degradation from Task 1.

2) Task 3 - Azimuth Accuracy:

Task 3 results show reduced azimuth accuracy in moving-source scenarios, with errors linked to changing source-sensor geometry, source orientation, voice inactivity, and increasing distance. Gating improves reported accuracy by excluding outliers, but complementary measures remain necessary.

  • Task 3 performance: Moving-source Task 3 reduces azimuth accuracy similarly across submissions compared with static single-source Task 1.The degradation is associated with source-orientation and source-sensor-distance variations.
  • Track behavior: Submission 7 produces accurate, smooth azimuth tracks during voice-active periods, whereas Submission 3 experiences temporary divergence during voice inactivity without VAD.Submission 3 diverges around [3.9,4.4] s and [8.5,9.2] s.
  • Track behavior: When the talker moves away beyond 2 m, decreasing direct-path contribution produces missing, false, or increasingly inaccurate DoA estimates.Submission 7 shows missing detections, while Submissions 3 and 6 show divergence or false estimates from early reflections.
  • Completeness and ambiguity: Probability of detection remains approximately constant from static to dynamic single-source tasks, with a small reduction trend.Gating removes outliers from azimuth accuracy, while FAR accounts for estimates during voice inactivity.

6) Timeliness:

Timeliness results characterize how VAD, tracking, and initialization affect response to speech onsets and inactive periods. No-VAD approaches can achieve very low latency, while track initialization can introduce delays for newly active sources.

  • Timeliness measures: Track latency measures sensitivity to speech onsets and robustness to temporal smearing at speech endpoints.The evaluation averages latency across recordings for Tasks 1, 3, and 5.
  • Latency results: Submissions 1, 3, 11, and 12 achieve 0 s track latency throughout Task 1, while Submissions 3 and 8 remain below 0.2 s in Tasks 3 and 5.These approaches do not incorporate VAD and extrapolate or provide estimates through voice-inactive periods.
  • Latency results: Submission 4 achieves negligible latency across Tasks 1, 3, and 5, whereas Submission 10 reaches 0.35 s in moving-source Task 3.Submission 4 estimates voice activity as a side-product of variational EM tracking; Submission 10 uses a noise-PSD VAD.
  • Multi-source effects: For multi-source Tasks 2, 4, and 6, overall performance decreases relative to single-source Tasks 1, 3, and 5 because simultaneous sources create interference and ambiguous DoA estimates.Overlapping speech energy can produce missing and false estimates affecting completeness, continuity, and ambiguity.
  • Track continuity: Track fragmentation increases from 0.53 fragmentations per second in Task 2 to 0.64 in Task 4 and 0.71 in Task 6 for Submission 4.The tasks progress from static loudspeakers to moving talkers and then moving talkers with moving arrays.

3) OSPA - Accuracy vs. Ambiguity, Completeness and Continuity:

LOCATA evaluates localization and tracking across static and dynamic scenarios using accuracy, cardinality, continuity, and latency-related measures. The results show that dynamic motion, speech inactivity, and array design create distinct trade-offs and leave substantial challenges unresolved.

  • OSPA interpretation: OSPA combines angular accuracy with cardinality errors, making false and missing track estimates visible alongside localization performance.For Task 2, false tracks during voice inactivity reduced the contrast between submissions that differed substantially in averaged azimuth error.
  • OSPA interpretation: During voice inactivity, detection errors produce cutoff-level OSPA errors, whereas active periods are dominated by angular errors when source and estimate counts match.The OSPA behavior changes sharply between one- and two-source activity periods and between inactive and active speech periods.
  • OSPA interpretation: VAD-based tracking mainly suffers missing detections and broken tracks, while prediction without VAD mainly produces false estimates during voice inactivity.These contrasting error profiles explain why both approaches exhibit cardinality-error jumps despite different tracking designs.
  • Dynamic scenarios: Dynamic source or array motion primarily degrades track continuity, completeness, and timeliness, although tracking improves moving-talker azimuth accuracy across Tasks 3–6.Reported accuracies reached 1.8° with the DICIT array, 3.1° with the robot head, and 7.2° with hearing aids.
  • Array design: Array geometry affects localization: spherical arrays benefit from spherical-harmonic processing, while larger microphone spacings increase spatial aliasing.The 12-microphone robot head consistently outperformed the 32-microphone Eigenmike in shared evaluations, and DICIT submissions used subarrays with spacings no greater than 32 cm.
  • Open challenges: Voice-activity handling and frame duration remain open design trade-offs because VAD can increase multi-source fragmentation, while dynamic scenes require short frames despite speech nonstationarity.The evaluation concludes that robust localization and tracking remains largely unsolved beyond relatively benign single-static-source conditions.
Loading 1909.01008v3…