PaperlayerView exact revision on arXiv

Source-linked AI summary

BCNav: Bearing-Conditioned Depth Policies for Sound Source Navigation

Yaozhong Kang, Jiang Wang, Takeshi Ashizawa, Benjamin Yen, Kazuhiro Nakadai

arXiv:2609.37084v1cs.RO

TL;DR

Sound-source navigation faces acoustic and kinematic transfer gaps when joint audio-visual policies depend on simulated audio and discrete actions. BCNav uses a DOA bearing with depth as input to an imitation-trained continuous-control policy, and demonstrates navigation in simulation and on a physical robot without prior mapping or audio collection.

  • Problem

    Joint audio-visual navigation depends on acoustic simulators and discrete action spaces, while reliable sound-source navigation in unknown environments requires inputs and commands that transfer to physical robots.

    Source: §I · p. 1

  • Method

    BCNav separates DOA estimation from the learned policy, then trains the depth-and-bearing policy by imitation learning on shortest-path demonstrations with calibrated bearing noise to output continuous velocities.

    Source: §I · p. 2, C. Expert Data and Training · p. 3

  • Results

    The sim-trained policy navigates to stationary and moving sound sources on a physical robot without fine-tuning, producing smooth commands at 0.3 m/s linear and 1 rad/s angular velocity in semi-anechoic trials.

    Source: B. Navigation Results · p. 5

  • Takeaways & Limitations

    The bearing-noise tolerance curve predicts navigation success from a DOA method’s accuracy without retraining, and experiments confirm zero-shot transfer to physical hardware.

    Source: §VI · p. 6

    DOA degrades under strong non-line-of-sight conditions; real-world validation covers two environments, and the policy assumes a single active source.

    Source: §VI · p. 6

Abstract

from arXiv · show

The ability to navigate toward sound sources extends a robot's reach beyond its visual field, enabling response to auditory events in unknown environments. To equip robots with this capability, existing methods couple acoustic and visual information through joint audio-visual learning in acoustic simulators. However, acoustic simulation is both low-fidelity and expensive, producing a domain gap that prevents reliable real-world deployment, while the discrete action spaces inherited from grid-based simulators introduce an additional kinematic gap on physical robots. To alleviate these issues, we propose BCNav, a decoupled framework that separates the acoustic module from the learned navigation policy using direction-of-arrival (DOA) estimation: an estimator provides a scalar bearing to the sound source, so the navigation policy only processes depth images and a bearing angle, two inputs whose domain gaps are well characterized. We collect shortest-path demonstrations with calibrated bearing noise injection and train the policy via imitation learning to output continuous velocity commands directly executable on ground robots. We demonstrate the method in simulation and on a physical robot, navigating unknown environments without any acoustic fine-tuning, prior mapping, or real-world audio data collection. Code is available at https://github.com/york1to/bcnav.

I. INTRODUCTION

BCNav addresses the acoustic and kinematic transfer gaps in sound-source navigation by separating DOA estimation from a depth-based policy. It trains the policy from simulated demonstrations to issue continuous commands and validates navigation in simulation and on a physical robot.

I. INTRODUCTION

  • Joint audio-visual policies rely on acoustic simulation that transfers unreliably, while discrete actions also mismatch the continuous commands required by real robots.SLAM-based alternatives still require real-world audio recordings, a pre-built map, and discrete actions in unknown environments.Source: §I · p. 1
  • BCNav places the learned/classical boundary at DOA estimation, giving the policy a scalar bearing and depth rather than raw audio.This sidesteps the acoustic sim-to-real gap without retraining on real-world recordings.Source: §I · pp. 1–2
  • Shortest-path demonstrations with calibrated bearing-noise injection train the policy via imitation learning to map depth and bearing directly to continuous velocity commands.The approach avoids lengthy online reinforcement learning and targets robustness to imprecise real-world DOA estimates.Source: §I · p. 2
  • BCNav is validated in simulation and on a physical robot navigating unknown environments without prior mapping or real-world audio data collection.The system also characterizes how DOA accuracy relates to navigation success.Source: §I · p. 2

II. RELATED WORK

Prior audio-visual navigation work relies on simulated audio and discrete actions, while prior real-robot deployment retains mapping and audio-data requirements. BCNav instead uses a scalar DOA bearing as the interface to navigation.

A. Audio-Visual Navigation

  • AudioGoal and its extensions use a shared simulator and discrete actions; the prior real-robot system still requires target-environment audio recordings and a pre-built map.BCNav removes the learned acoustic component and uses a classical DOA estimate to navigate unknown environments without those requirements.Source: A. Audio-Visual Navigation · p. 2

B. Sound Source Localization

  • SRP-PHAT estimates DOA from microphone-pair time delays and supports real-time deployment without training data, but degrades under heavy reverberation and non-line-of-sight conditions.BCNav uses DOA directly as the module interface and treats the estimator as a replaceable black box.Source: B. Sound Source Localization · p. 2

C. Continuous and Reactive Navigation

BCNav adapts continuous reactive navigation to sound-source seeking by replacing visual or coordinate goals with an acoustic bearing. Its policy conditions on depth and bearing history and produces a short velocity trajectory for receding-horizon execution.

C. Continuous and Reactive Navigation

  • Continuous reactive navigation policies map range observations and a goal cue to velocities, but prior methods use visual or point goals rather than audio guidance.BCNav replaces that goal cue with an acoustic bearing and operates in unknown environments.Source: C. Continuous and Reactive Navigation · p. 2

A. Problem Formulation

  • BCNav separates a classical DOA estimator from a learned policy that maps depth images and bearing to continuous velocity commands, without processing raw audio.The policy uses four observations—three past frames and the current frame at 10 Hz—and predicts eight velocity pairs, executing the first command before replanning.Source: §III · p. 2, A. Problem Formulation · pp. 2–3

B. Policy Architecture

The policy fuses depth and bearing tokens, aggregates four frames, and regresses a velocity trajectory. A separate collision reflex suppresses forward motion near obstacles.

B. Policy Architecture

  • A pretrained DeFM ResNet-18 converts each log-compressed depth frame into 16 spatial tokens for the policy.The backbone is fine-tuned during training and was pretrained by self-distillation on 60M depth images.Source: B. Policy Architecture · p. 3
  • Sinusoidal bearing encoding feeds both a token path and a current-bearing shortcut, preserving the bearing signal alongside the temporal context.The token joins the 16 depth tokens, while the shortcut is concatenated directly before the action head.Source: B. Policy Architecture · p. 3
  • Two-layer bidirectional cross-attention fuses depth and bearing tokens, with each modality querying the other for directional or geometric context.The fused tokens are mean-pooled into a per-timestep feature.Source: B. Policy Architecture · p. 3
  • A two-layer temporal transformer aggregates four per-timestep features, then a three-layer MLP predicts the velocity trajectory.The temporal context is combined with the bearing shortcut before the regression head.Source: B. Policy Architecture · p. 3
  • When minimum depth in the lower field of view falls below 0.2 m, the collision reflex suppresses forward velocity while retaining the policy’s angular command.This lets the robot rotate away without advancing into an obstacle.Source: B. Policy Architecture · p. 3

C. Expert Data and Training

BCNav trains a depth-and-bearing policy from simulated shortest-path demonstrations, adding calibrated bearing noise and on-policy expert corrections to support navigation with imperfect DOA estimates.

C. Expert Data and Training

  • 40K episodes across 800 HM3D training scenes provide shortest-path demonstrations with expert velocity commands as regression targets.Start–goal geodesic path lengths range from 2 m to 15 m, with an average of 121 steps per episode.Source: C. Expert Data and Training · p. 3
  • DAgger records expert corrections from states visited by the current policy, helping it recover from mistakes near walls and after wrong turns.DAgger episodes make up 25% of each training batch.Source: C. Expert Data and Training · p. 3, Figure 3 · p. 4
  • Training corrupts oracle bearings with a mixture of Gaussian noise and uniformly random errors to model empirical DOA error and catastrophic failures.The Gaussian component uses σ=0.17 rad (about 10°), and the random component has probability p=0.15.Source: C. Expert Data and Training · pp. 3–4
  • Behavioral cloning minimizes mean squared error on the predicted velocity trajectory, with an auxiliary head predicting minimum depth in the lower field of view.The total loss combines trajectory and obstacle losses; training runs for 30 epochs with AdamW.Source: C. Expert Data and Training · p. 4

IV. SIMULATION EXPERIMENTS

The simulation experiments assess bearing-noise tolerance, compare SRP-PHAT accuracy with that tolerance, and test whether each system component contributes.

IV. SIMULATION EXPERIMENTS

  • The SoundSpaces 2.0 evaluation asks how navigation success varies with bearing noise, where SRP-PHAT accuracy lies on that tolerance curve, and whether each component contributes.Source: §IV · p. 4

A. Setup

Evaluation uses unseen HM3D scenes and random initial headings; the reported bearing-noise result defines an operating envelope for navigation success.

A. Setup

  • Evaluation covers 463 valid episodes from 100 unseen HM3D validation scenes, with five episodes per scene and random initial headings.Scenes with insufficient navigable area were filtered out.Source: A. Setup · p. 4
  • Success Rate counts episodes reaching within 1 m of the goal, while SPL weights success by path length; episodes are capped at 500 steps.Source: A. Setup · p. 4

B. Bearing Noise Tolerance

  • 96.5% SR is reached for σ ≤15°, while success drops to about 75% beyond 30°, where the bearing becomes uninformative and the policy falls back to depth-only obstacle avoidance.The authors state that DOA methods with MAE below 15° enable near-perfect navigation.Source: B. Bearing Noise Tolerance · p. 4

C. DOA Accuracy in Simulation

The closed-loop system reaches 71.1% success on unseen HM3D scenes using rendered DOA. Simulated SRP-PHAT bearings are generally usable under line of sight, but correlated NLOS errors and simplified acoustic rendering constrain performance.

C. DOA Accuracy in Simulation

  • Under acoustic line of sight, SRP-PHAT stays within the policy’s 15° robust range for typical indoor distances, despite gradual degradation with distance.The evaluation covers 8,000 source–receiver pairs across 16 Replica scenes.Source: C. DOA Accuracy in Simulation · p. 4
  • Over a third of simulated NLOS estimates remain within the robust range; mild occlusion preserves usable bearings, while severe occlusion produces effectively random estimates.Degradation correlates with occlusion severity, measured by the geodesic-to-Euclidean ratio.Source: C. DOA Accuracy in Simulation · p. 4

D. Closed-Loop Audio Navigation

  • 71.1% SR (SPL 0.709) on unseen HM3D scenes with continuous actions; comparison systems used discrete actions on Replica scenes seen during training.The evaluation uses rendered DOA and is described as a strictly harder setting than those prior evaluations.Source: D. Closed-Loop Audio Navigation · p. 4
  • Temporally correlated NLOS errors persist in the wrong direction, and sustained errors above 30° push navigation beyond its operating envelope.These errors do not average out as the independent-noise model predicts.Source: D. Closed-Loop Audio Navigation · p. 4
  • Rendered acoustic simulations use reduced ray counts and omit ego-noise and material properties, limiting RIR realism; learned DOA or temporal filtering could improve NLOS robustness without changing the policy.The proposed improvements target the DOA input rather than the navigation policy.Source: D. Closed-Loop Audio Navigation · pp. 4–5

E. Ablation Studies

Ablations show that depth-based obstacle avoidance contributes more than bearing alone, while the real-world evaluation measures DOA accuracy across environments and source distances. Accuracy is strongest at 2–3 m and degrades at near and far ranges.

E. Ablation Studies

  • The proportional controller reaches only 11.8% SR even with perfect bearing; removing the collision reflex cuts success by roughly a third, and waypoint prediction causes oscillations and collisions.The ablations indicate that depth-based obstacle avoidance, rather than bearing alone, drives navigation success.Source: E. Ablation Studies · p. 5

A. DOA Accuracy

  • At 1 m, SRP-PHAT MAE is below 2.4° in the semi-anechoic room and reaches 15.2° in the corridor’s worst-case direction.The paper attributes the corridor increase to multipath reflections; both measurements are within or near the policy’s robust range.Source: A. DOA Accuracy · p. 5
  • At 2–3 m, navigation-time DOA MAE is 2.9°, while near-field (<1 m) and far-field (>4 m) errors rise to 15°–20°.The figure marks the ±15° robust operating range; the paper attributes extreme-range degradation to small-aperture arrays.Source: A. DOA Accuracy · p. 5, Figure 5 · p. 5

V. REAL-WORLD EXPERIMENTS

  • The real-world evaluation uses a semi-anechoic room with obstacles and an L-shaped corridor; the full pipeline runs on a rover with a depth camera and microphone array.Trials use arbitrary music from a Bluetooth speaker, with the source initially outside the camera’s field of view.Source: Figure 4 · p. 5, §V · p. 5

B. Navigation Results

BCNav transfers to physical navigation in a semi-anechoic room and a straight corridor, while a corner source exposes sensitivity to degraded DOA estimates. Onboard replanning fits the 10 Hz control budget, though the depth camera’s limited field of view leaves a collision risk.

a) Semi-anechoic room

  • The sim-trained policy transferred without fine-tuning in stationary and moving-source room trials, issuing smooth commands at 0.3 m/s and 1 rad/s with heading oscillation below 5° within 3 m.The primary failure mode was collision with obstacles outside the depth camera’s 60° horizontal field of view during sharp turns; the collision reflex or wider-FOV depth sensors mitigate this.Source: B. Navigation Results · p. 5

b) Corridor, straight path

  • The robot successfully navigated the full ∼8 m straight corridor, with DOA accuracy remaining within the robust range under line-of-sight conditions.Source: B. Navigation Results · p. 5

c) Corridor, T-junction

  • At the T-junction, corner attenuation and wall reverberation degraded SRP-PHAT bearings, making the robot oscillate beside a wall; at σ ≥30°, SR falls below 75% even in simulation.The robot approached the source, but bearings consistently pointed toward walls and changed rapidly.Source: B. Navigation Results · pp. 5–6

d) Onboard compute

  • On Jetson AGX Orin, policy and safety-filter medians were 15.5 ms and 9.2 ms per step, enabling ∼25 ms replanning within the 10 Hz budget; SRP-PHAT took 115 ms per chunk.Bearing updates were asynchronous at ∼9 Hz, sufficient for the slowly varying bearing; the learned policy was not the compute bottleneck.Source: B. Navigation Results · p. 6

VI. CONCLUSION

BCNav separates DOA estimation from a policy that uses depth and scalar bearing, and real-world experiments confirm zero-shot transfer. The conclusion identifies NLOS, limited real-world validation, and a single-source assumption as boundaries.

  • BCNav decouples acoustic sensing from the learned policy through DOA estimation, letting the policy use depth and a scalar bearing to sidestep the acoustic sim-to-real gap.Source: §VI · p. 6
  • Real-world experiments confirm zero-shot transfer, and the bearing-noise tolerance curve predicts navigation success from a DOA method’s accuracy without retraining.Source: §VI · p. 6
  • The real-world validation covers only two environments, DOA degrades under strong NLOS, and the policy assumes a single active source.Future work targets learned DOA for NLOS robustness and multi-source navigation.Source: §VI · p. 6

Related papers

Loading 2609.37084v1…