PaperlayerView exact revision on arXiv

Source-linked AI summary

Navigation with RF Cues: Embodied Perception Action under Multipath Uncertainty

Wenlihan Lu, Tianshun Li, Liuqing Yang, Shijian Gao

arXiv:2610.08105v1cs.RO

TL;DR

Multipath can distort RF direction cues, and existing data or simulations have limits for detailed, interactive navigation evaluation. The paper builds an action-conditioned Habitat–Sionna RT benchmark and a multimodal framework that estimates target direction and uncertainty from observation history. In unseen scenes, it reports relative improvements of 18.2% in SR and 11.5% in SPL over SAVN-CE, the strongest evaluated baseline.

  • Problem

    Multipath can make instantaneous RF direction cues unreliable, while fixed-trajectory datasets and simplified simulation geometry limit interactive evaluation in detailed scenes.

    Source: §I · p. 1

  • Method

    The paper builds an action-conditioned Habitat–Sionna RT benchmark and combines an uncertainty-aware RF goal predictor with multimodal scene memory for navigation.

    Source: §I · p. 1

  • Results

    18.2% relative SR and 11.5% relative SPL improvements over SAVN-CE, the strongest evaluated baseline, across 1,350 episodes in 27 unseen scenes.

    Source: §I · p. 1, B. Comparison Methods and Results · p. 5

  • Takeaways & Limitations

    Experiments across 27 unseen scenes demonstrate improved navigation success and path efficiency over the evaluated baselines.

    Source: §V · p. 6

    Real-robot validation and larger antenna arrays for improved angular resolution are left for future work.

    Source: §V · p. 6

Abstract

from arXiv · show

Smart factory inspection requires robots to reach connected equipment without a prior map or known target coordinates. Radio frequency (RF) signals from the target can provide directional cues to complement visual observations when occlusion or poor lighting limits target detection. However, multipath propagation can distort these cues, making it difficult to infer the target's true direction from instantaneous RF measurements. To enable navigation research under these conditions, we first construct a Habitat Sionna RT benchmark that uses detailed scene geometry and assigned material properties to generate aligned visual and RF observations in response to robot actions. Building on this benchmark, we propose an uncertainty aware multimodal navigation framework that jointly estimates target direction and its uncertainty from a history of RF, visual, and pose observations. These estimates inform action selection alongside visual context. Experiments in unseen scenes show relative improvements of 18.2% in success rate (SR) and 11.5% in success weighted by path length (SPL) over the strongest evaluated baseline.

I. INTRODUCTION

The paper addresses target navigation when multipath distorts RF direction cues and fixed-trajectory data or simplified geometry limit interactive evaluation. It introduces an action-conditioned Habitat–Sionna RT benchmark and a multimodal, uncertainty-aware navigation framework.

I. INTRODUCTION

  • Multipath reflections can skew direction-of-arrival estimates, while interference and changing paths make instantaneous RF cues unreliable for consistent navigation.Stronger reception does not necessarily mean the robot is approaching the target.Source: §I · p. 1
  • Fixed-trajectory Wi-Fi datasets cannot provide observations for arbitrary policy actions, while a prior simulator’s floorplan geometry omits finer scene details.These gaps motivate an interactive benchmark with detailed geometry and aligned visual and RF observations generated in response to robot actions.Source: §I · p. 1
  • Across 1,350 episodes in 27 unseen scenes, the method achieves 0.546 SR and 0.456 SPL, relative improvements of 18.2% and 11.5% over the strongest evaluated baseline.The framework estimates target-bearing distributions from RF, RGB, depth, and pose histories and integrates them with visual and pose features for action selection.Source: §I · p. 1

II. PROBLEM STATEMENT

  • The task is to reach and stop near an unknown goal without a prior map or goal coordinates, using visual, motion, and RF observations.The robot receives new observations after discrete forward and turning actions.Source: §II · p. 2, A. Task Setup · p. 2

B. Task Objective

The task objective combines successful stopping near the goal with efficient travel, using expected SPL to evaluate both.

B. Task Objective

  • The navigation policy chooses actions using all observations received so far and preceding actions.An episode terminates on STOP or at the action limit.Source: B. Task Objective · p. 2
  • An episode ends when the robot selects STOP or reaches the maximum action count, and success requires stopping within the goal-distance threshold.The threshold is measured using geodesic distance along traversable paths.Source: B. Task Objective · p. 2
  • The objective is to maximize expected SPL, balancing success with travel distance relative to the shortest path from the start to the goal.The expectation is over policy-induced trajectories and the distribution of scenes, initial poses, and goal locations.Source: B. Task Objective · p. 2

III. PROPOSED SOLUTION

The proposed solution combines RF direction cues with visual and pose context in a memory-based navigation policy. Its RF encoding retains the spatial spectrum because the strongest peak may reflect a multipath arrival rather than the goal bearing.

III. PROPOSED SOLUTION

  • The framework estimates a distribution over goal direction from observation history, then integrates it with visual and pose features over time to guide navigation.The scene memory transformer uses historical context retrieved by a query from the current observation.Source: §III · p. 2, Figure 1 · p. 3

A. RF and Visual Observation Encoding

  • The RF front end collects RSSI and a MUSIC spatial spectrum using a forward-facing 1×2 patch antenna array.The spectrum is formed using the array steering vectors and an estimated noise subspace.Source: A. RF and Visual Observation Encoding · p. 2
  • Because the dominant MUSIC peak may represent a reflected arrival, the RF observation retains the sampled spectrum and RSSI for later direction and uncertainty prediction.This preserves directional information beyond a single peak estimate.Source: A. RF and Visual Observation Encoding · p. 2
  • RF features provide directional cues, RGB and depth provide scene context, and pose features describe robot motion; their histories are fused for action selection.The resulting goal token is integrated with visual and pose tokens in scene memory to assess motion against obstacles and traversable space.Source: A. RF and Visual Observation Encoding · pp. 2–3

B. Uncertainty-Aware Goal Predictor

The goal predictor uses multimodal observation history to estimate a target bearing and its uncertainty. It represents the bearing with a von Mises distribution and passes direction and concentration into the scene memory transformer.

B. Uncertainty-Aware Goal Predictor

  • The predictor processes RF, RGB, depth, and pose history to estimate source bearing and uncertainty rather than relying on instantaneous MUSIC peaks.The current representation combines directional evidence with visual scene context across observations and robot motion.Source: B. Uncertainty-Aware Goal Predictor · p. 3
  • A von Mises distribution represents the periodic bearing with mean direction μ_t and nonnegative concentration κ_t, where larger κ_t indicates a more concentrated prediction.Ground-truth bearing supervision is used during training.Source: B. Uncertainty-Aware Goal Predictor · p. 3
  • The goal token encodes the mean direction as sine and cosine components and includes κ_t to carry directional uncertainty into the scene memory transformer.The token is formed from the predicted distribution parameters.Source: B. Uncertainty-Aware Goal Predictor · p. 3

C. Memory-Based Navigation Policy

The navigation policy augments scene memory with an uncertainty-aware goal token derived from RF, visual, and pose history. It retrieves historical context to inform action selection and trains the policy for navigation progress and successful stopping.

C. Memory-Based Navigation Policy

  • The adapted scene memory transformer combines visual and pose tokens with a goal token encoding estimated direction and its uncertainty.The goal predictor processes RF features together with visual and pose history; scene memory retains historical goal estimates alongside scene and motion context.Source: C. Memory-Based Navigation Policy · p. 3
  • Cross-attention retrieves relevant historical features from scene memory for use with the current observation representation.Memory stores current tokens after each decision and resets at the start of each episode.Source: C. Memory-Based Navigation Policy · p. 4
  • The actor maps the policy feature to a distribution over four navigation actions, while the critic estimates expected return.The policy acts on interaction history comprising observations received up to that time and preceding actions.Source: C. Memory-Based Navigation Policy · p. 4
  • The navigation reward encourages progress toward the goal and successful stopping, using distance and success signals that are not policy inputs.The reward assigns positive coefficients to progress and successful termination and a penalty to unsuccessful termination.Source: C. Memory-Based Navigation Policy · p. 4
  • PPO training combines the policy and value losses, policy entropy, and an auxiliary goal-prediction loss.The prediction objective learns a directional representation with uncertainty; the navigation objective learns to use it with scene memory for action selection.Source: C. Memory-Based Navigation Policy · p. 4

A. Benchmark and Experimental Setup

The benchmark pairs Habitat visual observations with Sionna RT RF simulation over MP3D scenes. Experiments use a held-out scene split and characterize both propagation conditions and navigation performance measures.

A. Benchmark and Experimental Setup

  • The study uses 90 MP3D scenes, with 63 for training and 27 held out; evaluation comprises 1,350 episodes at geodesic start-to-goal distances of 8–20 m.Training locations are sampled by Habitat Pathfinder, and evaluation uses 50 fixed episodes per held-out scene.Source: A. Benchmark and Experimental Setup · p. 4, Table II · p. 5
  • The benchmark pairs egocentric Habitat RGB-D observations with RF observations simulated in Sionna RT using semantically segmented MP3D meshes and assigned electromagnetic materials.A fixed coordinate transform aligns robot poses and transmitter positions across the simulators.Source: Figure 2 · p. 4, A. Benchmark and Experimental Setup · p. 5
  • 67.1% of shortest-path oracle observations are NLOS, compared with 27.4% LOS and 5.5% invalid observations.The figure depicts the LOS, NLOS, and invalid observation distribution; indirect propagation therefore dominates even along shortest paths.Source: Figure 3 · p. 4, A. Benchmark and Experimental Setup · p. 5
  • The qualitative trajectories mark start and goal locations and color route segments by LOS, NLOS, or unavailable RF signals.The examples illustrate trajectories under varying propagation conditions.Source: Figure 4 · p. 5
  • Navigation performance is reported using success rate, success weighted by path length, and final distance to goal.Higher SR and SPL and lower DTG indicate better performance.Source: A. Benchmark and Experimental Setup · p. 5

B. Comparison Methods and Results

On held-out scenes, the proposed method achieves higher SR and SPL than the strongest evaluated baseline, SAVN-CE. Comparisons include RF-only controllers and adapted learned navigation methods.

B. Comparison Methods and Results

  • The learned comparison methods include mmWave-LSC, which selects between directional guidance and visual exploration, and PiPRL, which uses symbolic programs and estimated link states.SAVN-CE provides a memory-based baseline with a deterministic goal representation.Source: B. Comparison Methods and Results · p. 5
  • 18.2% higher SR and 11.5% higher SPL than SAVN-CE are the proposed method’s relative improvements; its SR is 0.546 and SPL is 0.456.SAVN-CE, the strongest baseline, achieves SR 0.462 and SPL 0.409; the proposed method also reduces DTG from 5.810 m to 4.176 m.Source: B. Comparison Methods and Results · p. 5
  • The RSSI-gradient and MUSIC-DOA controllers achieve SRs of 0.002 and 0.074, respectively, while the comparison set also includes a random policy.The RSSI-gradient controller follows changes in signal strength, while MUSIC-DOA uses the spatial-spectrum peak as its bearing estimate.Source: B. Comparison Methods and Results · p. 5
  • Among adapted learned baselines, PiPRL achieves SR 0.328 and SPL 0.264, while SAVN-CE is the strongest baseline at SR 0.462 and SPL 0.409.All adapted learned baselines are retrained on the same training split.Source: B. Comparison Methods and Results · p. 5

C. Ablation Study

The ablations examine the effects of directional uncertainty, RF history, and reduced RGB exposure on navigation performance in unseen scenes.

C. Ablation Study

  • Removing κ from the goal token reduces SR from 0.546 to 0.489 and SPL from 0.456 to 0.420.Table III reports ablation results on unseen scenes.Source: C. Ablation Study · p. 6, Table III · p. 6
  • Using only current-step RF observations yields 0.263 SR and 0.217 SPL, the largest reported ablation drop.The comparison highlights the importance of RF history.Source: C. Ablation Study · p. 6, Table III · p. 6
  • At exposure 0.7, SR falls from 0.546 to 0.515 and SPL from 0.456 to 0.436; at 0.5, SR is 0.511 and SPL remains 0.436.The reported absolute decreases at exposure 0.5 are 0.035 SR and 0.020 SPL, indicating modest degradation across the tested reductions.Source: C. Ablation Study · p. 6, Table IV · p. 6

V. CONCLUSION

Across 27 unseen scenes, the method improves navigation success and path efficiency over evaluated baselines. The paper also presents a material-aware Habitat–Sionna RT benchmark and identifies real-robot validation and larger antenna arrays as future work.

V. CONCLUSION

  • Across 27 unseen scenes, the method improves navigation success and path efficiency over the evaluated baselines.The conclusion identifies real-robot validation and larger antenna arrays for improved angular resolution as future work.Source: §V · p. 6
  • The navigation method combines an uncertainty-aware RF goal representation with multimodal memory to address unreliable directional cues under multipath propagation.Source: §V · p. 6
  • The Habitat–Sionna RT benchmark assigns electromagnetic properties to semantically segmented MP3D meshes while preserving detailed geometry and generating spatially aligned visual and RF observations as the robot moves.Source: §V · p. 6

Related papers

Loading 2610.08105v1…