PaperlayerView exact revision on arXiv

Source-linked AI summary

Visual-Wi-Fi Navigation to Signal-Emitting Devices

Lihao Wang, Tianjian Xu, Tianmin Shu, Renjie Zhao

arXiv:2610.12733v1cs.RO

TL;DR

Navigation to small, occluded, or visually indistinct targets lacks a reliable visual cue, while ordinary devices emit Wi-Fi that can provide spatial information. The paper combines online ray-traced Wi-Fi sensing with motion-aware source localization and a learned visual-Wi-Fi policy. Fine-tuning improves success, the policy transfers zero-shot to unseen scenes, and it is deployed on a real robot without real-world training.

  • Problem

    Visual navigation can lack an explicit spatial cue when a target is small, occluded, visually indistinct, or has no direct visual manifestation, while ordinary devices emit potentially useful Wi-Fi signals.

    Source: §I · p. 1

  • Method

    WiFiSpaces renders online ray-traced Wi-Fi observations at continuous poses, while ViFiNav uses robot motion to fuse bearings into a confidence-aware source estimate for a learned policy.

    Source: §I · pp. 1–2

  • Results

    Fine-tuning on Wi-Fi observations in WiFiSpaces improves ViFiNav’s success rate; the policy generalizes zero-shot across indoor scene datasets and was deployed on a real robot without real-world training.

    Source: §I · p. 2

  • Takeaways & Limitations

    The system demonstrates visual-Wi-Fi navigation using packets from ordinary device traffic and a commodity-router sensor without connecting to the target or training on real-world data.

    Source: §I · pp. 1–2

Abstract

from arXiv · show

Indoor navigation to task-relevant targets remains difficult when the target is small, occluded, or visually indistinct. Meanwhile, smart appliances, mobile devices, and IoT tags emit Wi-Fi signals during everyday communication, providing a pervasive but underused spatial cue. We study visual-Wi-Fi navigation, where a robot reaches a signal-emitting device in an unmapped environment using onboard visual observations and Wi-Fi packets overheard from the device, without connecting to the device or modifying infrastructure. Realizing this requires pose-dependent Wi-Fi observations during closed-loop policy learning and reliable spatial cues from bearing measurements corrupted by indoor multipath propagation on a moving robot. We present WiFiSpaces, which extends the Habitat embodied simulator with online ray-traced Wi-Fi sensing, rendering Wi-Fi observations for a robot-mounted antenna array at continuous poses during policy rollouts. On top of it, we build ViFiNav, a localization-aware navigation framework: an online source-localization algorithm exploits robot motion to disambiguate and robustly aggregate bearings from multiple viewpoints into a confidence-aware source estimate, and a learned policy fuses this evolving estimate with depth observations and learns to act on it as it is refined. Experiments show that exploiting robot motion is key to accurate source estimates, and that fine-tuning on ray-traced Wi-Fi observations in WiFiSpaces improves the success rate of ViFiNav. The policy transfers zero-shot to unseen scene datasets, and we deploy it on a real robot with a Wi-Fi sensor built from a commodity router, without any real-world training.

I. INTRODUCTION

The paper formulates navigation to a signal-emitting device in an unmapped environment, where visual cues may be inadequate and Wi-Fi offers a complementary spatial cue. It introduces online ray-traced sensing and a localization-aware learned policy, with simulation improvements and zero-shot transfer reported.

I. INTRODUCTION

  • Visual–Wi-Fi navigation uses visual observations and packets overheard from a target device to reach its location in an unmapped environment, even when the target is not visible.The task addresses targets that are small, occluded, visually indistinct, or lack a direct visual manifestation.Source: §I · pp. 1–2
  • Wi-Fi provides a spatial cue beyond visual line of sight because ordinary devices emit it during communication and signals propagate through and around obstacles.Packets can be overheard by a commodity router without connecting to the device or its network; Wi-Fi is a deployment choice rather than a requirement of the broader setting.Source: §I · p. 1
  • ViFiNav uses robot motion to disambiguate and aggregate multipath-corrupted bearings into a confidence-aware source estimate, which a learned policy fuses with visual observations.The policy acts on the evolving estimate in closed loop rather than heuristically following each intermediate estimate.Source: §I · p. 2
  • WiFiSpaces extends Habitat with online ray-traced Wi-Fi observations synchronized to continuous agent poses during policy rollouts.The observations reflect source location and scene geometry, including propagation, reflection, and blockage.Source: §I · pp. 1–2
  • Fine-tuning on ray-traced Wi-Fi observations improves ViFiNav’s success rate, and the policy generalizes zero-shot across indoor scene datasets and runs on a real robot without real-world training.The real robot uses a commodity-router Wi-Fi sensor.Source: §I · p. 2

II. RELATED WORK

Prior navigation and wireless-localization systems leave a gap between embodied visual navigation and pose-dependent radio sensing. WiFiSpaces places Wi-Fi observations inside an embodied simulator’s step loop for closed-loop navigation learning.

II. RELATED WORK

  • Visual navigation can depend on target visibility or learned semantic priors, while audio-visual navigation requires the target to continuously emit informative sound.Wi-Fi signals instead arise from ordinary device traffic and provide spatial cues beyond visibility.Source: §II · p. 2
  • Existing indoor wireless localization systems often rely on site-specific data or calibrated anchors, limiting map-free use.Robot-motion AoA approaches use active infrastructure coordination and address localization or tracking rather than closed-loop navigation.Source: §II · p. 2
  • Embodied simulators support navigation and policy learning but lack radio propagation, whereas wireless simulators model propagation without embodied environments.Related platforms include outdoor 6G sensing and RF-guided Habitat navigation evaluated only in simulation.Source: §II · p. 2
  • WiFiSpaces renders robot-mounted-array CSI at each indoor agent pose and synchronizes it with visual observations and actions in the embodied simulator step loop.This makes policies trainable and evaluable in closed loop with pose-dependent Wi-Fi sensing.Source: §II · p. 2

III. TASK DEFINITION

The task is to reach a stationary Wi-Fi-emitting device at an unknown location in an unmapped indoor scene using onboard observations. The target is identified by its MAC address, and its location is inferred from Wi-Fi while vision supplies scene context.

III. TASK DEFINITION

  • Unlike PointGoal or ObjectGoal navigation, the agent receives neither a goal vector nor a visual target description; the target is identified only by its MAC address.The simulator uses the hidden target location to generate Wi-Fi observations and evaluate success, but does not expose it to the agent.Source: §III · pp. 2–3
  • At each step, the agent receives egocentric RGB-D observations, Wi-Fi CSI, and its onboard odometry pose, then chooses a standard Habitat action.The listed actions are move forward, turn left, turn right, and stop.Source: §III · p. 2
  • The system overview connects CSI and robot pose to an online source estimate, which the policy fuses with depth and previous-action features to select an action.A successful episode requires stopping within the source’s success distance before the 500-step limit.Source: Figure 2 · p. 3, §III · p. 3

IV. WIFISPACES: EXTENDING HABITAT WITH CONTINUOUS WI-FI SENSING

WiFiSpaces adds continuous-pose, ray-traced Wi-Fi sensing to Habitat, while the online localizer uses robot motion and robust multi-view fusion to turn ambiguous bearings into a confidence-tagged source estimate.

IV. WIFISPACES: EXTENDING HABITAT WITH CONTINUOUS WI-FI SENSING

  • WiFiSpaces ray-traces pose-dependent channels at each Habitat step and exposes synchronized CSI as a native observation for policy learning.It traces antenna elements individually and models line-of-sight, reflection, refraction, and diffraction.Source: §IV · p. 3
  • Persistent scene contexts and reusable static wireless meshes roughly halve Wi-Fi observation render time and GPU memory per worker.The comparison is against per-query scene construction on the full scene mesh, measured on an NVIDIA L40S GPU.Source: §IV · p. 3

A. Online Wi-Fi Source Localization

  • The localizer returns the estimated source range and bearing in the agent frame with a confidence score, freezing the estimate only after stability and support criteria are met.It requires mean displacement below 0.1 m over 50 updates and at least 50 verified bearings before confidence reaches 1.Source: A. Online Wi-Fi Source Localization · pp. 3–4
  • A ULA bearing has front–back ambiguity, but in-place rotation distinguishes the true world-fixed candidate from its rotating mirror.Near-in-place measurement pairs agreeing within 15° yield verified bearings, using turns made during navigation rather than a dedicated scan.Source: A. Online Wi-Fi Source Localization · p. 4
  • RANSAC-style fusion intersects sampled bearing rays and selects a source hypothesis with the largest inlier support, requiring at least six inliers.The estimator uses 100 iterations and a range-adaptive inlier threshold.Source: A. Online Wi-Fi Source Localization · p. 4

B. Localization-Aware Multimodal Policy and Training

The policy fuses depth with the evolving Wi-Fi source estimate and uses recurrent state to decide whether to follow the estimate or continue gathering viewpoints. Training first uses analytic ground-truth AoA, then fine-tunes on ray-traced Wi-Fi observations.

B. Localization-Aware Multimodal Policy and Training

  • A frozen depth encoder supplies geometry for obstacle avoidance, while an MLP embeds the Wi-Fi cue and a learned embedding represents the previous action.The concatenated features feed an LSTM and actor–critic heads.Source: B. Localization-Aware Multimodal… · p. 4
  • The LSTM tracks how the source estimate changes, letting the policy keep moving for new viewpoints instead of following every intermediate estimate.Source: B. Localization-Aware Multimodal… · p. 4
  • Training pretrains on analytic ground-truth AoA, then fine-tunes with CSI processed by the full localizer because direct ray-traced Wi-Fi training is slow and unstable.Only the cue MLP input layer is reinitialized; the recurrent core and heads carry over.Source: B. Localization-Aware Multimodal… · p. 4

VI. SYSTEM IMPLEMENTATION

The real-robot system uses a commodity router configured as a four-antenna ULA, and the simulated sensing and navigation setup is configured to match the platform.

VI. SYSTEM IMPLEMENTATION

  • The router’s four external antennas are fixed as a 2.6 cm-spaced ULA, and its CSI is captured in monitor mode without connecting to the target device or network.The router is mounted on a Hello Robot Stretch 3; onboard odometry and LiDAR scan matching provide pose without a prior map or external localization.Source: §VI · p. 5, Figure 5 · p. 5
  • WiFiSpaces matches the platform with a single-antenna transmitter, a 1 × 4 receiver array, 5.775 GHz carrier, and 80 MHz bandwidth.The simulated array spacing and antenna heights also match the platform configuration.Source: §VI · p. 5
  • The depth encoder is a frozen Habitat PointNav-pretrained ResNet-50, and the recurrent policy uses an LSTM with hidden size 512.Training uses 60M ground-truth-AoA pretraining steps and 12M ray-traced Wi-Fi fine-tuning steps.Source: §VI · p. 5

A. Experimental Setup

On HSSD test scenes, Wi-Fi fine-tuning improves ViFiNav over the same policy without fine-tuning, while oracle cues provide an upper-bound comparison. Episodes use targets mostly out of sight, and the control policy without wireless cues rarely succeeds.

A. Experimental Setup

  • 95.1% of episode start poses have the target out of sight, and the HSSD test set contains 1,260 episodes across 42 test scenes.Episodes keep start-target pairs with a geodesic-to-Euclidean distance ratio of at least 1.1.Source: A. Experimental Setup · p. 5

B. VIFINAV Learns Effective Navigation in WIFISPACES

  • 68.57% SR and 44.12% SPL: fine-tuned ViFiNav on ray-traced Wi-Fi observations; final localization error falls from 2.56 m to 1.24 m.The same policy without fine-tuning reaches 53.02% SR; PointNav reaches 49.76% SR and Loc-then-PointNav 17.78% SR.Source: B. VIFINAV Learns Effective Navigation… · p. 6
  • 87.22% SR: PointNav (oracle) with a ground-truth goal cue, compared with 82.30% SR for ViFiNav (oracle).ViFiNav’s oracle cue is front–back ambiguous and has no range, so its localizer must resolve the cue through motion.Source: B. VIFINAV Learns Effective Navigation… · pp. 5–6
  • 0.16% SR: no-goal ViFiNav, which receives depth and the previous action but no wireless cue.This control is reported as evidence that exploration or obstacle avoidance alone does not solve these episodes.Source: B. VIFINAV Learns Effective Navigation… · p. 6

C. Component Analysis of VIFINAV

Component tests show that ambiguity resolution and robust bearing fusion improve navigation, while two-stage training adds further gains in success and path efficiency.

C. Component Analysis of VIFINAV

  • For LS, resolving front–back ambiguity raises SR from 1.75% to 43.33% and reduces localization error from 48.74 m to 4.34 m; RANSAC alone reaches 47.62% SR and 2.03 m.Using both ambiguity resolution and robust fusion yields 52.22% SR and 1.26 m localization error.Source: C. Component Analysis of VIFINAV · p. 6
  • Across three HSSD episodes, ViFiNav’s estimates move toward the source and keep its path short, unlike detouring PointNav and frozen-estimate Loc-then-PointNav.Loc-then-PointNav cannot recover when its early estimate is wrong.Source: Figure 6 · p. 6
  • Motion-based ambiguity resolution reduces mean direction error from 12.2° to 5.1°, while mean localization error falls from 7.5 m in the lowest-confidence bin to 0.3 m in the highest.Raw AoA has a median direction error of 1.3° despite its heavy-tailed mean error.Source: Figure 7 · p. 6, C. Component Analysis of VIFINAV · pp. 6–7
  • 52.22% → 68.57% SR and 30.17% → 44.12% SPL; after two-stage training, final localization error changes only from 1.26 to 1.24 m.The remaining gain is in policy training rather than the estimator.Source: C. Component Analysis of VIFINAV · p. 6

D. Cross-Dataset Generalization

The HSSD-trained policy transfers zero-shot to ReplicaCAD and ProcTHOR, and its Wi-Fi target cue distinguishes a designated instance where category-based ObjectNav does not.

D. Cross-Dataset Generalization

  • VIFINAV reaches 75.05% SR on ReplicaCAD and 85.22% SR on ProcTHOR without fine-tuning, showing transfer to scene families beyond HSSD.The higher absolute success rates partly reflect shorter geodesic distances in the unseen datasets.Source: D. Cross-Dataset Generalization · p. 7

E. Comparison with Object-Goal Navigation

  • ObjectNav specifies a goal by category, whereas Wi-Fi identifies a specific instance by its MAC address.For the comparison, the Wi-Fi source is placed at a navigable viewpoint of the designated instance.Source: E. Comparison with Object-Goal… · p. 7
  • On designated-instance episodes, VIFINAV reaches 56.29% SR and 30.40% SPL, versus 25.67% instance-level SR for the official ObjectNav policy.The matched ObjectNav policy reaches 25.07% SR; VIFINAV’s result is below Table II because these episodes are out of distribution.Source: E. Comparison with Object-Goal… · p. 7
  • The official ObjectNav policy’s SR drops from 60.95% at category level to 25.67% for a designated instance, while mean goal distance rises from 2.68 m to 7.55 m.It stops near an object of the right category but lacks a cue to prefer the designated instance.Source: E. Comparison with Object-Goal… · p. 7

F. Real-World Deployment

ViFiNav succeeds in real-world line-of-sight and non-line-of-sight trials without real-world fine-tuning; the conclusion identifies sim-to-real and traffic conditions for future work.

F. Real-World Deployment

  • Without real-world fine-tuning, ViFiNav succeeds in 7 of 10 line-of-sight trials and 5 of 10 non-line-of-sight trials.The online source estimate’s mean final error is 1.16 m in line of sight, 1.92 m in non-line of sight, and 1.54 m overall; package search is also demonstrated.Source: F. Real-World Deployment · p. 7

VIII. CONCLUSION AND FUTURE WORK

  • The same policy runs on a real robot with a Wi-Fi sensor built from a commodity router and no real-world training.The conclusion also reports zero-shot transfer to unseen scene datasets and improved simulation success after fine-tuning on ray-traced Wi-Fi observations.Source: §VIII · pp. 7–8
  • Future work targets the sim-to-real gap with randomized surface materials and scanned-scene ray tracing, and calls for testing under sparse or bursty target traffic.The localization front end is Wi-Fi-specific, while the simulator exposes carrier, bandwidth, array geometry, and materials as configuration.Source: §VIII · p. 8

Related papers

Loading 2610.12733v1…