Source-linked AI summary
EgoNav: Bridging Learned Waypoints and Geometry-Aware Local Control for Robust Indoor Navigation
Jing Wang, Shiqi Zhao, Hairong Qu, Peng Yin
TL;DR
EgoNav addresses unsafe and directionally inconsistent learned waypoints, along with local-planning limitations, in lightweight topological-map image-goal navigation. It refines learned predictions using traversability-based geometric scoring and executes them with correction-aware adaptive planning. Simulation and humanoid-robot experiments report consistent gains in success rate and path efficiency over contemporary baselines.
Problem
Learned waypoint predictors lack explicit geometric guarantees, while existing image-goal systems often lack collision-avoiding planners or use fixed parameters that cannot adapt to confined spaces.
Method
EgoNav combines active topological-map guidance, geometry-informed waypoint refinement, and an adaptive local planner whose parameters respond to refinement outcomes.
Results
EgoNav consistently improves navigation success rate and path efficiency over contemporary baselines in simulation and on a physical humanoid robot.
Takeaways & Limitations
The system-level coupling of learned visual guidance, geometric correction, and adaptive control addresses deployment failure modes without replacing the learned waypoint prior.
Takeaways & Limitations
Waypoint generalization is bounded by the training distribution, and avoidance detours can leave the map’s visual coverage, increasing matching failures under viewpoint change.
Abstract
from arXiv · showhide
Image-goal navigation using lightweight topological maps is a practical paradigm for indoor robot deployment: the map requires only geotagged images, and localization relies on visual matching rather than precise pose estimation. However, learned waypoint predictors can produce targets that violate geometric constraints or deviate from the global path. Executing these waypoints safely further requires a local planner capable of collision avoidance, yet existing systems either lack one or rely on fixed parameters that cannot adapt to confined spaces. To address these limitations while retaining the navigational intuition of the learned predictor, we present EgoNav, a hierarchical system that implements this idea by generating candidates from semantically segmented traversable regions and scoring them alongside the learned waypoint for geometric safety, directional coherence, and fidelity to the learned prior. An adaptive local path planner then executes the refined waypoint with parameters modulated based on the refinement outcome. Experiments in Habitat-sim and on a physical humanoid robot show that EgoNav consistently outperforms contemporary baselines in both success rate and path efficiency.
I. INTRODUCTION
EgoNav targets practical indoor image-goal navigation by combining lightweight topological-map guidance with geometry-aware waypoint refinement and adaptive local control. It addresses unsafe or directionally inconsistent learned waypoints while preserving their useful navigation priors.
- Precise metric maps and global localization are fragile under drift and environmental change, motivating global path awareness without metric infrastructure.
- Image-goal navigation uses geotagged images and visual matching, avoiding absolute pose estimation and the inefficiency or instruction burden of mapless alternatives.
- Learned waypoints may intersect obstacles, violate traversability, or deviate from the intended global direction, while fixed or absent local planners limit safe execution in confined spaces.
- EgoNav couples learned waypoint prediction, geometry-informed refinement, and adaptive local planning through explicit information flow.
- The refinement module samples traversable-region candidates and scores geometric safety, global-direction coherence, and fidelity to the learned prior.
II. RELATED WORK
Prior navigation approaches differ in goal specification, map dependence, and local trajectory generation. EgoNav combines image-goal navigation with active map-guided subgoal retrieval, learned waypoint prediction, refinement, and adaptive local planning.
- Visual navigation: Visual navigation includes ImageNav, ObjectNav, and VLN, whose goals are specified by target images, object categories, and natural-language instructions respectively.
- Visual navigation: ObjectNav often requires inefficient long-distance exploration without a prior map, while VLN depends on detailed step-by-step instructions.
- Local path planning: Local planners include reactive, sampling-based, optimization-based, and path-deformation methods, trading among completeness, speed, local optimality, and smooth obstacle avoidance.
- EgoNav positioning: EgoNav assumes an RGB-D robot and a pre-collected topological map whose nodes store reference images and poses, with edges connecting traversed neighboring locations.
- EgoNav positioning: Its hierarchy retrieves map subgoals with VPR, predicts waypoints with a pretrained network, refines candidates, and generates trajectories with an adaptive planner.
B. Map-Guided Waypoint Prediction
EgoNav uses active visual place recognition to keep learned waypoint prediction aligned with the robot’s position along a planned topological route. Filtering addresses localization drift, while geometric correction remains necessary because the predictor lacks explicit geometric reasoning.
- Active subgoal retrieval localizes the robot with VPR and computes a shortest topological route using Dijkstra’s algorithm.
- GNM predicts an egocentric waypoint from the current image and a subgoal image selected ahead along the route.
- VPR matching can regress backward under perceptual aliasing, pulling subgoals toward previously visited locations and causing path retracing.
- The system prevents backward index movement by enforcing the current match to be no earlier than the previous match.
- Large implausible forward jumps are detected using a maximum per-cycle advance threshold, triggering corrective handling when exceeded.
- Because GNM lacks explicit geometric reasoning, its waypoints may approach obstacles, deviate under visual ambiguity, or fail in unseen environments, requiring depth- and segmentation-based correction.
C. Geometry-Informed Waypoint Refinement
EgoNav refines learned waypoints by evaluating geometrically grounded alternatives against safety, global-direction coherence, and prior fidelity. The selected correction provides information for subsequent adaptive local planning.
- Learned waypoint priors such as center-biased and smooth motion are preserved while geometric and directional inconsistencies are corrected.
- Candidate sampling: Semantic segmentation produces traversability-guided candidates, while a depth consistency check can bypass refinement when segmentation appears unreliable.
- Candidate sampling: The learned waypoint remains in the candidate set, allowing it to be selected when scoring finds it adequate.
- Waypoint scoring: Candidates are scored with a weighted energy function designed to correct waypoint failure modes while preserving the learned predictor’s prior.
- Waypoint scoring: The safety term sharply penalizes candidates within the robot-derived clearance threshold and assigns near-zero cost to candidates farther from obstacles.
- Waypoint scoring: The coherence term compares candidate direction with the global guidance vector, where zero denotes perfect alignment.
- Waypoint scoring: The prior term penalizes excessive deviation from the prediction, while the original waypoint receives zero prior cost and is displaced only when geometry strongly favors another candidate.
- Waypoint output: The module outputs a corrected waypoint, correction magnitude, and obstacle clearance for adaptive downstream planning.
D. Collision-Free Path Generation
EgoNav converts the refined waypoint into a collision-free velocity command using depth-derived geometry and a trajectory-library planner. Planner parameters adapt to waypoint clearance and correction magnitude, tightening control in confined or uncertain conditions.
- The local planner translates the corrected waypoint into a collision-free velocity command using a depth-derived point cloud.
- Trajectory Library Evaluation: Falco evaluates kinodynamically feasible trajectory groups and executes the group whose collision-free endpoints best approach and orient toward the refined waypoint.
- Adaptive Planner Modulation: Waypoint clearance and correction magnitude are normalized into modulation factors that vanish in open, well-predicted conditions.
- Adaptive Planner Modulation: Small clearance reduces planning scope and raises collision tolerance, while large correction shortens the planning horizon for finer planning.
- The resulting calibration reflects both perceived geometry and the extent of upstream waypoint intervention before the selected trajectory commands execution.
IV. EXPERIMENTS
The experiments evaluate EgoNav in simulation and the real world against contemporary baselines, with ablations examining the contribution of each component.
- EgoNav is evaluated in simulation and the real world against contemporary baselines, with ablation studies analyzing each component’s contribution.
A. Experimental Setup
The evaluation spans simulated and physical platforms, measures both safe goal-reaching and path efficiency, and compares EgoNav with image-goal and exploration-based baselines under varied indoor conditions.
- Platforms: Simulation uses Habitat-sim with Matterport3D and a LoCoBot, while real-world experiments use a humanoid robot with RGB-D sensing.
- Metrics: Success Rate measures collision-free arrival within 1 m, while SPL captures the shortest-to-actual path-length ratio over successful episodes.
- Datasets: Simulation covers 12 varied MP3D scenes with short, medium, and long geodesic-distance levels and 30 episodes per method per level.
- Baselines: Baselines include prior-map image-goal methods and VLFM, which performs frontier-based exploration without a prior map.
- Implementation Details: Learning-based methods use publicly released pre-trained weights without evaluation-environment fine-tuning.
- Implementation Details: Refinement and planning parameters remain fixed across simulation and real-world experiments, with empirical hyperparameters selected on five held-out simulation scenes.
B. Main Results
Across simulation and physical-robot evaluation, EgoNav outperforms contemporary baselines in success and efficiency, while refinement addresses directional errors and adaptive planning improves constrained-space execution.
- Simulation: Longer simulated routes expose collision failures and widen the gap between success rate and SPL as detours accumulate.
- Simulation: VLFM maintains substantially lower SPL than SR across distances, reflecting path inefficiency from frontier exploration without a prior map.
- Simulation: EgoNav outperforms PlaceNav+Falco at every simulated distance level, extending the gains of depth-based local planning alone.
- Real-World: The real-world ranking matches simulation, and EgoNav’s advantage over PlaceNav+Falco persists across all evaluated distances.
- Real-World: Lighting differences degrade VPR accuracy in the physical setting, with the effect becoming steeper on long paths.
- Real-World: Open-loop backtracking mitigates severe localization failures only for short detours of a few meters because accumulated error limits replay effectiveness.
- Qualitative Analysis: Waypoint refinement centers targets at door openings, corrects sharp-turn direction errors, and keeps EgoNav collision-free through a narrow doorway.
C. Ablation Studies
Ablations show that EgoNav’s modules progressively improve navigation, with refinement and adaptive planning addressing distinct geometric and efficiency failures. Qualitative paths and criterion removal further highlight directional coherence as especially important.
- Protocol: 30 runs on 10–20 m episodes evaluate EgoNav’s progressive component additions in simulation.The ablation protocol uses medium-distance episodes.
- Progressive ablation: EgoNav’s success improves consistently as modules are added, while SPL captures the trade-off between recovery and path efficiency.Relocalization can affect this trade-off.
- Progressive ablation: Adaptive planner modulation shortens and smooths trajectories through narrow passages without changing success rate.It relaxes collision sensitivity and refines search granularity near low-clearance regions.
- Criterion ablation: Removing directional coherence causes the largest success-rate drop by producing overshoot at sharp turns and irrecoverable localization failures.The criterion keeps waypoint selection aligned with the global path.
- Criterion ablation: Removing the prior degrades success and path quality, whereas removing safety leaves success unchanged but lowers SPL near obstacle boundaries.The prior avoids unnatural wall-hugging candidates, while safety reduces downstream planner search burden.
- Platform evaluation: The ablation section also reports average latency measurements for core components on the humanoid platform.The supplied table identifies the latency evaluation scope but provides no values.
D. Robustness to Dynamic Obstacles
EgoNav retains reactive dynamic-obstacle avoidance through Falco, but fast or unpredictable obstacles expose limits that can reduce success and require future predictive control.
- Evaluation setup: EgoNav evaluates varying numbers of random dynamic obstacles in simulation and real-world episodes.Simulation reuses 30 medium-difficulty episodes; real-world testing uses 10 episodes across 5 scenes.
- Results: Falco preserves reactive motion handling, so EgoNav inherits collision avoidance while retaining its static-scene margin.The refinement module changes waypoints and planner parameters without modifying Falco’s avoidance.
- Limitations: Fast obstacles on random trajectories remain difficult to evade, and detours can leave the planned path and trigger relocalization, lowering success.The stated limitation concerns reactive avoidance under higher-difficulty motion.
- Limitations: Velocity-level reasoning and trajectory prediction are identified as future requirements for handling such dynamic-obstacle cases.These capabilities are outside the presented avoidance design.
V. DISCUSSION
EgoNav is presented as a systems contribution that couples learned waypoint prediction with geometry-aware local planning. Experiments report gains over contemporary baselines, while discussion highlights map-coverage and generalization limits.
- Discussion: Experiments in simulation and on a physical humanoid robot show consistent gains in success rate and path efficiency over contemporary baselines.The discussion characterizes EgoNav’s contribution at the systems level.
- Robustness: Dynamic-obstacle evaluation includes static and dynamic simulation rows plus real-world results across 10 episodes and 5 scenes.Table VII covers the 10–20 m setting and identifies the simulation and real-world sample scopes.
- Limitations: The waypoint predictor’s generalization is bounded by its training distribution.This is stated as a remaining limitation of the system.
- Limitations: Avoidance detours can move the robot beyond the prior map’s visual coverage, where viewpoint changes increase matching failures.Panoramic map collection is proposed as a promising direction.