PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards Path-Creative Navigation: Robot Navigation through Embodied Interaction

Haoyu Xi, Siwei Cheng, Xiangyuan Liu, Brenda Li, Wei Zhang

arXiv:2610.11072v1cs.RO

TL;DR

Conventional navigation generally assumes a fixed environment and searches existing free space, although a blockage may require embodied interaction to reach the goal. The paper formulates Path-Creative Navigation and instantiates it with a mapless framework that decides when to interact. Simulation and real-world experiments report higher task completion than baseline methods.

  • Problem

    Conventional navigation assumes a fixed environment and searches existing free space, although routes blocked by doors, movable objects, or pedestrians may require environmental interaction to reach the goal.

    Source: §I · pp. 1–2

  • Method

    The mapless framework fuses vision, LiDAR, and odometry with a traversability-aware method that assesses whether blockages are actionable and safely bypassable.

    Source: §I · p. 2

  • Results

    PCN achieved 80% success in manipulation and 70% in social-interaction tasks; Proximity-Trigger achieved 20% and 10%, respectively.

    Source: B. Real-World Experiments Setup and… · p. 8

  • Takeaways & Limitations

    Simulation and real-world experiments show higher task completion than baseline methods when navigation and interaction are coordinated.

    Source: §VI · p. 8

Abstract

from arXiv · show

Autonomous navigation in cluttered and constrained environments typically assumes a fixed environment and searches only for paths within existing free space. However, a route can be blocked by an articulated structure, a movable object, or a pedestrian, and reaching the goal may therefore require appropriate embodied interaction with the environment. This paper formulates Path-Creative Navigation (PCN), a navigation paradigm in which the robot recovers free space by coordinating locomotion and embodied interaction, and addresses one class of PCN tasks with a mapless framework. The vision-LiDAR-odometry framework integrates local observations, obstacle geometry, and robot-state estimates through a unified navigation system, providing stable semantic and geometric information for navigation and interaction. The traversability-aware decision method utilizes visual and LiDAR distance information to determine whether a blockage is actionable and whether it can be safely bypassed, enabling the robot to interact only when necessary while supporting articulated structure pushing, movable object pushing, obstacle avoidance, and pedestrian requests. We design simulation scenarios and evaluate different methods in both simulation and two real-world tasks that cannot be completed by conventional navigation alone without embodied interaction. These results demonstrate that our framework achieves higher task completion than the baseline methods by coordinating navigation and interaction in cluttered and constrained environments. Code is available at:{https://anonymous.4open.science/r/path-creative-navigation/}.

I. INTRODUCTION

In cluttered, constrained settings, reaching a goal may require changing the environment, not just finding a path through existing free space. The paper formulates Path-Creative Navigation and presents a mapless framework for deciding when and how to interact while navigating.

I. INTRODUCTION

  • A route blocked by a door, movable object, or pedestrian may require changing the environment rather than searching for another collision-free path.The challenge includes deciding when interaction is necessary, not only how to interact.Source: §I · pp. 1–2
  • Path-Creative Navigation (PCN) treats restoring traversability as part of navigation, jointly deciding whether to use current free space or change the environment.This paper addresses blockages from movable objects, articulated structures, and pedestrians.Source: §I · p. 2
  • The traversability-aware method uses local observations to assess whether a blockage is actionable and safely bypassable, triggering interaction only when necessary.It coordinates semantic reasoning about objects with geometric reasoning about blockage and bypass feasibility.Source: §I · p. 2
  • Vision-LiDAR-odometry fusion combines object semantics, local obstacle geometry, and robot-state estimates to support door and object pushing, pedestrian requests, and navigation.The framework also supports obstacle avoidance.Source: §I · p. 2
  • Simulation and real-world experiments report higher task completion and more reliable interaction selection while coordinating navigation and interaction.The real-world tasks include cases that conventional navigation alone cannot complete without embodied interaction.Source: §I · p. 2

II. RELATED WORK

Prior work supplies object-moving, interaction, planning, and loco-manipulation capabilities, but this framework focuses on local traversability decisions rather than global placement or long-horizon action planning. Its formulation couples locomotion and interaction in a changeable environment, subject to feasibility and goal-reaching constraints.

A. Robot Mobile Manipulation

  • NAMO and interactive-navigation methods address object movement or predefined tasks, while this framework uses local corridor observations and does not optimize final object placements or perform global planning.Learning-based interaction methods often target specific settings; the framework applies to new visual instances in supported categories without fine-tuning or retraining.Source: A. Robot Mobile Manipulation · p. 2

B. Robot Loco-manipulation

  • Loco-manipulation methods provide physical skills, while this framework coordinates predefined responses to blocker types; door opening is selected only when the door blocks goal reaching and opening creates sufficient free space.Unlike door-opening work that treats opening and traversing as a coupled task, opening the door is not this paper's task goal.Source: B. Robot Loco-manipulation · p. 3

III. PROBLEM FORMULATION

  • The formulation treats fixed obstacles, movable objects, and articulated structures as distinct environment elements, and pedestrians as a special class of movable objects.The robot selects locomotion actions or embodied interactions; interactions may change an object's position or configuration, or request a pedestrian to clear the passage.Source: §III · p. 3
  • Path-Creative Navigation uses current observations to judge blockage and bypassability without a pre-built map or global planning, and seeks a feasible policy that reaches the goal region.Robot and environment states evolve together; interaction may change available free space, subject to robot dynamics, collision avoidance, and interaction constraints.Source: §III · p. 3

A. Overview of Our Framework

The framework combines semantic targets and local geometry to choose between normal navigation, obstacle avoidance, and interaction. Its vision-LiDAR-odometry fusion supplies the observations and robot-state information used for closed-loop interaction.

A. Overview of Our Framework

  • The framework has two modules: vision-LiDAR-odometry fusion supplies semantic targets and local geometry, while traversability-aware decisions choose navigation, avoidance, or interaction.The framework reassesses traversability after the scene changes and resumes navigation toward the goal.Source: A. Overview of Our Framework · p. 3, Figure 2 · p. 4

B. Vision-LiDAR-Odometry Fusion

  • Vision and depth provide semantic object targets in the odometry frame, while LiDAR and odometry provide local geometric information for navigation and interaction.The observation also includes RGB and aligned depth images, a LiDAR point cloud, robot state, and the final goal; YOLO-World processes image text prompts.Source: B. Vision-LiDAR-Odometry Fusion · p. 3, 1) Semantic Target Detection · p. 4
  • The decision jointly evaluates object actionability, corridor geometry, and bypass feasibility to determine whether interaction is needed; fixed or bypassable obstacles remain navigation cases.After an interaction changes the scene, traversability is reassessed before navigation resumes.Source: Figure 2 · p. 4

1) Semantic Target Detection:

Semantic detections distinguish articulated structures, movable objects, and people, while object size supports geometric reasoning rather than classification. Depth-assisted estimation derives object distance and stable geometry for later traversability and interaction reasoning.

1) Semantic Target Detection:

  • Articulated structures and movable objects are mapped to separate semantic sets, while person labels are handled separately by the pedestrian interaction strategy.Object size is used for geometric reasoning rather than semantic classification.Source: 1) Semantic Target Detection · p. 4

2) Depth-Assisted Size Estimation:

  • The detector’s RGB bounding box maps to aligned depth, where a central region limits background and edge contamination and valid samples estimate object depth.A fraction of the nearest valid samples is retained and their median is used, providing a more robust estimate than a single center pixel.Source: 2) Depth-Assisted Size Estimation · p. 4
  • The selected depth-region center is back-projected into a 3-D camera-frame point and transformed into the odometry frame using camera-to-base and base-to-odometry transforms.The camera focal lengths and principal-point coordinates are used in back-projection.Source: 2) Depth-Assisted Size Estimation · p. 4
  • Estimated object width supports geometric reasoning and task-corridor construction.Source: 2) Depth-Assisted Size Estimation · p. 4
  • A target is accepted after consecutive spatially consistent observations, averaged to update its position, then locked upon entering the close-range interaction region.The selected candidate becomes the interaction target.Source: 2) Depth-Assisted Size Estimation · pp. 4–5

3) Local Geometric Information:

The local geometric observation combines LiDAR-derived distances and clearances with the main frontal structure’s lateral position, alongside object targets in the odometry frame.

3) Local Geometric Information:

  • LiDAR points are transformed into the robot frame using odometry, filtered by height, distance, and range, then divided into frontal and lateral regions.Source: 3) Local Geometric Information · p. 5
  • The local geometry summarizes front and front-left/front-right distances, left/right scene clearances, and the main frontal structure’s lateral position.Source: 3) Local Geometric Information · p. 5
  • The fusion framework outputs object targets in the odometry frame together with local geometry for subsequent traversability-aware decision making.Source: 3) Local Geometric Information · p. 5

C. Traversability-Aware Decision Method

The decision method distinguishes navigation from embodied interaction using object semantics, task-corridor relation, and LiDAR clearance. It treats movable or articulated objects as interaction candidates only when they block the route and cannot be safely bypassed.

C. Traversability-Aware Decision Method

  • The decision jointly considers an object’s semantic category, relation to the task corridor, and available LiDAR clearance to choose navigation or embodied interaction.Source: C. Traversability-Aware Decision Method · p. 5
  • Fixed obstacles are handled through navigation and avoidance, while movable objects and articulated structures are considered for interaction only when they block the active route and cannot be safely bypassed.Source: C. Traversability-Aware Decision Method · p. 5

1) Task Corridor and Bypass Evaluation:

  • An object overlaps the task corridor when it is in front of the robot and its lateral extent intersects the required passage, accounting for robot half-width, object width, and a safety margin.Source: 1) Task Corridor and Bypass Evaluation · p. 5
  • Object-specific side clearances are estimated from supported LiDAR clusters; an object is non-bypassable when both clearances are at or below the threshold.The condition must persist across several scans with supported observations on both sides; sufficient clearance on either side or unreliable LiDAR evidence leaves navigation active.Source: 1) Task Corridor and Bypass Evaluation · p. 5

2) Interaction Necessity:

Interaction is triggered only for an actionable object that overlaps the task corridor and cannot be safely bypassed; the selected candidate then receives an admissible interaction mode.

2) Interaction Necessity:

D. Embodied Interaction Execution

The section describes how the robot selects and executes embodied interactions, then evaluates the framework in simulation. Results report stronger performance than non-oracle baselines and stage-level trial pass rates.

D. Embodied Interaction Execution

  • After NeedInteract selects a target and mode, fixed obstacles remain for avoidance; doors and movable objects receive physical interaction, while pedestrians receive a stop-and-request protocol.Physical interaction uses approach, body alignment, arm positioning, and closed-loop feedback; navigation resumes when the passage is traversable.Source: D. Embodied Interaction Execution · p. 5

A. Simulation Experiments Setup and Results Analysis

  • Across 70 varied Isaac Sim scenarios, each method ran twice; comparisons included navigation-only, always- and proximity-triggered, oracle, and no-LiDAR/no-decision variants.The evaluation metrics include path efficiency, interaction selection and initiation measures, retries, and mean time on successful trials.Source: A. Simulation Experiments Setup and… · pp. 5–6
  • Among non-oracle methods, ours leads in success rate, SPL, interaction recall, and F1; it reduces Proximity-Trigger MIR from 7.86% to 6.43%.Always-Interact has a UIR of 57.41% and a success rate of 45.00%; the Oracle-Selective reference reaches 100% success using simulator ground truth.Source: A. Simulation Experiments Setup and… · p. 6, Table I · p. 7
  • Trial-level pass rates are 94.3% for navigation, 73.6% for articulated-structure pushing, and 97.9% for movable-object pushing.A stage counts as successful only if the trial reaches that stage and completes all required steps.Source: A. Simulation Experiments Setup and… · p. 6, Figure 5 · p. 7

B. Real-World Experiments Setup and Results Analysis

Real-world tests examine sequential navigation and manipulation, and non-contact interaction with a pedestrian, in cluttered, narrow corridors. PCN completes both tasks more often than the reported baselines while reducing intervention rates relative to Proximity-Trigger.

B. Real-World Experiments Setup and Results Analysis

  • The 20-trial evaluation uses cluttered narrow corridors with previously unseen obstacles, including barriers, suitcases, and chairs, sometimes placed close to walls.The platform uses a Unitree G1, Livox Mid-360 LiDAR, Intel RealSense D435i depth camera, and Jetson Xavier NX; detection runs at 15 Hz and planning at 10 Hz.Source: B. Real-World Experiments Setup and… · p. 7
  • The manipulation task requires opening a blocking door, navigating between areas, avoiding static obstacles, and completing subsequent interactions; the social task requests that a blocking pedestrian clear the passage.For the social task, the robot stops at a safe distance and resumes navigation only after the passage becomes available.Source: B. Real-World Experiments Setup and… · p. 8
  • PCN succeeds in 80% of manipulation and 70% of social-interaction trials, versus 20% and 10% for Proximity-Trigger; DRL-DCLP and Always-Interact fail both tasks.PCN intervention rates are 20% and 30%, respectively; the passage attributes Proximity-Trigger's poor results to unnecessary interactions that can reduce free space, cause the robot to get stuck, or cause collisions.Source: B. Real-World Experiments Setup and… · p. 8

VI. CONCLUSIONS

The conclusion frames Path-Creative Navigation as restoring traversability through embodied interaction. It reports a mapless framework and higher task completion than baseline methods in simulation and real-world experiments.

VI. CONCLUSIONS

  • Path-Creative Navigation treats restoring traversability through embodied interaction as part of navigation.Source: §VI · p. 8
  • The framework combines mapless vision, LiDAR, and odometry with a decision method for judging whether blockages are actionable and bypassable.It coordinates navigation, obstacle avoidance, articulated-structure interaction, object pushing, and pedestrian requests while maintaining progress toward the goal.Source: §VI · p. 8
  • Simulation and real-world experiments show higher task completion than baseline methods; future work will integrate whole-body control and extend PCN to more challenging tasks.Source: §VI · p. 8
Loading 2610.11072v1…