PaperlayerView exact revision on arXiv

Source-linked AI summary

CanonNav: Disentangling Navigation Behavior from Camera Geometry in Cross-Platform Visual Navigation

Dong-Wook Kim, Ji-Hoon Hwang, E-In Son, Mintaek Oh, Seung-Woo Seo

arXiv:2608.30242v1cs.RO

TL;DR

Cross-platform visual-navigation demonstrations entangle navigation behavior with camera geometry and omit intermediate planning decisions. CanonNav canonicalizes observations and trajectories, adds safety and local-progress supervision from offline traversability estimates, and consistently outperforms RGB baselines while surpassing RGB-D methods in challenging scenarios despite RGB-only inference.

  • Problem

    Image–trajectory correspondences vary with camera geometry, while imitation trajectories leave intermediate decisions about safe local progress implicit.

  • Method

    CanonNav transforms observations and trajectories into a camera-consistent representation and derives safety and local-progress supervision from offline traversability estimates.

  • Results

    CanonNav consistently outperforms RGB-based baselines and surpasses RGB-D-based methods in challenging scenarios despite using only RGB at inference.

  • Takeaways & Limitations

    Canonicalized behavior and complementary planning supervision support collision-aware and goal-directed navigation from cross-platform demonstrations.

  • Takeaways & Limitations

    Embodiment-specific planning parameters are fixed during training, so each new deployment platform requires retraining.

Abstract

from arXiv · show

While visual navigation has advanced through imitation learning from cross-platform demonstrations, fully leveraging such data remains challenging. First, directly learning from image-trajectory pairs entangles navigation behavior with platform-dependent camera geometry. This hinders consistent learning by forcing the policy to implicitly infer camera geometry from visual observations, an inherently ill-posed problem. Second, imitation learning from demonstrated trajectories captures the expert's chosen motion but leaves the intermediate decisions underlying that motion implicit. To address these issues, we propose CanonNav, a visual navigation framework that disentangles navigation behavior from camera geometry and incorporates complementary planning supervision into learning from cross-platform demonstrations. CanonNav introduces camera geometry canonicalization, which transforms visual observations and trajectories into a camera-consistent representation space. Building on this representation, we derive safety and local-progress supervision using pseudo-labels from an offline traversability estimator. Safety supervision penalizes unsafe trajectories, while local-progress supervision guides where the robot should advance. Experiments across diverse camera configurations and environments show that, despite using only RGB at inference, CanonNav consistently outperforms RGB-based baselines and even surpasses RGB-D-based methods in challenging scenarios.

1 Introduction

CanonNav addresses two barriers to learning from cross-platform visual-navigation demonstrations: camera-dependent image–trajectory correspondences and missing supervision for intermediate planning decisions. It canonicalizes camera geometry and adds safety and local-progress supervision to support consistent navigation learning.

  • Motivation: A canonical representation space is needed to align visual observations and trajectories across platforms.The proposed direction separates navigation behavior from platform-dependent camera geometry.
  • Motivation: Cross-platform demonstrations associate the same robot-centric trajectory with different image regions because camera geometries vary.This forces policies to infer camera geometry while learning navigation behavior, an inherently ill-posed coupling.
  • Motivation: Imitation trajectories expose expert motion but leave local advancement and safe-path decisions implicit when the goal is not directly reachable.These intermediate decisions include identifying a local-progress region and generating a safe path toward it.
  • CanonNav: CanonNav transforms observations and trajectories into a camera-consistent space and derives safety and local-progress supervision from offline traversability estimation.The framework is designed to support consistent learning from heterogeneous cross-platform demonstrations.
  • CanonNav: Safety supervision penalizes unsafe trajectories, while local-progress supervision guides prediction of the region where the robot should advance.Together, these signals provide complementary planning supervision beyond demonstrated motion.

2 Related Work

Related work scales imitation learning across heterogeneous navigation data, adds planning signals for safe and goal-directed behavior, and addresses platform or camera-view variation through specialized policies and augmentation.

  • Cross-platform imitation learning: GNM and ViNT scale imitation-learning navigation to heterogeneous data collected across diverse environments and platforms.Recent generative-planner methods sample multiple trajectories to extend this paradigm.
  • Planning supervision: Prior planning methods add differentiable costmap or semantic traversability objectives to support safer, more goal-directed navigation.iPlanner learns a local planner with differentiable costmap objectives, while ViPlanner uses semantic traversability costs.
  • Platform and camera variation: Platform variation has been addressed through embodiment-specific policies, policy distillation, viewpoint augmentation, and camera randomization.X-Nav uses embodiment-specific policies and distills them into a unified policy, while ExAug and RoVi-Aug generate varied viewpoints.

3 Method

CanonNav canonicalizes camera-dependent observations and trajectories into a consistent representation, then augments diffusion-based imitation with safety and local-progress supervision. Its planning signals use offline traversability-derived pseudo-labels to shape collision-aware trajectories and intermediate advance regions.

  • Overall Framework: A diffusion policy predicts multiple height-normalized trajectory candidates from fused image-goal features, while auxiliary branches estimate Scope-of-Reach and waypoint traversability.Image and goal features are combined through cross-attention before trajectory and planning predictions are produced.
  • Camera Geometry Canonicalization: Camera geometry canonicalization maps images, goals, and trajectories into a camera-consistent space for cross-platform policy learning.The method assumes a front-centered camera, canonicalizes the view, and height-normalizes trajectories and goals to reduce dependence on camera intrinsics, pitch, and height.
  • Safety Supervision: Offline traversability estimation produces pseudo-BEV collision maps and signed-distance fields used to supervise trajectory safety without requiring traversability inference at deployment.Trajectory-level penalties discourage insufficient clearance, while waypoint-level traversability scores support candidate selection.
  • Local-Progress Supervision: Scope-of-Reach supervision represents a locally appropriate advance region for cluttered scenes where the goal cannot be reached directly.The target is selected as the last expert waypoint before collision or leaving the field of view, or the final waypoint when the trajectory remains visible and collision-free.
  • Local-Progress Supervision: The local-progress objective fits the predicted region to a height-normalized target, penalizes overlap with collision cells, and encourages generated trajectories to pass near it.The objective combines target fitting, collision avoidance, and consistency between the predicted region and generated trajectory.

4 Experiments

CanonNav is evaluated across diverse simulated and real-world camera configurations, comparing RGB and RGB-D baselines on navigation robustness, planning, component contributions, and deployment performance. Across these evaluations, canonicalization and explicit safety/local-progress supervision support robust, collision-aware navigation.

  • Experimental Setup: Experiments vary horizontal FoV, camera pitch, camera height, and subgoal distance across two simulation environments and 18 camera configurations.The evaluation produces 56,160 and 47,520 rollouts for the two environments.
  • Simulation Results: CanonNav achieves the highest success rate at the longest subgoal interval in both environments, reaching 86.4% at 20 m in CitySim and 85.5% at 12 m in AWS Hospital.It remains competitive with depth-based NavDP at shorter intervals while using only RGB.
  • Camera Robustness: CanonNav varies by only 1.17 percentage points under pitch change in CitySim’s 20 m setting, while its largest pitch-induced change is 3.87 points in AWS Hospital.It benefits from wider horizontal FoVs while maintaining stable performance under pitch variation.
  • Planning Supervision: Safety and local-progress supervision enable CanonNav to plan earlier detours toward predicted Scope-of-Reach regions rather than reacting only near obstacles.At 20 m in CitySim, collisions decrease from 2.21 to 0.89 per 100 m compared with NavDP.
  • Ablation Study: Ablations show that canonicalization improves imitation learning, either supervision objective substantially improves performance, and combining both gives the best results.Applying both objectives without canonicalization reduces SR to 77.1% and 75.2% and increases standard deviations from 3.8 to 7.7 and from 3.9 to 7.6.
  • Real-World Experiments: In real-world experiments, CanonNav achieves higher success rates and requires fewer interventions than NavDP across all three environments.The system runs onboard at 4.5 Hz and uses different camera FoVs and pitches across deployments.

5 Conclusion

CanonNav combines camera geometry canonicalization with safety and local-progress supervision for robust visual navigation from cross-platform demonstrations. Despite RGB-only inference, it outperforms RGB baselines and can surpass RGB-D methods in challenging scenarios.

  • Camera geometry canonicalization creates a camera-consistent representation that separates navigation behavior from platform-dependent camera geometry.
  • Offline traversability estimates provide safety and local-progress supervision for collision-aware and goal-directed planning.
  • CanonNav consistently outperforms RGB-based baselines and even surpasses RGB-D-based methods in challenging scenarios despite RGB-only inference.
  • Canonicalization may reduce visual context when the original camera has a wider field of view than the predefined canonical view.
  • CanonNav does not model policy variations across robot embodiments.

S1 Additional Method Details

The supplementary method details describe how CanonNav estimates each candidate path’s reachable point, scores paths for progress, displacement, and temporal consistency, and constructs training supervision from traversability predictions and BEV collision maps.

  • Trajectory selection: Traversability scores estimate each candidate path’s expected reach point before path scoring and selection.The expected reach point is obtained by accumulating segment-level survival probabilities and interpolating by arc length.
  • Trajectory selection: Candidate paths are evaluated using goal advance, displacement, and temporal consistency, and the highest-scoring path is selected.The goal-advance score measures reduced goal distance, while displacement measures the expected reach point’s distance from the current position.
  • Pseudo-label generation: A frozen offline traversability estimator predicts per-pixel masks during preprocessing, so it is not required at inference time.
  • Pseudo-label generation: The predicted traversability mask is projected into a robot-centric BEV collision map using known camera geometry.Non-traversable projected pixels mark BEV cells as occupied, while near-field blind-region cells are excluded from the Scope-of-Reach target.
  • Pseudo-label generation: Figure S2 visualizes traversability masks, their BEV projections, and resulting Scope-of-Reach targets across diverse environments.
  • Pseudo-label generation: The Scope-of-Reach target is the last valid expert-trajectory pose before collision or another terminating event, evaluated over the robot footprint.Sampling all footprint-covered cells reduces unreliable collision labels caused by noise in the traversability mask.

S2 Network and Training Details

The supplementary network and training details describe a DINOv3-based visual encoder, a conditional diffusion policy, geometric data augmentation, and embodiment-specific planning parameters.

  • Network architecture: CanonNav uses a DINOv3 ViT-S+ backbone with LoRA and aggregates features from transformer layers 3, 6, 9, and 12.Goal features query cross-attention over image features to produce the conditioning representation.
  • Network architecture: The diffusion policy uses a Conditional 1D U-Net conditioned on fused visual and goal features.Training corrupts expert trajectories with sampled noise and trains the policy to predict that noise.
  • Training: Training applies yaw augmentation and horizontal flipping while transforming images and expert trajectories consistently.
  • Training: The model is trained for 70 epochs with AdamW, batch size 256, and an EMA model used for real-time navigation.The EMA decay is warmed up to 0.9999.
  • Embodiment-specific planning parameters: Robot radius and safety margin are embodiment-specific parameters that shape traversability and collision losses.These parameters are fixed during training, requiring retraining for each new deployment platform.

S3 Additional Qualitative Results

Additional qualitative results cover simulated and real-world environments, camera-setting variations, and representative failure cases. They show planning across diverse settings while identifying incorrect Scope-of-Reach predictions as the main failure source.

  • Simulation results: CitySim and AWS Hospital figures show reference paths and representative planning results across platform camera settings.The settings vary the camera’s horizontal field of view, mounting height, and pitch angle.
  • Real-world results: Real-world qualitative results reconstruct complete robot trajectories with start positions, subgoals, and temporally ordered planning segments.
  • Failure cases: Most CanonNav failures arise from incorrect Scope-of-Reach predictions that overlap non-traversable regions.The policy then generates trajectories toward those unsafe areas.
  • Failure cases: Failures are associated with obstacles that are poorly represented in training or difficult to distinguish from surrounding visual context.
Loading 2608.30242v1…