PaperlayerView exact revision on arXiv

Source-linked AI summary

One Scan is Enough: Demonstration-Free Adaptation for Language-Guided Navigation

Zhuoyuan Yu, Yuxing Long, Du Wang, Junyang Wang, Hongwei Fan, Hao Dong

arXiv:2610.33794v1cs.ROcs.AIcs.CLcs.CVcs.LG

TL;DR

Language-guided navigation policies degrade in new buildings, while collecting demonstrations in each target space is slow, expensive, and unscalable. The paper compiles a passive scan into grounded episodes and adapts the policy with scene-derived Pareto utility; RealNav reaches 85% success on scanned regions versus 49% for a human-demonstration-post-trained baseline, and retains gains in unscanned regions.

  • Problem

    Language-guided navigation policies perform reasonably on benchmark trajectories but degrade in new buildings, while collecting target-space demonstrations is slow, expensive, and unscalable.

    Source: §1 · p. 1

  • Method

    Scan2Instruction compiles a passive scan into feasible, grounded language-navigation episodes, and PRPT-KS adapts RealNav using scene-derived Pareto utility without demonstrations, a learned critic, or human labels.

    Source: §1 · pp. 1–2

  • Results

    85% real-robot success on scanned regions versus 49% for the same model post-trained on human demonstrations; unscanned regions yield 82% versus 41%.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Across five real scenes, a single passive scan supports demonstration-free scene adaptation that performs strongly on scanned and unscanned regions.

    Source: §1 · p. 2

    Scanning still requires a human operator, and substantial layout changes make the geometry stale and require a rescan.

    Source: §5 · p. 9

Abstract

from arXiv · show

Deploying a language-guided navigation policy in a new building usually means watching it fail. Public-benchmark VLAs degrade scene by scene as layout, appearance, and language drift, and the standard fix of teleoperating fresh trajectories on site does not scale. We replace human demonstration with a single brief iPhone scan. Scan2Instruction compiles that scan into a complete training environment, automatically synthesizing executable navigation episodes with grounded instructions. PRPT-KS then adapts the policy through a closed-form Pareto utility read directly off the reconstructed scene, with no learned critic and no human labels. Across five real environments, the two designs together drive our RealNav to an 85% real-robot success rate, compared with 49% for a RealNav baseline post-trained on human demonstrations, and still achieve 82% in unscanned regions of the same buildings, compared with 41% for the same baseline. Our work turns demonstration-free scene adaptation for language-guided navigation from an aspiration into a working reality.

1 Introduction

Language-guided navigation policies lose performance outside benchmarks, while collecting demonstrations in each target building is slow, expensive, and unscalable. This paper uses one passive scan to generate grounded navigation episodes and scene-derived post-training signals, with strong real-robot results.

1 Introduction

  • The paper asks whether a robot can adapt to a new scene from one passive scan without demonstrations, addressing costly and unscalable per-building data collection.Policies perform reasonably on benchmark trajectories but degrade in new buildings as scenes and user requests change.Source: §1 · p. 1
  • Scan2Instruction turns a passive scan into feasible episodes with collision-aware trajectories and visually grounded, spatially aligned instructions.It covers Route following, Object Goal, and Human Demand tasks; generation and adaptation require one-time manual scene calibration but no demonstrations.Source: §1 · pp. 1–2
  • PRPT-KS reads a three-axis Pareto utility from the known scene and uses it for policy gradients, planner admissibility, and preference re-weighting without a learned critic or human labels.The axes represent path progress, obstacle safety, and correct stopping.Source: §1 · p. 2
  • 85% real-robot success on scanned regions versus 49% for the same model post-trained on human demonstrations; unscanned regions yield 82% versus 41%.Evaluations cover five real scenes.Source: §1 · p. 2
  • The two components replace the per-scene teleoperation-and-annotation bottleneck with scan-derived training episodes and post-training signals.The stated contributions combine automatic episode compilation with scene-grounded adaptation.Source: §1 · p. 2

2 Related Work

Prior work uses reconstructed scenes for navigation simulation and training, while reinforcement post-training commonly depends on interaction, expert paths, or real-robot trials. This paper compiles language-action episodes and derives post-training signals directly from scanned geometry.

2.1 3D Gaussian Splatting for Embodied Navigation

  • Prior 3D Gaussian Splatting navigation work uses reconstructed scenes for simulation and policy training [Kerbl et al., 2023, Chhablani et al., 2025, Liu et al., 2026, Zhu et al., 2025, Xie et al., 2025]; this work adds grounded language-action episode compilation and geometry-derived post-training signals.Scan2Instruction plans collision-aware trajectories, renders first-person observations, and generates instructions grounded in those trajectories.Source: §2.1 · p. 3

2.2 Reinforcement Post-Training for Navigation Policies

  • Prior navigation RL uses simulator interaction, map priors, expert paths, or real-robot trial-and-error; PRPT-KS instead queries scanned geometry for progress, safety, and stopping signals.Those signals become a scalar advantage, planner admissibility test, and preference-pair dominance weight.Source: §2.2 · p. 3

3 Method

The method formalizes adaptation from a single target-scene scan without human trajectories, then combines RealNav, Scan2Instruction, and geometry-grounded PRPT-KS for known-scene training.

3.1 Problem formulation

  • Adaptation maps a base policy and target-scene scan to parameters that maximize expected instruction success, with no human-provided target-scene trajectory.The base policy is pretrained without access to the target scene.Source: §3.1 · p. 3

3.2 RealNav: A Continuous-Trajectory Navigation VLA

  • RealNav uses Qwen3-VL navigation-token features and an MLP head to predict eight future local waypoints, with stopping inferred from consecutive waypoint changes.Stopping is correct only when the robot is within the target radius.Source: §3.2 · pp. 3–4

3.3 Scan2Instruction: Compiling a scan into language-navigation episodes

  • The scan supplies a Gaussian-splat appearance model and reachable-free-space geometry; clearance-aware planning generates paths within a safety-eroded map.Goals are moved near objects, and planned routes use clearance-penalized costs and safety-constrained simplification.Source: §3.3 · pp. 4–5
  • A pretrained MLLM generates route-following, object-goal, and human-demand instructions from guided route images and goal views.Each episode pairs an instruction with policy observations and planner supervision to form Scan2Instruction's training dataset.Source: §3.3 · p. 5

3.4 PRPT-KS: Pareto-utility post-training for known-scene navigation

  • PRPT-KS reuses a known-scene Pareto vector for policy-gradient updates, rollout admissibility, and preference reweighting, avoiding learned critics and reward models.Its three components measure path progress, obstacle safety, and correct stopping.Source: §3.4 · p. 5
  • PRPT-KS rejects actions that violate per-axis admissibility, substitutes planner actions, and retains rejected actions as preference examples.The test keeps checked waypoints inside the eroded planning map and prevents progress gains from compensating for safety violations.Source: §3.4 · p. 6
  • A step-wise planner-expert buffer, trajectory-length curriculum, and non-learned covariance schedule stabilize post-training alongside the three-term objective.The objective combines policy-gradient, planner-action anchoring, and Pareto-weighted preference losses.Source: §3.4 · p. 7

4 Experiments

Across five real scenes, PRPT-KS improves RealNav’s success rate on the robot, and the adapted policy retains strong performance in unscanned regions. Ablations support the contribution of all three loss terms, though terminal navigation error does not improve uniformly.

4.1 Setup

  • Evaluation covers an office, cottage, teaching building, and two apartments, with success rate measured in simulation and on the robot using a 1.5 m radius.SPL, NE, and OS are reported only in the ablation because they require groundtruth path length and pose unavailable reliably through the real-robot interface.Source: §4.1 · pp. 7–8, Table 2 · p. 8

4.2 Simulation and real-world evaluation

  • PRPT-KS sharply increases success rate across five scenes, and the post-trained policy preserves strong performance on the physical robot despite rendering-camera gaps and execution drift.The human-demo-sft variants improve over their untuned counterparts but remain far below RealNav + PRPT-KS.Source: §4.2 · p. 8, Table 2 · p. 8

4.3 Generalization to unscanned regions of the same scenes

  • RealNav + PRPT-KS retains strong real-robot success in regions never scanned, reconstructed, included in synthesized tasks, or visited during RL rollouts.The evaluation covers 20 trials per scene, 100 total; Table 3 reports real-robot SR in this setting.Source: §4.3 · p. 8, Table 3 · p. 9

4.4 Ablations

  • Removing the behavior-cloning anchor reduces SR to 0.30 and SPL from 0.834 to 0.293, the largest ablation degradation; the other loss removals also reduce SR, SPL, and OS.The variants without policy-gradient or preference supervision achieve lower NE (1.31 m and 1.32 m) than the full model (1.42 m), so gains do not extend uniformly to terminal navigation error.Source: §4.4 · pp. 8–9, Table 4 · p. 9

5 Limitations and Broader Impacts

Scanning still requires a human operator, and substantial layout changes make the reconstructed geometry stale and require a rescan. The paper also identifies privacy and dynamic-space safety concerns, alongside anonymization, owner consent, and local execution practices.

5 Limitations and Broader Impacts

  • A human operator is still needed to scan, and substantial layout changes make the geometry stale, requiring a rescan.Active incremental mapping is proposed as future work to detect divergence and refresh the representation autonomously.Source: §5 · p. 9
  • Private-space scans may capture identifiable people or documents; released assets are anonymized, scanner consent is required, and the pipeline runs locally without cloud upload.Stale geometry is a safety consideration in highly dynamic spaces, where rescans are recommended.Source: §5 · p. 9

Related papers

Loading 2610.33794v1…