PaperlayerView exact revision on arXiv

Source-linked AI summary

Data-Centric Recursive Self-Improvement for Navigation via Policy-Synchronized Language-Action Dual Bootstrapping

Songze Li, Haoming Gu, Ziyang Gong, Zhitong Wang, Gengze Zhou, Zun Wang, Yuhan Zhu, Tianxiang Jiang, Juanzi Li, Qi Wu, Weinan Zhang, Yu Qiao, Wenwei Zhang, Yi Wang

arXiv:2610.15782v1cs.CV

TL;DR

VLN needs high-quality language supervision, but annotation is costly and existing synthesis often separates navigation from instruction generation or uses fixed annotators. RSI-Nav recursively generates and execution-filters instructions with a shared model, updating on small trajectory shards. It improves navigation over supervised initialization, reaching 68.6% SR on R2R-CE and 72.4% SR on RxR-CE, with modest spatial-understanding gains and stronger instruction utility.

  • Problem

    Human annotation is costly, while existing synthesis often separates navigation and instruction generation or uses fixed-model annotation that can leave supervision mismatched with the evolving policy.

    Source: §1 · p. 2

  • Method

    RSI-Nav uses a shared model to generate instructions for unlabeled trajectory shards, execute and filter them, then update immediately before processing the next shard.

    Source: §1 · p. 2

  • Results

    RSI-Nav improves navigation over supervised initialization, reaching 68.6% SR on R2R-CE and 72.4% SR on RxR-CE, with modest spatial-understanding gains and stronger downstream utility from generated instructions.

    Source: §4.1 · pp. 7–8, §5 · p. 10

  • Takeaways & Limitations

    Generated instructions provide useful downstream navigation supervision while the evolved model maintains strong spatial understanding with modest gains over supervised initialization.

    Source: §4.1 · p. 8, §5 · p. 10

    RSI-Nav relies on simulated environments and an offline trajectory pool, and execution-based filtering incurs considerable computational cost.

    Source: §4.2 · p. 9

Abstract

from arXiv · show

Vision-language navigation requires agents to follow natural-language instructions to reach target locations, making high-quality language supervision essential. Since human annotation is costly, prior work has explored scalable data synthesis methods. However, existing methods typically either use separate models for navigation and instruction generation or couple the two tasks through task-specific components; most also annotate the full trajectory corpus with a fixed model before updating, which can cause generated supervision to lag behind the evolving policy. Therefore, we introduce RSI-Nav, a data-centric recursive self-improvement framework for navigation via policy-synchronized language-action dual bootstrapping, which unifies vision-language-to-action instruction following and vision-action-to-language instruction generation within a shared model. Specifically, after supervised fine-tuning establishes fundamental capabilities, RSI-Nav progressively processes unlabeled trajectories in small shards. The current model generates instructions for these trajectories, executes them to evaluate goal completion and path efficiency, retains qualified samples, and immediately updates itself before processing the next shard. This shard-wise update keeps newly generated supervision aligned with the evolving policy. We also incorporate spatial understanding data, since stronger spatial reasoning benefits navigation. Extensive experiments show that RSI-Nav reaches 68.6% SR on R2R-CE and 72.4% SR on RxR-CE, while providing stronger instruction-generation utility and modestly improved spatial understanding after recursive evolution.

1 INTRODUCTION

Existing VLN data-synthesis pipelines often separate instruction generation from navigation or use fixed annotators, leaving a need for shared, policy-synchronized learning. RSI-Nav uses one evolving model to generate and execute instructions on trajectory shards, while incorporating spatial understanding and evaluating navigation, generation, and spatial skills.

1 INTRODUCTION

  • Most prior synthesis pipelines separate instruction generation and navigation or use task-specific dual-learning architectures, while fixed-model annotation can leave supervision mismatched with an improving policy.These limitations motivate jointly exploiting both directions of language-action grounding while synchronizing supervision with the current policy.Source: §1 · p. 2
  • Instruction generation maps visual-action experience to language, while navigation maps language and visual observations back to actions; execution can directly evaluate whether generated instructions guide efficient goal-reaching.The paper argues that feedback between the capabilities can filter language supervision and improve later navigation learning.Source: §1 · p. 2
  • RSI-Nav processes unlabeled trajectories in small shards: it generates instructions, executes them to assess goal-reaching efficiency, retains qualified pairs, and updates before the next shard.This keeps generated supervision aligned with the evolving policy rather than relying on a fixed annotator.Source: §1 · p. 2
  • RSI-Nav also incorporates spatial-understanding data and evaluates navigation, trajectory-to-instruction generation, and spatial understanding.The authors report improved navigation over supervised initialization, stronger downstream utility from generated instructions, and modest spatial-understanding gains.Source: §1 · p. 3
  • The framework unifies instruction following, instruction generation, and spatial understanding in one model, and the authors report leading success rates among compared VLN-CE methods while maintaining strong spatial understanding.The authors describe it as the first navigation framework combining unified language-action dual learning with policy-synchronized recursive self-improvement.Source: §1 · p. 3

2 RELATED WORK

Related work spans VLN architectures, scalable synthetic instruction data, complementary learning, and recursive self-improvement. RSI-Nav is positioned against approaches that use separate or task-specific models and coarse update rounds.

Vision-and-Language Navigation

  • VLN research has advanced through pretraining and history modeling, map construction and planning, and vision-language models that directly predict actions.High human-annotation costs motivate automatic augmentation and synthesis, including methods that build large instruction-trajectory datasets.Source: §2 · p. 3
  • VLN spans detailed-instruction navigation, coarse-grained remote object navigation, and continuous environments, with growing emphasis on spatial reasoning.Earlier data synthesis includes speaker models and backtranslation, followed by large-scale environments and synthetic instructions.Source: §2 · pp. 3–4

Complementary Learning and Mutual Bootstrapping

  • Complementary-learning methods use one capability to supervise or provide feedback to another, including dual translation and solver-verifier setups.In VLN, prior work has used separate speaker-follower models or task-specific architectures and consistency objectives.Source: §2 · p. 4

Recursive Self-Improvement

  • Prior recursive-self-improvement methods generate, filter, or verify outputs for later training across language reasoning and embodied agents.In VLN, related approaches refine instructions, mine failure trajectories, or reuse successful exploration, generally in coarse rounds that can let supervision lag behind the evolving policy.Source: §2 · p. 4

3 METHODOLOGY

RSI-Nav combines instruction following, instruction generation, and spatial understanding in one model, then evolves it over unlabeled trajectory shards through generate-execute-filter-and-update cycles.

3.1 A UNIFIED VISION-LANGUAGE-ACTION MODEL

  • One shared Qwen3.5-9B model handles instruction following, generation, and spatial understanding through task-specific prompts, without separate generators, verifiers, or output heads.Source: §3.1 · pp. 4–5

3.2 POLICY-SYNCHRONIZED RECURSIVE SELF-IMPROVEMENT

  • Each shard’s generated instructions are executed for goal completion and efficiency; qualified samples update the model before the next shard.Executions qualify when the endpoint is within 3 m and SPL meets a fixed threshold.Source: §3.2 · pp. 5–6, Figure 3 · p. 5
  • Accepted trajectory–instruction pairs supervise both instruction following and generation, while spatial-understanding samples are replayed during shard updates.The original trajectory is retained as the supervision target rather than the executed trajectory.Source: §3.2 · p. 6

3.3 TRAINING DATA AND SUPERVISION

  • Initialization mixes multi-source navigation, reverse-constructed generation, and spatial data; recursive evolution yields about 2.5M following and 1.1M generation samples.Source: §3.3 · p. 6

4 EXPERIMENTS

Experiments compare RSI-Nav with prior methods on continuous navigation and spatial-understanding benchmarks, and test recursive evolution, dual learning, instruction-generation utility, and real-world behavior. Results include strong navigation scores, improvements over supervised initialization, and limits on the evidence for real-world robustness.

4.1 MAIN RESULTS

  • 68.6% SR and 58.6% SPL on R2R-CE, and 72.4% SR, 56.5% SPL, and 70.6 nDTW on RxR-CE Val-Unseen.Table 1 covers these navigation benchmarks as well as four spatial-understanding benchmarks.Source: Table 1 · p. 7, §4.1 · p. 7
  • Recursive evolution improves navigation success and trajectory alignment over supervised initialization; spatial scores rise on VSI-Bench (64.6 to 64.9), ScanQA (93.2 to 96.4), SQA3D (65.1 to 65.4), and MindCube (91.8 to 93.5).Source: §4.1 · p. 8
  • Real-world demonstrations show closed-loop navigation in obstacle avoidance, narrow spaces, multi-step instructions, landmark grounding, long-horizon room-to-room tasks, and outdoor settings.The evaluation deploys RSI-Nav on a DEEP Robotics M20 robot using a remote server for action generation.Source: §4 · p. 7, §4.1 · p. 8, Figure 4 · p. 8

4.2 ABLATION STUDIES

  • On a fixed 64K-trajectory subset, 256 candidates per GPU perform best; 64 yields too little policy change, while 1,024 approaches offline updating and widens the policy-data gap.The selected setting corresponds to a global shard of 16,384 trajectories on 64 GPUs.Source: Table 3 · p. 7, §4.2 · p. 8
  • Adding VA2L raised R2R-CE SR/SPL from 44.7%/38.6% to 48.8%/41.0%; VA2L also beat V2L, 48.8% vs 47.0% SR and 41.0% vs 38.8% SPL.Source: Table 2 · p. 7, §4.2 · pp. 8–9
  • Although SRDF scores higher on most NLP metrics, RSI-Nav-generated supervision yields better downstream R2R-CE performance: 41.3% vs 36.8% SR and 35.1% vs 30.4% SPL.Prior NLP-score comparisons are not strictly controlled because the methods use different environments and observations; the downstream test trains identical navigation models on the generated data.Source: §4.2 · p. 9, Table 4 · p. 9
  • Adding spatial-understanding supervision to VA2L improves all navigation metrics, reducing navigation error while increasing success rate and efficiency.Source: §4.2 · p. 9
  • The method relies on simulated environments and an offline trajectory pool, incurs considerable execution-filtering cost, and lacks systematic robustness evidence across diverse real-world settings, unseen domains, and substantially longer horizons.Source: §4.2 · p. 9

5 CONCLUSION

RSI-Nav unifies instruction following, generation, and spatial understanding in one shared policy, using generated and filtered supervision from unlabeled trajectories. Experiments report navigation improvements, modest spatial-understanding gains, and useful downstream supervision from generated instructions.

5 CONCLUSION

  • RSI-Nav turns unlabeled trajectories into supervision through instruction generation, execution-based filtering, and shard-wise updates in a shared policy.The latest model refreshes the generated supervision while jointly training instruction following and generation.Source: §5 · p. 10
  • Experiments on R2R-CE and RxR-CE demonstrate strong navigation improvements, while spatial-understanding evaluations show modest gains over supervised initialization.Source: §5 · p. 10
  • Generated instructions provide useful downstream navigation supervision, and ablations examine update granularity, dual-task training, action conditioning, and spatial supervision.Source: §5 · p. 10
  • The paper concludes that large-scale unlabeled navigation trajectories support model improvement through generated and filtered supervision.It describes this as a scalable direction for self-improving embodied navigation.Source: §5 · p. 10

AI USE DISCLOSURE

The authors report using generative AI tools for manuscript language polishing and editing, and for software programming.

AI USE DISCLOSURE

A EVALUATION METRICS

The evaluation defines navigation metrics for goal distance, success, efficiency, and route agreement, with scores reported using metric-specific scales.

A EVALUATION METRICS

  • NE measures geodesic distance from the final position to the goal, while SR records whether the agent executes STOP within the 3 m success threshold.NE is reported in meters; SR is reported on a percentage scale, where higher values indicate better performance.Source: A EVALUATION METRICS · p. 15
  • OSR measures whether the agent enters the success region at any point during navigation, regardless of where it stops.Source: A EVALUATION METRICS · p. 15
  • SPL combines success and navigation efficiency: failed episodes receive zero, and successful episodes are penalized for detours.It compares traveled distance with the shortest start-to-goal distance.Source: A EVALUATION METRICS · p. 15
  • nDTW measures how closely the executed trajectory follows the reference route while preserving traversal order; higher scores indicate better agreement.The implementation removes consecutive duplicate predicted positions and uses FastDTW by default to approximate alignment cost.Source: A EVALUATION METRICS · p. 15

B ACTION REPRESENTATION

The action-representation ablation compares symbolic arrows, single-word commands, and multiword commands, with Single performing best across four metrics.

B ACTION REPRESENTATION

  • Arrow represents actions with symbols, Single with forward/left/right/stop, and Double with move forward/turn left/turn right/stop.Source: B ACTION REPRESENTATION · p. 16
  • Single improves SR and SPL over Arrow by 5.44 and 4.87 percentage points, respectively, and achieves the best performance across all four metrics.It also outperforms Double.Source: B ACTION REPRESENTATION · p. 16
  • The results support using single-word action representations in the navigation model.Source: B ACTION REPRESENTATION · p. 16

C ACTION INPUTS AND TRAJECTORY SAMPLING

The section compares image-only instruction generation with action-conditioned generation and evaluates frame-sampling strategies. Uniform-32 and action inputs improve navigation metrics, with the combined Single, VA2L, and Uniform-32 setting achieving the best reported performance.

C ACTION INPUTS AND TRAJECTORY SAMPLING

D DISCRETE-TO-CONTINUOUS TRAJECTORY CONVERSION

The appendices describe converting discrete paths into executable Habitat trajectories, give the navigation and instruction-generation prompt formats, and visualize spatial-question-answering predictions and navigation trajectories.

D DISCRETE-TO-CONTINUOUS TRAJECTORY CONVERSION

  • Discrete paths are mapped into Habitat coordinates using a 1.25 m camera-height offset and navigation-mesh snapping, then followed waypoint by waypoint to form executable trajectories.If snapping fails, the converted camera-offset height is used; trajectories exceeding the 500-action budget are rejected.Source: D DISCRETE-TO-CONTINUOUS TRAJECTORY… · pp. 16–17

E.1 NAVIGATION

  • The navigation prompt supplies the instruction, up to eight history frames, and the current view; the model selects 15-degree turns, 25-centimeter moves, or STOP.The model predicts up to four actions per turn, and the history clause is omitted when no history is available.Source: §E.1 · p. 17

E.2 INSTRUCTION GENERATION

  • The instruction-generation prompt pairs sampled viewpoint images with action spans; V2L removes the action fields.Action spans are rendered as phrases such as “forward 2 steps, then left.”Source: §E.2 · p. 18

F VISUALIZATION OF SPATIAL QUESTION ANSWERING

G VISUALIZATION OF R2R-CE AND RXR-CE

  • Figure 6 visualizes navigation trajectories on R2R-CE and RxR-CE.Source: Figure 6 · p. 19

H LIMITATIONS AND FUTURE WORK

The authors identify two limits: self-improvement depends on the current policy, and execution-based filtering cannot ensure semantic fidelity of accepted instructions.

H LIMITATIONS AND FUTURE WORK

  • Self-improvement is constrained by the current policy, which may reinforce its biases and reject valid but difficult-to-execute instructions, favoring easier trajectories.The authors suggest difficulty-aware selection, adaptive filtering, and complementary verification signals as future directions.Source: H LIMITATIONS AND FUTURE WORK · p. 19
  • Execution-based filtering may accept instructions that reach the destination efficiently despite incorrect landmark descriptions or spatial relations, allowing semantic errors to persist and propagate.The authors propose combining trajectory-level verification with grounded semantic checks and targeted human evaluation.Source: H LIMITATIONS AND FUTURE WORK · p. 19

Related papers

Loading 2610.15782v1…