Learning Goal-Oriented Vision-and-Language Navigation with Self-Improving Demonstrations at Scale

1 Introduction

• Existing imitation-learning agents typically imitate shortest-path samples, leaving them without effective exploration priors and limiting generalization in unseen environments.The introduction contrasts this with costly human demonstrations and reinforcement learning’s reward-engineering, stability, and compute demands.Source: §1 · p. 2
• SID trains an initial agent on shortest-path demonstrations, then collects its successful exploration trajectories as self-demonstrations for training a more capable agent.The next agent is pretrained on self-demonstrations and finetuned on both self-demonstrations and shortest-path data before generating trajectories for another round.Source: §1 · p. 2
• Improved agents generate better exploration trajectories, which in turn improve subsequent training across repeated iterations.When performance saturates in initial environments, SID adds environments such as HM3D and continues training.Source: §1 · p. 3
• SID-VLN uses VLM-generated captions as instructions for exploration trajectories, producing over 46M language-goal trajectories, 20× the scale of prior datasets.The resulting trajectories are used to pretrain an agent and finetune it on downstream tasks.Source: §1 · p. 3
• 50.9% success on SOON, exceeding previous approaches by 13.9%; SID-VLN also achieves state-of-the-art performance on REVERIE and SOON.The paper reports strong transferability to object-goal navigation and VLN-CE.Source: §1 · pp. 3–4

2 Related Work

• Prior VLN methods include LLM planning, visual-linguistic alignment, and map or history-based planning, while costly human-annotated trajectories remain a key obstacle to generalizable agents.Existing data augmentation approaches still do not provide the needed large-scale, high-quality demonstrations of exploration strategies.Source: §2 · p. 4
• Self-improving agents use iterative feedback to improve without extensive human intervention, but robotics makes complex instruction-trajectory alignment sampling impractical.SID treats demonstration quality, rather than correctness alone, as the key obstacle: exploration trajectories can demonstrate more than shortest paths.Source: §2 · p. 4
• Prior demonstration methods include learning from video, human household-task videos, RL-generated trajectories, and human demonstrations for object-navigation exploration.SID instead bootstraps iteratively from its own demonstrations to build an exploration policy without external human effort.Source: §2 · pp. 4–5
• SID’s paradigm trains an initial agent on MP3D trajectories, retains successful agent-generated exploration paths, and uses them to train a more capable agent.The process is presented as iterative semi-supervised learning that can produce transferable exploration demonstrations at scale.Source: Figure 3 · p. 5

3.1 Self-Improving Demonstrations on Image Goal Navigation

• SID builds base training data from MP3D shortest paths of 5–7 viewpoints, yielding over 180,000 trajectories and over 6M visual-goal-trajectory pairs.The agent is initialized with DUET, pretrained and finetuned on this data.Source: §3.1 · p. 6
• After shortest-path training, the agent generates exploration trajectories; only paths that reach the correct target within the length limit are retained for the next iteration.Training on the new data and regenerating demonstrations repeats iteratively; the method also alternates teacher forcing with student forcing supervised by shortest-path oracle actions.Source: §3.1 · pp. 7–8, Figure 4 · p. 7
• Across iterations, trajectories become more learnable, with consistent step lengths, and the agent reaches the correct target directly by the third round.The examples include an initial misdirection into a similar but incorrect room before the agent finds the target.Source: §3.1 · p. 7

3.2 Scaling and Continuing SID in New Environments

• After performance converges on MP3D, SID adds HM3D environments and continues training; the scaled-up process uses one iteration across 800 HM3D environments.A separate experiment adds 60 HM3D environments to the original 60 MP3D scans and continues for two rounds.Source: §3.2 · p. 8

3.3 Transfer to Goal-Oriented Vision-and-Language Navigation

• For transfer to REVERIE and SOON, SID augments exploration trajectories with VLM-generated captions, selecting 18 of 36 panorama images to reduce redundancy.The resulting corpus contains over 46M language-goal exploration trajectories, averaging 9.01 viewpoints, and provides large-scale exploration-strategy demonstrations for goal-oriented VLN.Source: §3.3 · pp. 8–9

4 Experiments

• Experiments cover REVERIE, SOON, Object-Goal Navigation, and VLN-CE, evaluated with path-fidelity and object-grounding metrics.Reported metrics include TL, SR, OSR, SPL, NE, RGS, and RGSPL.Source: §4 · pp. 9–10

4.1 Self-Improving Demonstration Running Results

• In unseen environments, performance improves from 40.24% to 44.62%; successfully sampled trajectories increase while their average length concurrently decreases from 8.27 to 7.27 viewpoints.The authors describe this as self-bootstrapping: agents iteratively learn better exploration strategies through their own demonstrations.Source: Table 3 · p. 10, §4.1 · p. 11
• Adding trajectories from 800 HM3D environments yields a 10% improvement in unseen SR and SPL; SID also retains self-improvement with more HM3D data.The latter experiment adds 60 HM3D environments and continues SID for two iterations.Source: Table 4 · p. 10, §4.1 · p. 11
• Agents trained on self-exploration data visit more and more varied rooms, including more rooms of the target type, than shortest-path trajectories.Room exploration, navigation efficiency, and success rate improve over SID iterations.Source: Table 5 · p. 10, §4.1 · p. 11

4.2 Ablation and Analysis

• Self-demonstrations benefit pretraining and finetuning, outperform random paths, and surpass adding HM3D shortest-path data alone.Random-path training reaches performance equivalent to shortest-path training, while shortest-path scaling modestly improves performance but is distinctly surpassed by agent exploration trajectories.Source: §4.2 · pp. 11–12, Table 6 · p. 12, Table 7 · p. 12

4.3 Comparisons on Downstream VLN tasks

• SID achieves new state-of-the-art results across SOON navigation metrics and on REVERIE, with the SOON test-unseen leaderboard showing the highest success rate.The reported SOON improvement is attributed to more efficient exploration routes rather than merely longer successful episodes.Source: §4.3 · pp. 12–13, Table 9 · p. 13, Table 10 · p. 13
• SID-VLN training reaches 76% success on transferred ObjectNav validation unseen and improves VLN-CE NE, OSR, and SR, with a slight SPL decrease.The VLN-CE passage attributes the SPL decrease to increased exploration.Source: §4.3 · pp. 13–14, Table 11 · p. 13, Table 12 · p. 14

5 Conclusion

• SID is a scalable, transferable learning paradigm in which agents iteratively improve through self-generated, exploration-rich trajectories.The paper says this approach mitigates reliance on costly human annotations for large-scale demonstrations.Source: §5 · p. 14
• Experiments report consistent improvement on image-goal navigation and state-of-the-art performance across diverse goal-oriented VLN tasks.The paper describes transforming SID trajectories into VLN with VLM caption augmentation.Source: §5 · p. 14
• SID-VLN data transfers across different models, ObjectNav, and VLN-CE, supporting the paper's stated potential for unified goal-oriented navigation.The conclusion frames SID as a direction for self-improving embodied learning.Source: §5 · p. 14

Appendix

• Section 1 covers implementation details, pretraining objectives, and Self-Improving Demonstrations experiments.The passage identifies this as the appendix's first section.Source: Appendix · p. 21
• Section 2 details transferring SID exploration priors to VLN tasks and examines caption styles.The appendix overview describes this as an experimental section.Source: Appendix · p. 21
• Sections 3 and 4 give downstream navigation experiment details and visualize trajectories toward targets; Section 5 discusses potential SID limitations.These are the topics listed in the appendix overview.Source: Appendix · p. 21

1.1 Navigation Agent

• DUET combines a global graph-aware encoder for long-range spatial reasoning with a local encoder for visual grounding and action choice.The local encoder processes panoramic imagery and object features, while the global encoder reasons over a topological map.Source: §1.1 · p. 21

1.2 Pretraining Objectives

• Pretraining uses MLM to predict masked target-description words and SAP to predict the next action; sampling emphasizes self-demonstration paths and challenging negative viewpoints.SAP is used throughout SID rounds, while MLM is used during transfer; the revised sampling assigns 75% to self-demonstration points and 25% to challenging negatives.Source: §1.2 · pp. 21–22

1.3 Details of experiments of Self-Improving Demonstration

• The experiments construct 6,524,856 visual-goal trajectories from MP3D and continue SID with HM3D data, ultimately generating 60M trajectories from 860 environments.The reported training process includes three SID rounds on MP3D followed by scaling to 800 HM3D environments and another iteration.Source: §1.3 · pp. 22–24

Caption augmentation

• SID transfers exploratory trajectories to VLN by replacing target images with VLM-generated captions while retaining agent-created trajectories, producing a 3M caption-trajectory corpus.To limit spatial redundancy, 18 of 36 panorama images are selected in an interleaved way.Source: §2 · p. 24

Caption-style comparison

• Three caption styles are tested: Detail-style adds object localization and environmental context, while REVERIE-style and SOON-style follow their respective task annotation settings.The Detail-style prompt includes positional relations; SOON-style captions include object names, attributes, relationships, and target and neighboring areas.Source: §2 · p. 24
• Detail-style caption-trajectory pretraining led to the best zero-shot and finetune performance on REVERIE and SOON among the three styles; finetuned SOON performance surpassed the baseline, with comparable performance on REVERIE.The comparison is reported in Table 14.Source: §2 · p. 24, Table 14 · p. 25

Downstream settings

• For REVERIE and SOON experiments, DUET is finetuned with SID data augmentation on REVERIE, while SOON finetuning uses no augmentation.The reported training configurations use 20k iterations for REVERIE and 5k for SOON.Source: §3 · p. 25

Transfer to VLN-CE

• SID-VLN trajectories are transferred to continuous VLN-CE to train StreamVLN, which uses low-level actions and raw RGB-D observations rather than teleporting between discrete nodes.The baseline uses R2R-CE, R2R-EnvDrop, and ScaleVLN trajectories; adding SID-VLN exploration priors improves success rate on the unseen validation split.Source: §3 · p. 25

• Across rounds, the agent moves from an initial shortest-path trajectory to exploring two similar rooms, then reaches the intended target directly in the third round.The first room lacks distinguishing target elements, including a red valance and a computer on the desk to the left.Source: §4 · p. 26
• Exploration and error correction help the agent distinguish adjacent rooms with highly similar appearances and select the room containing the target.Source: §4 · p. 26
• The visualization presents predicted actions at each step alongside the initial path, exploration trajectory, and final trajectory with the target image and caption.Source: §4 · p. 26

• SID exploration can fail at critical viewpoints with many choices; continued exploration may exceed the maximum step limit and force the agent to stop.The paper identifies more efficient exploration and error correction as future work.Source: §5 · p. 26
• Real-world use faces a potential embodiment gap and a sim-to-real gap, while VLN transfer depends substantially on VLM captions that may be incomplete or inaccurate.The paper notes that navigation-domain caption quality is difficult to evaluate and proposes verifying and refining generated captions.Source: §5 · p. 26