Source-linked AI summary
SimpleVLN: Towards Simple and Frontier Vision-Language Navigation
Lingyi Hong, Weiren Cai, Jiayu Chen, Xinyu Zhou, Bailin He, Yunfei Guo, Qiao Sun, Boyu Ma, Jianfei Yang, Wenqiang Zhang
TL;DR
VLN asks embodied agents to ground language in visual observations and make long-horizon decisions, while task-specific additions leave native MLLM capability unclear. SimpleVLN adapts native MLLMs through navigation formulation, data construction, and progressive optimization. It reports 76.7% SR on R2R-CE and 80.1% SR on RxR-CE val-unseen, substantially outperforming existing methods.
Problem
Because augmented MLLM systems combine backbone and task-specific components, their performance leaves the capability of a native MLLM alone unclear for VLN.
Source: §1 · p. 1Method
SimpleVLN adapts native MLLMs to VLN with single-turn inputs, managed visual history, action chunks, expert and DAgger data, synthesized RL tasks, and progressive IL–DAgger–RL training.
Source: §1 · p. 2, §3.2 · pp. 3–4, §3.4 · p. 5Results
76.7% SR on R2R-CE and 80.1% SR on RxR-CE val-unseen; SimpleVLN substantially outperforms existing methods.
Source: §1 · p. 2Takeaways & Limitations
The results demonstrate that native MLLM navigation capabilities can be effectively unlocked without task-specific architectural designs.
Source: §5 · p. 10
Abstract
from arXiv · showhide
Vision-and-Language Navigation (VLN) requires embodied agents to ground language in visual observations and make long-horizon decisions. Existing methods often address these demands through task-specific architectures. As multimodal large language models (MLLMs) become increasingly capable, an important question remains: Have we truly unlocked the potential of native MLLMs for VLN before introducing task-specific architecture? We present SimpleVLN, an architecture-preserving framework developed through studies of navigation formulation, data construction, and optimization. At the formulation level, SimpleVLN combines single-turn decision inputs with fixed-budget global-local memory to efficiently use navigation history without unbounded context growth, while action chunks enable efficient output of multiple primitive actions per model invocation. For data construction, we combine expert demonstrations with policy-dependent DAgger experience and develop a scalable synthesis pipeline that generates large-scale tasks for reinforcement learning. At the optimization level, a progressive IL-DAgger-RL pipeline advances from action imitation to task-level outcome optimization. Comprehensive experiments validate the effectiveness of these three aspects. We instantiate SimpleVLN across multiple MLLM backbones and model sizes, demonstrating its generality. SimpleVLN achieves 76.7% and 80.1% success rates on the R2R-CE and RxR-CE splits, respectively, substantially outperforming existing methods. These results show that the navigation capabilities of native MLLMs can be effectively unlocked without task-specific architectural designs.
1 Introduction
SimpleVLN asks whether native MLLMs can navigate effectively without task-specific architectural additions. It combines a navigation-specific formulation, data construction, and progressive optimization, and reports strong results on two continuous-environment benchmarks.
1 Introduction
- The motivating question is whether native MLLMs' VLN capability can be established independently of task-specific components in augmented systems.The passage notes that augmented-system performance reflects both the backbone and task-specific components.Source: §1 · p. 1
- 76.7% SR on R2R-CE and 80.1% SR on RxR-CE val-unseen; SimpleVLN substantially outperforms existing methods.The comparison spans multiple MLLM backbones and model sizes.Source: Figure 1 · p. 2, §1 · p. 2
- Single-turn inputs, fixed-budget global–local memory, and action chunks adapt native MLLMs to sequential navigation without task-specific architectural modules.The memory uses navigation history without unbounded context growth, while action chunks predict multiple primitive actions per model invocation.Source: §1 · p. 2
- Training combines expert demonstrations, policy-dependent DAgger experience, scalable RL task synthesis, and progressive IL–DAgger–RL optimization.The stages progress from action imitation through policy-induced states to task-level outcome optimization.Source: §1 · p. 2
2 Related Work
Prior VLN work spans discrete viewpoints and continuous environments, while MLLM-based approaches add specialized components for long-horizon navigation. SimpleVLN instead adapts a pretrained MLLM directly and combines imitation, DAgger, and reinforcement learning.
2 Related Work
- VLN-CE extends earlier navigation over predefined discrete viewpoints to continuous environments with low-level actions.The cited background describes instruction-following grounded in visual observations.Source: §2 · p. 2
- Recent MLLM-based VLN methods add streaming contexts, 3D geometry encoders, dual-system architectures, or explicit chain-of-thought reasoning.SimpleVLN contrasts by directly adapting a pretrained MLLM without architectural modifications.Source: §2 · p. 2
- SimpleVLN feeds sampled history frames, the current observation, and the instruction to an MLLM, which predicts an action chunk.The overview depicts the framework's decision-level input and output.Source: Figure 2 · p. 3
- SimpleVLN combines imitation learning, DAgger, and RL in a progression from expert supervision to task-completion optimization.The background characterizes imitation learning as vulnerable to policy-execution distribution shift, which DAgger addresses with supervision on model-induced states.Source: §2 · p. 3
3 SimpleVLN
SimpleVLN adapts native MLLMs to VLN without architectural modifications by redesigning decision inputs, training data, and optimization. Its pipeline bounds visual history and progresses from expert action supervision to task-completion feedback.
3.1 Overview
- At each primitive step, the agent observes an egocentric RGB image and chooses forward, a 15° turn, or STOP; success requires stopping within 3 m of the target.An episode also terminates when the primitive-action budget is reached.Source: §3.1 · p. 3
3.2 Decision-Level Navigation Formulation
- Each decision uses the instruction, current observation, and sampled history—not prior dialogue or actions—while bounded global-local history supports action chunks of multiple primitive actions.At most N + M + 1 visual inputs are used per decision, independently of episode length; the environment executes each chunk sequentially before the next model query.Source: §3.2 · pp. 3–4
3.3 Navigation Data Construction
- Training combines R2R, RxR, and ScaleVLN expert trajectories, deviation-based DAgger aggregation, and newly synthesized RL tasks with generated instructions.The RL tasks use sampled start–goal pairs in different rooms in training scenes, excluding val-unseen scenes; reference paths construct tasks rather than provide action targets.Source: §3.3 · p. 5
3.4 Progressive Optimization
- Optimization progresses from expert action-chunk imitation through supervised adaptation on aggregated DAgger data to task-completion RL, rewarding any successful trajectory.RL uses a binary task-level reward on synthesized episodes rather than requiring agreement with a reference action sequence.Source: §3.4 · p. 5
4 Experiments
Experiments compare SimpleVLN with prior methods, ablate its navigation formulation and training choices, and evaluate it on a real robot. Results cover R2R-CE and RxR-CE val-unseen benchmarks and include a real-world demonstration.
4.1 Experimental Setup
- Evaluation uses R2R-CE and RxR-CE val-unseen in Matterport3D with Habitat, reporting five metrics and treating SR as primary.The main-results RL configuration uses DAPO with batch size 256 and 16 rollouts per prompt.Source: §4.1 · pp. 5–6
4.2 Comparison with State-of-the-Art
- 76.7 SR on R2R-CE and 80.1 SR on RxR-CE; SimpleVLN outperforms existing methods using single-RGB observations and the native MLLM architecture.Table 1 compares methods on both val-unseen splits, with pre- and post-RL SimpleVLN variants.Source: Table 1 · p. 6, §4.2 · p. 6
4.3 Formulation
- Action chunking raises SR from 45.6 to 53.6; single-turn input raises it from 35.7 to 53.6, and eight sampled global frames reach 53.6, with memory set to M = 8 and N = 1.The memory setting allows up to eight global and one local history frame, plus the current observation.Source: §4.3 · p. 7, Figure 4 · p. 7
4.4 Data Construction
- Broader expert coverage, policy-dependent DAgger data, and synthesized scene-pool mixtures improve success across both benchmarks.Adding RxR to R2R raises SR from 53.6 to 56.7 on R2R-CE and 50.3 to 54.5 on RxR-CE; adding ScaleVLN raises them to 57.9 and 55.3. DAgger adds 7.5 and 8.1 SR points, reaching 65.4 and 63.4; mixed synthesized scene pools reach 70.1 and 71.0.Source: §4.4 · p. 8, Table 4 · p. 8, Table 5 · p. 8
4.5 Optimization
- Binary success rewards and the evaluated DAPO configuration improve reported SR, while RL training trends toward higher success and fewer action-chunk decisions.With batch size 32, DAPO reaches 71.5/73.7 SR versus 70.1/71.0 for GRPO; the authors do not attribute the gain solely to the optimization algorithm.Source: §4.5 · p. 9, Table 6 · p. 9
4.6 Real-world Experiments
- On a Daystar quadruped robot, SimpleVLN follows multi-step instructions using onboard inference and RGB observations.Demonstrated behaviors include turns, room entry, and landmark-based stopping over extended trajectories.Source: Figure 6 · p. 9, §4.6 · p. 10
5 Conclusion
SimpleVLN adapts native MLLMs to VLN through managed inputs, memory, action chunks, and staged training, achieving strong navigation performance without task-specific architectural designs.
5 Conclusion
- SimpleVLN combines single-turn inputs, global–local memory, action chunking, expert and policy-dependent DAgger data, and synthesized RL tasks.A progressive IL–DAgger–RL pipeline shifts learning from action imitation to task-level outcome optimization.Source: §5 · p. 10
- SimpleVLN achieves strong navigation performance while demonstrating native MLLM capabilities can be unlocked without task-specific architectural designs.Source: §5 · p. 10
- SimpleVLN is a framework for adapting native MLLMs to VLN.Source: §5 · p. 10