Source-linked AI summary
VLN-R1: Vision-Language Navigation via Reinforcement Fine-Tuning
Zhangyang Qi, Zhixiong Zhang, Yizhou Yu, Jiaqi Wang, Hengshuang Zhao
TL;DR
VLN-R1 targets the limitations of graph-based VLN by directly mapping egocentric video to continuous navigation actions with LVLMs. It combines VLN-Ego data, SFT, and GRPO-based RFT with Time-Decayed Reward, achieving state-of-the-art performance on VLN-CE and efficient cross-domain adaptation.
Problem
Existing VLN approaches rely on fixed navigation graphs, while continuous methods may require additional sensors, maps, or specialized vision-language modules that limit generalizability.
Method
VLN-R1 trains LVLMs on Habitat-generated VLN-Ego videos using SFT followed by GRPO-based RFT with Long-Short Memory Sampling and Time-Decayed Reward.
Results
VLN-R1 achieves state-of-the-art performance on VLN-CE, with RFT allowing a 2B model to match 7B performance and supporting effective cross-domain adaptation with minimal data.
Takeaways & Limitations
The results support using LVLMs with reward-driven post-training for continuous embodied navigation and data-efficient transfer across navigation domains.
Takeaways & Limitations
Evaluation is limited to simulated indoor environments, and the discrete action space restricts fine-grained control.
Abstract
from arXiv · showhide
Vision-Language Navigation (VLN) is a core challenge in embodied AI, requiring agents to navigate real-world environments using natural language instructions. Current language model-based navigation systems operate on discrete topological graphs, limiting path planning to predefined node connections. We propose VLN-R1, an end-to-end framework that leverages Large Vision-Language Models (LVLM) to directly translate egocentric video streams into continuous navigation actions, adopting GRPO-based training inspired by DeepSeek-R1. To enable effective training, we first construct the VLN-Ego dataset using a 3D simulator, Habitat, and propose Long-Short Memory Sampling to balance historical and current observations. While large language models can supervise complete textual instructions, they lack fine-grained action-level control. Our framework employs a two-stage training approach: a) Supervised fine-tuning (SFT) to align the model's action sequence text predictions with expert demonstrations, followed by b) Reinforcement fine-tuning (RFT) enhanced with a Time-Decayed Reward (TDR) mechanism that strategically weights multi-step future actions. Experimental results show VLN-R1 achieves strong performance on VLN-CE benchmark. VLN-R1 proves LVLMs can drive embodied navigation and enhance task-specific reasoning through data-efficient, reward-driven post-training.
1 Introduction
VLN-R1 addresses the limitations of graph-based navigation by using egocentric video and LVLMs for continuous navigation. It combines VLN-Ego data, two-stage SFT/RFT training, and Time-Decayed Reward to improve long-horizon action prediction.
- Problem: Existing VLN systems rely on fixed-connectivity graphs, while continuous-space methods often require depth maps, navigation maps, or specialized vision-language models.These dependencies limit generalization to unseen or continuous environments and human-agent interactions.
- Framework: VLN-R1 directly processes egocentric video streams with LVLMs to navigate continuous environments without relying on predefined navigation graphs.The framework uses first-person visual input rather than third-person path planning over discrete positions.
- Data: VLN-Ego pairs Habitat-generated egocentric video streams with future action predictions and uses Long-Short Memory Sampling to balance historical context with real-time sensitivity.The dataset is designed for training LVLMs on continuous vision-and-language navigation tasks.
- Training: Training first applies SFT to align textual action predictions with demonstrations, then applies GRPO-based RFT with Time-Decayed Reward for multi-step action evaluation.The action space contains FORWARD, TURN-LEFT, TURN-RIGHT, and STOP, while TDR targets long-horizon navigation performance.
- Results: VLN-R1 achieves state-of-the-art navigation performance on continuous R2R and RxR benchmarks and also reports embodied question answering results.The navigation evaluation uses VLN-CE versions of both benchmarks.
2 Related work
Related work spans continuous VLN methods, language-model-based navigation, and reinforcement fine-tuning for LVLMs. VLN-R1 extends this trajectory by applying GRPO and verifiable rewards to continuous navigation decision-making.
- Vision-and-Language Navigation: VLN evolved from discrete node-based navigation toward VLN-CE, which supports low-level actions and free movement in continuous simulated environments.Approaches include direct control prediction, waypoint-based subgoals, and vision-language or task-specific pre-training.
- Language models as VLN agents: LLM-based VLN methods improve instruction generalization and cross-scene adaptability by converting visual observations into textual descriptions for language-guided planning.These systems commonly combine off-the-shelf language models with vision foundation models.
- Reinforcement Fine-Tuning: RFT has mainly been used for LVLM hallucination mitigation or preference alignment, while VLN-R1 applies verifiable rewards and GRPO to continuous navigation decision-making.The paper characterizes this as the first application of RFT to continuous navigation decision-making in VLN.
3 Methodology
VLN-R1 combines Habitat-based data construction, Long-Short Memory Sampling, and two-stage SFT/RFT training to produce continuous navigation actions from egocentric visual input.
- 3.1 Preliminary: RLVR uses verifiable rewards for objective-answer tasks, while VLN-R1 extends this post-training direction with GRPO and navigation-specific reward design.GRPO operates online through group-wise reward normalization without an explicit reward model.
- 3.2 Dataset Engine: VLN-Ego: VLN-Ego is collected in Habitat and organizes action-by-action annotations into instruction, vision, and action components.The vision component includes historical frames and the current observation, while the action component contains four choices and six ground-truth future actions.
- 3.2 Dataset Engine: VLN-Ego: Long-Short Memory Sampling selects recent frames densely within a short-term window and older frames more sparsely to preserve both local details and long-range context.Short-term sampling uses interval δ1 within M steps, whereas long-term sampling uses δ2 with δ2 > δ1.
- 3.3 Supervised Fine-Tuning (SFT) Stage: SFT autoregressively aligns multi-step action-text predictions with ground-truth text using cross-entropy over action identifiers and motion descriptions.The model jointly predicts discrete option identifiers and linguistically coherent action descriptions.
- 3.4 Reinforcement Fine-Tuning (RFT) stage: RFT applies GRPO after SFT and introduces Time-Decayed Reward to provide position-aware supervision for temporal action predictions.The reward gives greater weight to earlier correct actions through an exponentially decaying γk term, addressing SFT’s limited action-level guidance.
4 Experiments
Experiments evaluate VLN-R1 on VLN-CE R2R and RxR Val-Unseen, including main comparisons and training-stage ablations. Results support RGB-only navigation, RFT-based smaller-model performance, cross-domain transfer, and the proposed memory and reward designs.
- Experimental setup: Experiments evaluate VLN-R1 on VLN-CE R2R and RxR using held-out Val-Unseen scenes and standard navigation metrics.The setup uses 18 held-out scenes and reports SR, OS, and SPL.
- Main results: VLN-R1 achieves state-of-the-art performance on VLN-CE R2R using only RGB video inputs.The 2B model after RFT performs similarly to the 7B model, and exceeds the 7B SFT result.
- Main results: RFT enables cross-domain adaptation on RxR: after R2R training, only 10K RxR samples outperform a counterpart trained on the complete dataset.The result indicates strong transfer with a small amount of new-domain data.
- SFT ablations: SFT ablations find that predicting the next 6 actions performs best, while single-action prediction degrades because it misses future-step dependencies.The Long-Short Memory strategy also provides the best frame selection by balancing current observations and historical context.
- Qualitative results: Qualitative results show VLN-R1 converting egocentric video into navigation through a continuous environment toward the target location.The figure illustrates the system’s continuous-environment behavior.
- RFT ablations: RFT ablations select 8 generations because improvement from 6 to 8 is marginal, while Table 3d supports the effectiveness of Time-Decayed Reward.The reward study evaluates alternative reward functions during the RFT stage.
5 Conclusion and Limitation
VLN-R1 is presented as an end-to-end LVLM framework for continuous navigation, combining RFT with GRPO and TDR. It achieves SOTA on VLN-CE, but evaluation is limited to simulated indoor environments and a discrete action space.
- Conclusion: VLN-R1 processes egocentric visual input with LVLMs for continuous VLN without navigation graphs or additional sensors.The framework integrates Qwen2-VL with an end-to-end embodied navigation pipeline.
- Conclusion: RFT with GRPO and TDR improves long-horizon decision-making, while the 2B model matches 7B performance after RFT.The conclusion also reports efficient cross-domain adaptation with minimal data.
- Limitations: The method’s limitations are evaluation only in simulated indoor environments and a discrete action space that restricts fine-grained control.These constraints limit real-world generalization and action precision.