Source-linked AI summary
SeeNav-Agent: Enhancing Vision-Language Navigation with Visual Prompt and Step-Level Policy Optimization
Zhengcheng Wang, Zichuan Lin, Yijun Yang, Haobo Fu, Deheng Ye
TL;DR
LVLM-based VLN agents encounter perception, reasoning, and planning errors that limit navigation performance. SeeNav-Agent combines dual-view visual prompting with SRGPO, which estimates step-level advantages by randomly grouping navigation steps. On EmbodiedBench-Navigation, its reported comparisons show gains over prior state-of-the-art models.
Problem
LVLM-based VLN agents encounter perception, reasoning, and planning errors that limit their navigation performance.
Source: §1 · p. 1Method
Results
GPT4.1+VP surpasses the current SOTA close-source model by 20 pp, while Qwen2.5-VL-3B-Instruct+VP+SRGPO exceeds the previous SOTA close-source model by 5.6 pp.
Source: §1 · p. 2Takeaways & Limitations
SRGPO exhibits superior convergence speed, training stability, and generalization ability compared with GRPO and GiGPO.
Source: §1 · p. 2The authors identify evaluation on more diverse and complex navigation benchmarks as future work.
Source: §5 · p. 9
Abstract
from arXiv · showhide
Existing Vision-Language Navigation (VLN) agents based on Large Vision-Language Models (LVLMs) often suffer from perception errors, reasoning errors, and planning errors, which significantly hinder their navigation performance. To address these limitations, a novel VLN agent framework, named SeeNav-Agent, is proposed in this work. First, to reduce perception hallucinations of the visual module of the VLN agent, a dual-view Visual Prompt (VP) technique is introduced in the input space, which can also improve the agent's understanding of current spatial states. Subsequently, a novel step-level Reinforcement Fine-Tuning (RFT) method, Step Reward Group Policy Optimization (SRGPO), is designed for the post-training of VLN agents. In SRGPO, we first define verifiable process rewards for the navigation task, and then perform efficient step-level advantage estimation by randomly grouping different navigation steps. SRGPO provides dense reward signals for the reinforcement learning process of the VLN agent and enhances its planning capability. Experimental results on the EmbodiedBench Navigation benchmark indicate that by introducing the zero-shot VP module, the GPT-4.1 achieves a navigation success rate of 86.7%, surpassing the current best LVLM by approximately 20 percentage points (pp). Through post-training based on SRGPO, the Qwen2.5-VL-3B model reaches a navigation success rate of 72.3%, outperforming the best existing LVLM model by 5.6 pp. Moreover, compared to RFT algorithms such as GRPO and GiGPO, the proposed SRGPO demonstrates significant improvements in training stability, convergence efficiency, and generalization capability.
1. Introduction
LVLM-based VLN agents face perception, reasoning, and planning errors, while prior visual-prompt and process-based reinforcement fine-tuning approaches leave gaps in multi-view prompting and efficient step-level optimization. SeeNav-Agent combines dual-view visual prompting with SRGPO and reports improved benchmark performance.
1. Introduction
- Prior VP work adds image markers, but the introduction identifies a lack of research on effective multi-view inputs and systematic study of how VP types complement one another.The cited figure is described as covering navigation error types and examples, but does not provide further findings in the supplied passage.Source: §1 · p. 1, Figure 1 · p. 2
- Existing RFT methods are described as relying on sparse outcome rewards or identical-state anchors, making process-based advantage estimation inefficient when anchor states are hard to define.The introduction frames efficient, stable process-based RFT for navigation as an unresolved challenge.Source: §1 · p. 2
- Dual-view visual prompting is proposed to reduce hallucinations and improve spatial understanding, while action projection reframes navigation as a VQA problem.The framework also applies SFT and RFT to enhance reasoning and planning; SRGPO uses verifiable step rewards and randomly groups navigation steps to compute relative advantages.Source: §1 · p. 2
- GPT4.1+VP surpasses the current SOTA close-source model by 20 pp, while Qwen2.5-VL-3B-Instruct+VP+SRGPO exceeds the previous SOTA close-source model by 5.6 pp.The comparisons are reported for the EmbodiedBench Navigation benchmark.Source: §1 · p. 2
2. Related Work
Related work reviews LVN pipelines and visual prompting, alongside reinforcement fine-tuning methods for navigation. It positions SeeNav-Agent’s multi-view prompting and random step grouping against limitations in prior approaches.
Vision-and-Language Navigation (VLN) / Visual Prompt (VP) for VLN Agents
- Prior LVLM-based VLN work uses end-to-end multimodal models, while earlier VP approaches project actions or annotate walkable areas but lack systematic study of multi-view inputs and coordination between VP modules.The passage also contrasts these approaches with traditional pipelines that separate perception, map segmentation, reasoning, and planning.Source: §2 · p. 2
Reinforcement Fine-Tuning (RFT) for VLN Agents
- GRPO is described as using sparse outcome rewards, while GiGPO groups identical states and can require many rollouts, motivating SRGPO’s random step grouping.The passage says SRGPO uses a robust, verifiable process reward and is designed to address GiGPO’s identical-state grouping requirement.Source: §2 · p. 3
3. Methodology
SeeNav-Agent combines dual-view visual prompting with a reinforcement fine-tuning method that uses episode- and step-level navigation rewards. Its visual prompts clarify scene and action information, while SRGPO estimates step advantages by grouping steps without requiring identical anchor states.
3.1. Framework and Problem Formulation
- The LVLM navigation policy uses instructions, environmental images, and action-feedback history to produce analysis and an action at each step.The framework also represents the agent's policy as a parameterized policy network.Source: §3.1 · p. 3
3.2. Dual-View Visual Prompt for VLN Agent
- Dual-view prompts combine FV and BEV with target boxes, navigation lines, an oriented agent marker, action projection, and view alignment.Action projection makes potential movement and rotation actions visible, recasting action planning as visual question answering.Source: §3.2 · pp. 3–4
3.3. Step Reward Group Policy Optimization
- SRGPO rewards steps for reducing target distance or restoring target visibility, randomly groups steps without identical-state anchors, and combines step- and episode-level advantages.Its process reward compares consecutive-step distance and front-view target visibility, allowing flexible step grouping across states.Source: §3.3 · p. 5
4. Experiments
On EmbodiedBench Navigation, SeeNav-Agent improves success rates over the reported baselines; ablations and training analyses examine its visual prompts, SRGPO, and generalization.
4.1. Experimental Setup
- The EmbodiedBench Navigation benchmark has 60 in-room tasks, and success depends on whether the robot comes within a predefined distance of the target.The setup also uses randomized scenes and goals; o.o.d. training uses 60 new AI2-THOR scenes.Source: §4.1 · p. 6
4.3. Ablation Study for Visual Prompt
- 21.7 pp improvement in zero-shot success rate is reported with VP; dual-view input and view alignment alone may reduce performance, while BB, AP, and VA are especially important.The ablation results indicate that the full VP recipe is needed for the optimal success rate. In the illustrated case, VP helps the agent recognize that the target is out of view and rotate it back into view.Source: §4.3 · p. 6, Table 2 · p. 7
4.2. Main Results
- 21.7 pp over Claude-3.5-Sonnet, GPT4.1 with VP improves success rate; Qwen2.5-VL-3B-Ins with the method exceeds InternVL3-78B by 5.6 pp.The results are reported in Table 1; RFT cases use in-distribution training settings.Source: §4.2 · p. 6, Table 1 · p. 7
4.4. Effectiveness of SRGPO
- 14.5 pp higher test success than VP+GiGPO(w. VPR) and 31.7 pp higher than VP+GRPO are reported for VP+SRGPO.On training curves, SRGPO also shows faster convergence and smaller standard deviation than GRPO and vanilla GiGPO; GiGPO with VPR approaches SRGPO's training success but converges more slowly.Source: §4.4 · p. 7, Figure 4 · p. 7
4.5. Discussion of Hyperparameter for SRGPO
- With NS reduced from 16 to 8, SRGPO's test success rate averaged 0.411 with standard deviation 0.103; increasing NS requires no additional rollout computation cost.The authors associate smaller groups with less accurate and robust step-level advantage estimates and weaker generalization.Source: §4.5 · pp. 7–8
4.6. Out-Of-Domain Generalization Capability
- Under o.o.d. training, SRGPO's test success curve is higher than those of the other three algorithms, indicating stronger navigation capability in that setting.The authors interpret this result as evidence of improved generalization to new scenes.Source: §4.6 · p. 8
5. Conclusion
SeeNav-Agent combines zero-shot dual-view visual prompting with SRGPO post-training. On EmbodiedBench-Navigation, the reported results exceed prior state-of-the-art models, while broader and more complex benchmark evaluation remains future work.
5. Conclusion
- Figure 6 evaluates RFT algorithms trained on o.o.d. scenes and tested on base scenes of EmbodiedBench-Navigation.Source: Figure 6 · p. 8
- SRGPO uses state-independent verifiable process rewards and randomly groups navigation steps for efficient step-level advantage estimation.The conclusion reports advantages over GRPO and GiGPO in training stability, convergence efficiency, and generalization ability.Source: §5 · p. 8
- Dual-view visual prompting is reported to reduce visual hallucinations and improve spatial understanding, with ablations identifying an optimal recipe for aligning dual-view inputs.The framework uses visual prompting at the input and SRGPO for post-training.Source: §5 · p. 8
- GPT4.1+VP surpasses the previous SOTA close-source model by 20 pp, while VP+SRGPO exceeds the previous SOTA model by 5.6 pp.Both comparisons are reported on EmbodiedBench-Navigation.Source: §5 · p. 9
- The authors identify evaluation on more diverse and complex navigation benchmarks as future work.Source: §5 · p. 9
A.1. Parameters for EmbodiedBench-Navigation Environment
The EmbodiedBench-Navigation setup specifies the agent’s visual range and image dimensions, movement and rotation increments, and task-distance success conditions.
A.1. Parameters for EmbodiedBench-Navigation Environment
- The environment uses 500px-wide, 500px-high images, a 10m visible distance, and a 100° field of view.Source: §A.1 · p. 11
- Each movement step covers 0.25m, while each rotation turns the view by 90° and each camera tilt turns it by 90°.Source: §A.1 · p. 11
- The starting position is 2.5–3m from the target, and success requires reaching within 1m of it.Source: §A.1 · p. 11
A.2. Example of Language Prompts
The prompt template combines the navigation instruction, recent interaction history, and visual input with explicit action-selection guidance and a structured output format.
A.2. Example of Language Prompts
- The prompt includes an image placeholder, the human navigation instruction, and interaction history limited to the previous five steps.The prompt removes instructions for visual-prompt modules not used in a given experiment.Source: §A.2 · p. 11
- The interaction-history example records action outcomes and feedback, including two lateral moves reported as invalid because cubes blocked them.Source: §A.2 · p. 11, Table 3
- The input combines top-down and first-person views; colored markers show position and orientation, while red boxes mark the target and blue arrows identify candidate actions.Source: §A.2 · p. 12
- Actions 0–3 are translations and actions 4–7 are rotations or camera tilts, with arrows indicating movement direction or view rotation.The action set includes 0.25-step translations, 90° rotations, and 30° camera tilts.Source: §A.2 · p. 12
- The strategy instructions ask the agent to account for its orientation, obstacles, target visibility, and prior invalid actions when choosing a step.Rotation or camera tilt is reserved for when the target is out of view.Source: §A.2 · p. 12
- The requested JSON output separates visual-state description, reasoning and reflection, language plan, and executable action plan.Source: §A.2 · p. 12