PaperlayerView exact revision on arXiv

Source-linked AI summary

OctoNav: Towards Generalist Embodied Navigation

Chen Gao, Liankai Jin, Xingyu Peng, Jiazhao Zhang, Yue Deng, Annan Li, He Wang, Si Liu

arXiv:2506.09839v1cs.CVcs.AIcs.RO

TL;DR

Embodied navigation research is divided among task-specific benchmarks and methods, leaving a gap in support for generalist agents following free-form, multimodal instructions. OctoNav introduces a benchmark and a VLA-based agent trained with supervised fine-tuning and reinforcement learning. The authors report that OctoNav-R1 outperforms prior methods.

  • Problem

    Navigation benchmarks and methods are divided among task-specific settings, while existing benchmarks cited here do not support free-form instructions combining capabilities and modalities.

    Source: §1 · p. 2

  • Method

    OctoNav pairs a benchmark of free-form multimodal, multi-capability instructions and TBA-CoT annotations with a VLA model trained through Action-/TBA-SFT, Nav-GRPO, and online RL.

    Source: §1 · pp. 2–3, §4 · p. 5

  • Results

    Experiments confirm OctoNav-R1's superiority over prior methods.

    Source: §6 · p. 10

  • Takeaways & Limitations

    The authors conclude that OctoNav-Bench and OctoNav-R1 aim to build generalist agents capable of following free-form multimodal, multi-capability instructions.

    Source: §6 · p. 10

    VLM hallucinations can degrade navigation performance; due to TBA's cost, OctoNav-R1 uses fixed-frequency thinking, while when and where to think remains future work.

    Source: E Appendix: Limitations and Broader… · p. 31

Abstract

from arXiv · show

Embodied navigation stands as a foundation pillar within the broader pursuit of embodied AI. However, previous navigation research is divided into different tasks/capabilities, e.g., ObjNav, ImgNav and VLN, where they differ in task objectives and modalities, making datasets and methods are designed individually. In this work, we take steps toward generalist navigation agents, which can follow free-form instructions that include arbitrary compounds of multi-modal and multi-capability. To achieve this, we propose a large-scale benchmark and corresponding method, termed OctoNav-Bench and OctoNav-R1. Specifically, OctoNav-Bench features continuous environments and is constructed via a designed annotation pipeline. We thoroughly craft instruction-trajectory pairs, where instructions are diverse in free-form with arbitrary modality and capability. Also, we construct a Think-Before-Action (TBA-CoT) dataset within OctoNav-Bench to provide the thinking process behind actions. For OctoNav-R1, we build it upon MLLMs and adapt it to a VLA-type model, which can produce low-level actions solely based on 2D visual observations. Moreover, we design a Hybrid Training Paradigm (HTP) that consists of three stages, i.e., Action-/TBA-SFT, Nav-GPRO, and Online RL stages. Each stage contains specifically designed learning policies and rewards. Importantly, for TBA-SFT and Nav-GRPO designs, we are inspired by the OpenAI-o1 and DeepSeek-R1, which show impressive reasoning ability via thinking-before-answer. Thus, we aim to investigate how to achieve thinking-before-action in the embodied navigation field, to improve model's reasoning ability toward generalists. Specifically, we propose TBA-SFT to utilize the TBA-CoT dataset to fine-tune the model as a cold-start phrase and then leverage Nav-GPRO to improve its thinking ability. Finally, OctoNav-R1 shows superior performance compared with previous methods.

1 Introduction

Embodied navigation research is divided across task-specific benchmarks and methods, limiting support for agents that follow free-form instructions combining modalities and capabilities. OctoNav introduces a unified benchmark and a VLA-based agent trained to handle such instructions with thinking-before-action and reinforcement learning.

1 Introduction

  • Task-specific navigation benchmarks and methods divide ObjNav, PointNav, ImgNav, Ins-ImgNav, and VLN by settings, modalities, and objectives, challenging generalist-agent development.For example, a VLN agent cannot achieve visual goals specified by reference images.Source: §1 · p. 2
  • GOAT-Bench and LHPR-VLN each contain only two capabilities and use single-capability instructions, so they do not support free-form combinations of tasks and modalities.The paper characterizes them as multi-task benchmarks with independent tasks.Source: §1 · p. 2
  • OctoNav-Bench provides 400+ scenes and 45k+ instruction-trajectory pairs, with free-form instructions combining navigation capabilities and text, images, and spatial descriptions.Its TBA-CoT dataset captures deliberative reasoning behind action decisions.Source: §1 · p. 2
  • OctoNav-R1 generates low-level actions from step-wise egocentric visual observations and follows free-form instructions combining multiple modalities and capabilities.Its Hybrid Training Paradigm combines Action-/TBA-SFT, Nav-GRPO, and online RL.Source: §1 · p. 3

2 Related Work

Prior embodied-navigation agents use language and vision-language models for planning, control, or task transfer, but remain limited on instructions that combine modalities and capabilities. Related work also applies reasoning-focused reinforcement learning to models, while robot scenarios remain uninvestigated in that line of work.

2 Related Work

  • Navigation agents use LLMs and MLLMs for high-level planning, low-level control, policy generation, task-specific navigation, or transfer across tasks.Uni-NaVid [32] and NaviLLM [33] fine-tune MLLMs for task transfer, but NaviLLM operates in discrete environments.Source: §2 · p. 3
  • Existing unified navigation models still handle tasks separately and fall short on free-form instructions combining modalities and capabilities.They leverage existing datasets through direct multi-task learning.Source: §2 · p. 3
  • DeepSeek-R1 [16] shows that outcome-only rewards can guide reasoning without step-level annotations, but this training philosophy had not been investigated in robot scenarios.The paper builds OctoNav-Bench with TBA-CoT data and applies its training paradigm to navigation.Source: §2 · p. 3

3 OctoNav-Bench

OctoNav-Bench is a large-scale benchmark for generalist embodied navigation, built from grounded instruction-trajectory pairs and TBA-CoT annotations. Its instructions combine capabilities and modalities, and its comparison with earlier benchmarks highlights these features alongside continuous environments.

3 OctoNav-Bench

  • OctoNav-Bench is proposed as a large-scale benchmark to facilitate generalist embodied navigation.Its scalable, automated pipeline is shown in Figure 2.Source: §3 · p. 3, Figure 2 · p. 4

3.1 Instruction-Trajectory Pairs Construction

  • The benchmark construction pipeline generates diverse capability-combining templates, grounds placeholders in sampled trajectories, extends instructions with variants, and applies automatic and manual checks.Trajectory sampling uses more than 400 indoor environments; visual and positional attributes ground elements such as reference images, coordinates, and object categories.Source: §3.1 · pp. 3–4, Figure 2 · p. 4

3.2 Multi-Modal TBA-CoT Dataset Construction

  • TBA-CoT annotations are produced by converting current, historical, and reference-image cues into language descriptions, then using DeepSeek-R1 to generate reasoning traces tied to ground-truth actions.Qwen-VL produces the current-view description from the image, instruction, and prompts.Source: §3.2 · pp. 4–5, Figure 3 · p. 5

3.3 Data Analysis.

  • OctoNav-Bench unifies navigation tasks in free-form instructions that combine modalities and capabilities, and is presented as the first benchmark with TBA annotations.GOAT-Bench has independent single-modality, single-capability instructions and lacks VLN; LHPR-VLN uses language-only instructions and lacks visual-goal capability.Source: §3.3 · p. 5, Table 1 · p. 5

4 OctoNav-R1

OctoNav-R1 is built to turn multimodal instructions and visual observations into low-level navigation actions through a three-stage Hybrid Training Paradigm (HTP).

4 OctoNav-R1

  • HTP trains OctoNav-R1 through Action-/TBA-SFT, Nav-GRPO, and simulation-based online RL for trial-and-error and active learning.The method is built on LLaMA-VID and produces low-level actions end to end.Source: §4 · pp. 5–6

4.1 Action and TBA Supervised Fine-Tuning

  • Action-SFT learns actions from instruction-trajectory pairs, while TBA-SFT uses TBA-CoT data to train structured <Think> reasoning and <Action> outputs.Prompts can select direct-action or TBA-formatted output, allowing TBA frequency to be regulated at inference.Source: §4.1 · pp. 6–7

4.2 Nav-GRPO

  • After TBA-SFT provides a cold start, Nav-GRPO scores generated actions by action-and-magnitude agreement, giving partial reward for a correct action with an incorrect magnitude.The stepped reward is 1 for an exact match, 0.5 for matching action only, and 0 for a wrong action.Source: §4.2 · p. 7

4.3 Online Reinforcement Learning

  • Online RL uses A2C and rewards successful goal completion with 1, otherwise using distance change; slight forward motion after non-moving actions yields non-zero effective rewards.OctoNav-Bench's continuous simulation environments support this online learning stage.Source: §4.3 · p. 7

5 Experiments

OctoNav-R1 is evaluated on unseen continuous-environment scenes using ordered subtask success metrics, and comparisons show that existing generalist and task-specific models remain weak on the benchmark. Ablations report gains from successive training stages, alongside capability trade-offs and open questions about when to think.

5.2 Comparison with Previous Methods

  • Video-LLaVA achieves 0.80% overall SR, NavGPT-2* reaches 2.00% SR, and Uni-NaVid† reaches 9.20% SR, still far below OctoNav-R1.The revised NaviLLM and NavGPT-2 models are fine-tuned for continuous environments; multi-task NaVid and Uni-NaVid models are also fine-tuned and evaluated.Source: §5.2 · p. 8

5.3 Ablation Study and Visualizations

  • HTP performance improves as stages are added; Nav-GRPO raises overall SR from 14.40% to 17.00%, and online RL adds 2.85% VLN SR and 4.55% ImgNav SR.Online RL also improves overall SPL by 1.73%.Source: Table 3 · p. 9, §5.3 · p. 9

5.1 Experimental Setting

  • The base model is stronger on ObjNav and VLN, but its SR is 5.37% on ImgNav and 7.57% on PointNav.The benchmark uses 400+ training scenes and 40+ unseen test scenes, with 45k+ instruction-trajectory pairs and 10k+ TBA-CoT pairs; success requires ordered completion of all subtasks.Source: §5.1 · p. 8, §5.3 · p. 9

5.3 Ablation Study and Visualizations

  • Action-SFT raises overall SR from 5.80% to 8.80% but lowers VLN SR from 25.71% to 20.00%.The authors describe this as sacrificing VLN capability while improving weaker capabilities and generalist ability.Source: §5.3 · p. 9
  • TBA-SFT improves overall SR by 5.60%, with PointNav SR up 11.55% and ObjNav SR up 15.16%.Nav-GRPO raises VLN SPL to 29.14%; the stepped reward performs best, diverse prompts improve SR, and per-20-step versus per-40-step thinking yields 19.40% versus 18.80% SR.Source: §5.3 · p. 9
  • In the illustrated trajectory, recognizing the chair as already present lets the agent mark the first task complete and plan the next one.The example also shows inference of the bed’s location from common indoor layout.Source: Figure 5 · p. 9, §5.3 · p. 10

6 Conclusion

The conclusion presents OctoNav-Bench and OctoNav-R1 as steps toward following free-form, multimodal, multi-capability instructions, pairing a TBA-CoT dataset with a visual-input VLA model and three-stage training. The accompanying construction details describe how goals, trajectories, instructions, and reasoning data are generated and checked.

6 Conclusion

  • OctoNav-Bench and OctoNav-R1 target free-form, multimodal, multi-capability instructions; the model generates low-level actions from 2D visual observations and uses Action-/TBA-SFT, Nav-GRPO, and online RL.The benchmark includes instruction-trajectory pairs and a TBA-CoT dataset capturing reasoning processes.Source: §6 · p. 10

A.2.1 Ins-ImgNav Goal Generation

  • Ins-ImgNav goals use camera positions that are navigable and visible, then retain views where the target covers more than 20% of the image.Candidate positions are sampled around the target object; cameras point toward its center to maximize its image projection.Source: §A.2.1 · p. 15, Figure 6 · p. 15

A.2.2 Trajectory Generation

  • General trajectories are filtered by distance and straightness, while VLN paths are extended with segments before and after the reference path and screened for continuity.For instructions with k capabilities, trajectory distance is constrained to [3k, 10k] meters; the VLN segment filters use distance-ratio criteria.Source: §A.2.2 · p. 16

A.2.3 Instruction Template Generation

  • Instruction templates combine an initial position and orientation, a sequence of randomly selected capability instructions, and a stop command.Capability templates, conjunctions, and stop instructions are generated into pools, with human checks of generated templates.Source: §A.2.3 · pp. 16–17

A.2.4 PointNav Setting

  • PointNav instructions give starting coordinates and orientation plus absolute target coordinates, but do not provide real-time positional feedback.The design uses the scene’s intrinsic coordinate system rather than relative coordinates.Source: §A.2.4 · p. 17

A.2.5 Instruction Extension and Quality Check.

  • Instruction variants preserve core intent while varying linguistic form, and GPT filtering plus human review removes or corrects problematic examples.The checks target inconsistent variants, hallucinations, and vague goals.Source: §A.2.5 · p. 17

A.2.6 Think-Before-Action Chain-of-Thoughts Generation Pipeline.

  • The TBA-CoT pipeline extracts image information, aggregates historical information, then prompts a reasoning model with observations, trajectory history, instructions, and optional image goals.Historical aggregation retains the most iconic 30% of identified objects and merges synonyms; reference actions are included through multiple rejection sampling without explicit output constraints.Source: §A.2.6 · pp. 17, 19

A.3 Data Analysis

The data analysis describes varied capability combinations, trajectories, target categories, and instruction lengths, with task complexity tracking instruction and path length. The benchmark uses public scenes and reports compliance with their licenses and terms of use.

A.3 Data Analysis

  • Instructions contain one to five capability types in varying orders; five-capability instructions are relatively rare, while one-to-four-capability instructions occur more frequently.VLN occurs less often, but its longer instructions keep its total textual proportion balanced.Source: §A.3 · pp. 20–21
  • Trajectory lengths approximately follow a normal distribution, and ObjNav and Ins-ImgNav target categories span a broad range.The analysis characterizes the average trajectory length as long and its range as wide.Source: §A.3 · p. 21
  • Instruction length and trajectory step length correlate with task complexity, and instruction length is roughly proportional to step length.The scatter chart excludes instructions over 100 words; task groups are easy (one capability), medium (two), and hard (three or more).Source: §A.3 · p. 21

A.4 Data Usage.

  • OctoNav-Bench uses public scenes and simulators, with data use stated to comply with their licenses and terms.Source: §A.4 · p. 21

A.5 Metrics Details

The appendix defines ordered subtask success and path-efficiency measures using capability-specific goal regions and trajectory lengths. It also describes a real-world deployment in which RGB observations go to a remote model server and returned actions are executed by a local motion-planning system.

A.5 Metrics Details

  • A subtask succeeds when the trajectory reaches its successful area after reaching earlier subtask areas in order; success distances are 0.36m for PointNav and ImgNav, 1m for ObjNav and Ins-ImgNav, and 3m for VLN.Source: §A.5 · p. 22
  • The evaluation computes capability-specific metrics and overall metrics from each sample’s ordered subtasks, shortest successful path lengths, and actual trajectory lengths.Unsuccessful subtasks assign infinite values to the corresponding trajectory lengths.Source: §A.5 · pp. 21–22

B.1 Robot Setup

  • The Unitree GO2 setup uses 640 × 480 RGB frames at 90° HFOV; its LiDAR is reserved for the local motion-planning API.A RealSense D455 camera captures the RGB frames, and portable Wi-Fi carries images and commands to and from the server.Source: §B.1 · p. 23, Figure 15 · p. 23

B.2 Deployment Details

  • A remote server processes transmitted images and sends action commands back for execution by Unitree’s off-the-shelf local motion-planning model.The images are compressed before transmission.Source: §B.2 · p. 23

B.3 Visualization in the Real-World

Real-world demonstrations show OctoNav-R1 recognizing spatial relationships and completing a multi-stage task. The authors report preliminary sim-to-real transfer without real-world fine-tuning.

B.3 Visualization in the Real-World

  • The robot distinguishes left from right chairs and successfully navigates to a requested point.These demonstrations test its recognition of spatial relationships.Source: §B.3 · p. 23
  • In a multi-stage task, the robot finds a chair and then navigates to a TV shown in an image.The authors describe this as evidence of multimodal cognition and task planning.Source: §B.3 · p. 24
  • The authors observe preliminary sim-to-real transfer without real-world fine-tuning.Source: §B.3 · p. 24

C.1 Model Architecture

OctoNav-R1 adapts a language-and-vision model for multimodal inputs and low-level navigation actions. The appendix describes its architecture, training settings, and the modifications used to evaluate baseline models in continuous environments.

C.1 Model Architecture

  • OctoNav-R1 builds on LLaMA-VID and adds task-specific designs for multimodal instruction input and end-to-end low-level action generation.The implementation uses EVA-CLIP, BERT, and Vicuna-7B.Source: §C.1 · pp. 24–25
  • Visual observations and language are encoded, while query tokens and cross-attention align observation features with the LLM embedding space.Goal-image embeddings replace the corresponding image-goal placeholders.Source: §C.1 · pp. 24–25

C.2 Implementation Details.

  • Training uses Action-SFT for nearly 10k steps, TBA-SFT for nearly 6k, GRPO for 1,000, and online reinforcement learning for 500 steps on a single A800.LoRA is used across all training stages.Source: §C.2 · p. 25

C.3 Baseline Methods

  • Open-source MLLM baselines receive combined or video-based observation inputs, with image goals included in their inputs.Qwen-VL uses a combined image, while Video-LLaVA and LLaVA-NeXT-Video receive historical observations as video.Source: §C.3 · pp. 25–26
  • For continuous-environment evaluation, NaviLLM and NavGPT-2 are adapted to output actions, while NaVid and Uni-NaVid are evaluated with image-goal embeddings.NaviLLM's output head is changed from waypoint selection to action selection.Source: §C.3 · p. 26

D.1 Visualization on OctoNav-Bench

Visualizations illustrate OctoNav-R1 handling chained image, point, and language-navigation tasks, adjusting plans, and correcting navigation errors. Additional comparisons report state-of-the-art performance across instruction lengths, with a larger hard-instruction success rate than the comparison methods.

D.1 Visualization on OctoNav-Bench

  • OctoNav-R1 completes chained InstanceImageNav, ImageNav, and object-finding tasks, while Uni-NaVid ignores the first two tasks in one example.Other visualizations show it combining image and point targets and following a VLN instruction after an ImageNav goal.Source: §D.1 · pp. 26, 28
  • When a bed target is not in the current area, the agent changes direction and moves toward the bedroom; another example shows it exploring an adjacent space when a target is absent.The authors present these behaviors as navigation error correction and planning under difficulty.Source: §D.1 · pp. 28–29

D.3 More Comparison Results

  • 7.24% vs. 2.63% SR for hard instructions; OctoNav-R1 achieves state-of-the-art performance on easy, medium, and hard instructions.The test groups contain one task, two tasks, and at least three tasks, respectively.Source: Table 8 · p. 30, §D.3 · p. 30

D.2 Ablation Study

E Appendix: Limitations and Broader Impacts

The authors identify hallucinations and the cost of the think-before-action paradigm as limitations. They also note safety risks from real-world navigation errors and point to possible relevance beyond ground robots.

Loading 2506.09839v1…