PaperlayerView exact revision on arXiv

Source-linked AI summary

NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoning

Bingqian Lin, Yunshuang Nie, Ziming Wei, Jiaqi Chen, Shikui Ma, Jianhua Han, Hang Xu, Xiaojun Chang, Xiaodan Liang

arXiv:2403.07376v2cs.CVcs.AIcs.CLcs.RO

TL;DR

LLM-based VLN faces a domain gap between general training corpora and navigation, alongside limited guidance for filtering visual input. NavCoT trains LLMs to imagine future surroundings, select matching observations, and reason toward actions; it outperforms direct-action variants across benchmarks and exceeds a recent GPT4-based approach on R2R.

  • Problem

    LLM-based VLN approaches commonly use offline models, which face a substantial domain gap between VLN and the LLM training corpus.

    Source: A INTRODUCTION · p. 1

  • Method

    NavCoT uses parameter-efficient in-domain training and formalized labels to train LLMs to imagine future surroundings, filter observations, and make navigation decisions through chain-of-thought reasoning.

    Source: A INTRODUCTION · pp. 1–2

  • Results

    Across R2R, R4R, RxR, and REVERIE, NavCoT significantly outperforms direct action prediction and zero-shot inference; on R2R, it surpasses a recent GPT4-based model by ∼7 points in both SR and SPL.

    Source: A INTRODUCTION · p. 2

  • Takeaways & Limitations

    The authors present parameter-efficient in-domain training as a cost-effective step toward scalable LLM-based VLN methods.

    Source: A INTRODUCTION · p. 2

Abstract

from arXiv · show

Vision-and-Language Navigation (VLN), as a crucial research problem of Embodied AI, requires an embodied agent to navigate through complex 3D environments following natural language instructions. Recent research has highlighted the promising capacity of large language models (LLMs) in VLN by improving navigational reasoning accuracy and interpretability. However, their predominant use in an offline manner usually suffers from substantial domain gap between the VLN task and the LLM training corpus. This paper introduces a novel strategy called Navigational Chain-of-Thought (NavCoT), where we fulfill parameter-efficient in-domain training to enable self-guided navigational decision, leading to a significant mitigation of the domain gap in a cost-effective manner. Specifically, at each timestep, the LLM is prompted to forecast the navigational chain-of-thought by: 1) acting as a world model to imagine the next observation according to the instruction, 2) selecting the candidate observation that best aligns with the imagination, and 3) determining the action based on the reasoning from the prior steps. Through constructing formalized labels for training, the LLM can learn to generate desired and reasonable chain-of-thought outputs for improving the action decision. Experimental results across various training settings and popular VLN benchmarks (e.g., Room-to-Room (R2R), Room-across-Room (RxR), Room-for-Room (R4R)) show the significant superiority of NavCoT over the direct action prediction variants. Through simple parameter-efficient finetuning, our NavCoT outperforms a recent GPT4-based approach with ~7% relative improvement on the R2R dataset. We believe that NavCoT will help unlock more task-adaptive and scalable LLM-based embodied agents, which are helpful for developing real-world robotics applications. Code is available at https://github.com/expectorlin/NavCoT.

A INTRODUCTION

VLN agents using LLMs face a domain gap and limited guidance for filtering noisy visual information. NavCoT trains LLMs to imagine, filter observations, and reason toward navigation actions, with reported gains over direct prediction and high-cost LLM-based approaches.

A INTRODUCTION

  • NavCoT trains the LLM as both a world model and navigation reasoning agent: it imagines future surroundings, filters observations against that imagination, and then decides an action.The paper frames this as a trainable adaptation of a mental-model process to chain-of-thought reasoning.Source: A INTRODUCTION · p. 2
  • Formalized ground-truth labels constrain reasoning to a unified format, while imagination labels are restricted to objects or scenes named in the instruction.The paper explores pretraining and finetuning, as well as full-data and low-resource settings.Source: A INTRODUCTION · p. 2
  • Across R2R, R4R, RxR, and REVERIE, NavCoT outperforms direct action prediction and zero-shot inference; on R2R, it surpasses a recent GPT4-based model by ∼7 points in both SR and SPL.The reported comparisons cover multiple VLN benchmarks and training settings.Source: A INTRODUCTION · p. 2
  • The method uses parameter-efficient in-domain training to adapt LLMs to VLN cost-effectively and support scalable LLM-based navigation.Source: A INTRODUCTION · p. 2
  • NavCoT generates explicit reasoning and is reported to be more explainable than traditional cross-modal VLN models.Source: A INTRODUCTION · p. 2

B RELATED WORK

Related work identifies domain gap, scalability, and interpretability as challenges for LLM-based VLN, while NavCoT combines in-domain training with constrained navigational reasoning. The section also outlines the navigation setup and the language-model backbones used.

B.1 Vision-Language Navigation

  • Prior VLN work includes augmentation, learning mechanisms, model architectures, and pretraining; LLM-based approaches address commonsense and interpretability needs, while NaviLLM directly predicts actions without reasoning output.NavCoT is presented as parameter-efficient in-domain training intended to improve scalability and narrow the domain gap.Source: §B.1 · pp. 2–3

B.2 LLMs for Embodied AI

  • Embodied-AI methods have used offline LLMs or in-domain adaptation; this work constructs navigational chain-of-thought labels from VLN training data for lower-cost adaptation.Source: §B.2 · p. 3

B.3 Chain-of-Thought Prompting

  • Standard chain-of-thought prompting elicits multi-step reasoning, but most variants prompt LLMs to produce it offline and without constraints; NavCoT adds world-model reasoning and unified-format ground truths.The paper says this enables self-guided navigational reasoning and simplifies training.Source: §B.3 · p. 3

C.1 Problem Setup

  • At each timestep, the agent follows instruction I using a panoramic observation of K single-view observations, and chooses among navigable views or stop; success requires stopping within 3 m of the target.Source: §C.1 · p. 3

C.2 Large Language Models (LLMs)

  • The navigation backbones are open-source LLaMA-Adapter and LLaMA 2; the latter uses bias tuning for parameter-efficient fine-tuning.The paper describes LLaMA-Adapter as an efficient LLaMA 1 adaptation method.Source: §C.2 · p. 3

D METHOD

NavCoT converts observations into textual navigation inputs, then trains an LLM to generate structured reasoning that guides action decisions. Its reasoning labels connect instruction-guided imagination, matching observations, and ground-truth actions.

D.1 Vision-to-Text System

  • The vision-to-text system describes visual and directional information in text, concatenates these into observation descriptions, and labels views as action options.The directions include six basic directions such as “turn left” and “go up.”Source: §D.1 · pp. 3–4

D.2 Navigational Chain-of-Thought Prompt

  • NavCoT prompts the LLM to imagine an instruction-guided next object or scene, select the observation matching that imagination, then predict an action.The prompt includes an example and navigation input consisting of the instruction, textual observations, and history.Source: §D.2 · p. 4

D.3 Reasoning Ground-Truth Collection

  • For imagination labels, an LLM extracts instruction-mentioned landmarks, and CLIP selects the landmark most similar to the ground-truth observation.The intended imagination corresponds to an object or scene in the following observation and is drawn from the instruction.Source: §D.3 · p. 5
  • The ground-truth chain-of-thought aligns the filtered observation and action with the ground-truth action option.The label format contains an imagination, a matching filtered observation, and an action.Source: §D.3 · p. 5

D.4 In-domain Chain-of-Thought Training

NavCoT uses in-domain pretraining and finetuning to teach an LLM to generate complete navigational chain-of-thoughts for sequential action decisions.

D.4 In-domain Chain-of-Thought Training

  • Pretraining treats the three reasoning procedures as separate tasks, then imitation-learning finetuning trains the LLM to produce complete navigation reasoning.The finetuning data reformulates expert trajectories using the constructed chain-of-thought labels.Source: §D.4 · p. 5
  • The pretraining objective uses textual observations and navigation history to predict the ground-truth outputs for imagination, observation filtering, and action prediction.The outputs are denoted U, V, and a, with corresponding ground-truth outputs U*, V*, and a*.Source: §D.4 · p. 5
  • After in-domain training, the LLM generates constrained, disentangled navigational reasoning intended to guide its action decisions.The trained model is prompted with the in-context example and prompt described for the reasoning format.Source: §D.4 · p. 5

E.1 Experimental Setup

The evaluation covers four VLN benchmarks and reports task-specific navigation and instruction-following metrics. The setup also describes the computing configuration and a reduced validation subset for ablations.

E.1 Experimental Setup

  • The evaluation uses R2R, RxR, REVERIE, and R4R, which differ in instruction complexity, instruction level, and trajectory construction.RxR evaluation uses its English subset; REVERIE uses high-level instructions, while R4R joins adjacent R2R trajectories.Source: §E.1 · p. 5
  • R2R and REVERIE use trajectory length, navigation error, success rate, SPL, and oracle success rate; RxR and R4R also use CLS, nDTW, and SDTW.Success rate counts stopping within three meters of the target viewpoint.Source: §E.1 · pp. 5–6
  • Figure 3 presents zero-shot LLM failure cases and marks the ground-truth actions with red boxes.The examples include confusing visual-filter reasoning and an incorrect “bed” imagination that leads to a wrong action.Source: Figure 3 · p. 6
  • Training uses 4 V100 GPUs for about 10 hours, while ablations use 90 instruction-trajectory pairs from eight validation-unseen scans.Inference runs on one V100 GPU.Source: §E.1 · p. 6

E.2.1 Comparison with Existing Approaches

On R2R, NavCoT with LLaMA 2 outperforms the GPT-4-based NavGPT across nearly all reported metrics, with larger gains after adding a small amount of augmentation data. The paper presents trainable reasoning as a scalable way to adapt affordable open-source LLMs to VLN.

Comparison with Existing Approaches

  • Against NavGPT, NavCoT with LLaMA 2 improves SPL by 4.17%, SR by 2.4%, and OSR by 1.98%.With small-scale augmentation data, its improvements are 7.64% in SPL, 6.23% in SR, and 6.11% in OSR.Source: §E.2.1 · p. 6
  • NavCoT with LLaMA-Adapter has a large performance gap with NavGPT, while NavCoT with LLaMA 2 outperforms NavGPT in almost all metrics.The authors relate the adapter result to the challenge of adapting a smaller-capacity LLM to VLN.Source: §E.2.1 · p. 6
  • The authors frame trainable reasoning as a scalable solution for mitigating the domain gap between LLM training corpora and VLN.They identify extending NavCoT to larger vision-language models as a future direction.Source: §E.2.1 · p. 7

E.2.2 Ablation Study

The ablations compare training settings, reasoning tasks, prompts, backbones, and an imagination-matching variant. They report gains from explicit reasoning and in-domain training, with the complete reasoning-task combination performing best across the two training settings.

Training settings

  • NavCoT surpasses Direct Action Prediction in all training settings; training-based settings outperform zero-shot, and finetuning contributes more to SR and SPL gains than pretraining.In-domain pretraining and finetuning also improve OSR, which the authors associate with selecting actions relevant to the target position.Source: §E.2.2 · p. 7

Reasoning tasks

  • AP+FI+VIF achieves the best performance in both pretraining-only and pretraining-plus-finetuning settings, with FI and VIF each reported as indispensable.The table covers proxy-task ablations on the R2R Val Unseen subset.Source: §E.2.2 · p. 7, Table 2 · p. 7

Chain-of-thought prompts

  • NavCoT, benefiting from training with formalized labels, outperforms GPT-4’s zero-shot variant and original CoT by approximately 43 and 36 SPL points, respectively.Source: §E.2.2 · p. 7

Backbones

  • NavCoT with LLaMA 2 outperforms its LLaMA-Adapter alternative, while in-domain training improves both backbones; LLaMA 2 also outperforms GPT-4 in the zero-shot NavCoT setting.The authors suggest constrained, information-light prompts may make GPT-4’s advantage over LLaMA 2 less apparent.Source: Table 3 · p. 7, §E.2.2 · p. 8

Imagination and CLIP similarity

  • LLM & CLIP* outperforms several language-model baselines, but NavCoT with LLaMA 2 surpasses LLM & CLIP* by a large margin.The authors note that navigation decisions rely on landmarks, history, and direction, among other factors.Source: §E.2.2 · p. 8

E.2.3 Generalization on Other Datasets

The paper evaluates NavCoT on RxR’s longer instructions and REVERIE’s high-level instructions, comparing it with Direct Action Prediction and other methods. NavCoT outperforms DAP on both datasets, with reported instruction-following gains on RxR.

Generalization on Other Datasets

  • The generalization experiments target RxR’s longer instructions and REVERIE’s high-level instructions.The authors use Direct Action Prediction as a language-only baseline and also compare against HAMT, a pretrained cross-modal backbone.Source: §E.2.3 · p. 8
  • On RxR, NavCoT significantly outperforms DAP, especially on CLS and nDTW, and the authors report improved instruction following for long-horizon navigation.NavCoT still has a performance gap with cross-modal methods.Source: §E.2.3 · p. 8
  • On REVERIE, methods show a significant performance drop relative to R2R, but NavCoT remains substantially better than DAP.The authors relate the drop to the challenge of navigating under high-level instructions with limited information.Source: §E.2.3 · p. 8

E.2.4 Low-Resource Experiments

In low-resource experiments on R2R and R4R, NavCoT substantially outperforms Direct Action Prediction and narrows its performance gap with HAMT. The R4R results also indicate gains in instruction-following metrics for longer trajectories.

Experimental setting

  • The R4R low-resource experiment uses 5,000 training trajectories from 50 scans and 500 validation trajectories from 11 scans.For R2R, the authors randomly extract 3,000 training trajectories from 61 scans.Source: §E.2.4 · p. 8

Low-resource results

  • In the low-resource setting, NavCoT substantially outperforms DAP on both R2R and R4R, while its CLS gap with HAMT is reduced to approximately 8.The authors compare methods using the same training strategy.Source: §E.2.4 · p. 8
  • On R4R, NavCoT significantly surpasses DAP in OSR, CLS, and nDTW, supporting instruction-following ability on long trajectories as well as correct action decisions.Source: §E.2.4 · p. 8

E.3 Visualization

The visualizations report high imagination-label and landmark-extraction accuracy, illustrate NavCoT’s action reasoning and its comparison with other CoT methods, and show examples of generalization to special cases. The conclusion notes a limitation from information loss in vision-to-text transformation.

E.3 Visualization

  • ∼73% CLIP accuracy on 200 R2R trajectories is reported for the imagination ground-truth collection process.The authors state that a small proportion of noisy imagination labels is tolerable because decisions use comprehensive information.Source: §E.3 · p. 9
  • Landmark extraction success rates are 100% for R2R and 98% for both REVERIE and RxR; R4R is consistent with R2R.The authors report evaluation on 100 randomly selected instructions per dataset and say extraction is robust to different instruction styles.Source: §E.3 · p. 9
  • NavCoT’s action-decision examples show it matching imagined landmarks to observations, including an office inferred from an observation containing a wooden desk.The authors note that observations do not always contain the imagination because of the domain gap between NavCoT and the vision-to-language model.Source: §E.3 · pp. 9–10
  • Compared with three zero-shot CoT methods, NavCoT produces less redundant, more formalized reasoning and uses imagination to guide correct action selection.The examples contrast NavCoT’s choice of option B containing stairs with a zero-shot NavCoT variant that chooses option D.Source: §E.3 · p. 10
  • In special cases, NavCoT uses direction information when instructions mention few landmarks and generates reasonable imaginations for repeated landmarks.The authors attribute the latter ability to training data that naturally contain continuous steps pointing to the same landmark.Source: §E.3 · pp. 10–11

F CONCLUSION

  • The conclusion reports superiority over a high-cost LLM-based VLN approach and direct action prediction variants, while noting that vision-to-text detail loss can impair decisions.The authors identify applying NavCoT to powerful large vision-language models as future work.Source: F CONCLUSION · p. 11

Related papers

Loading 2403.07376v2…