PaperlayerView exact revision on arXiv

Source-linked AI summary

A Recurrent Vision-and-Language BERT for Navigation

Yicong Hong, Qi Wu, Yuankai Qi, Cristian Rodriguez-Opazo, Stephen Gould

arXiv:2011.13922v2cs.CV

TL;DR

VLN poses a challenge for V&L BERT because navigation is partially observable and requires history-dependent attention and decisions, while long episodes make full self-attention costly. The paper introduces recurrent V&L BERT, which maintains cross-modal state within the Transformer and achieves state-of-the-art results on R2R and REVERIE, including navigation and referring-expression tasks.

  • Problem

    V&L BERT has had limited application to VLN because VLN requires history-dependent processing in a partially observable decision process and long episodes make repeated self-attention memory-intensive.

  • Method

    The model adds recurrence to a pre-trained V&L BERT architecture, maintaining cross-modal state without an explicit memory buffer or external recurrent module.

  • Results

    The model achieves state-of-the-art results on R2R and REVERIE, including 8% absolute R2R Success Rate improvement, 57% R2R SPL, 23.99% REVERIE navigation SPL, and 13.51% Remote Grounding SPL.

  • Takeaways & Limitations

    Recurrent V&L BERT can serve as a navigator network itself and support navigation with remote referring expression multi-task learning.

  • Takeaways & Limitations

    The paper applies recurrent BERT only to VLN and presents its application to other sequential-interaction tasks as future work.

Abstract

from arXiv · show

Accuracy of many visiolinguistic tasks has benefited significantly from the application of vision-and-language(V&L) BERT. However, its application for the task of vision-and-language navigation (VLN) remains limited. One reason for this is the difficulty adapting the BERT architecture to the partially observable Markov decision process present in VLN, requiring history-dependent attention and decision making. In this paper we propose a recurrent BERT model that is time-aware for use in VLN. Specifically, we equip the BERT model with a recurrent function that maintains cross-modal state information for the agent. Through extensive experiments on R2R and REVERIE we demonstrate that our model can replace more complex encoder-decoder models to achieve state-of-the-art results. Moreover, our approach can be generalised to other transformer-based architectures, supports pre-training, and is capable of solving navigation and referring expression tasks simultaneously.

1. Introduction

The paper adapts V&L BERT to VLN by adding recurrence for history-dependent decisions in a partially observable environment, while reducing the computational cost of long navigation episodes. Experiments on R2R and REVERIE report state-of-the-art performance.

  • Motivation: VLN is a partially observable Markov decision process in which observations depend on the agent’s current state and action.At each step, the agent must track navigation progress and localize the relevant part of the instruction.
  • Motivation: Applying V&L BERT to VLN is difficult because long episodes make repeated self-attention over visual and textual sequences memory-intensive.The paper identifies excessive GPU memory use as a key computational challenge.
  • Approach: The proposed recurrent VLN BERT adds a recurrent function to V&L BERT so its state representations capture history without an explicit memory buffer or external LSTM.Self-attention is restricted so language tokens serve as keys and values, but not queries, during navigation.
  • Evaluation: The model is initialized from pre-trained V&L BERT and evaluated on the R2R and REVERIE datasets.The datasets differ in visual clues, instructions, and goals.
  • Results: 8% absolute Success Rate improvement and 57% Success weighted by Path Length are reported on the R2R test split.For REVERIE remote referring expressions, the model obtains 23.99% navigation SPL and 13.51% Remote Grounding SPL.

2. Related Work

Related work spans VLN methods, visual-linguistic pre-training, and unified multi-task V&L models. The paper positions recurrent V&L BERT as a single network that performs navigation and remote referring expression tasks.

  • Vision-and-Language Navigation: VLN research includes instruction-following, dialog-based, remote-object localization, continuous-environment, and multilingual navigation settings.R2R and Touchdown introduced natural-language guidance in photo-realistic environments.
  • Vision-and-Language Navigation: Existing VLN methods improve visual-textual correspondence through cross-modal attention, data augmentation, self-correction, adversarial paths, and auxiliary losses.These methods target relevant observations, instruction grounding, correction, generalization, or comprehensive representations.
  • Visual BERT Pre-Training: Visual BERT models pre-train on image/video-text pairs using either two-stream or one-stream Transformer architectures.Two-stream models encode modalities separately before fusion, whereas one-stream models directly perform inter-modal grounding.
  • Visual BERT Pre-Training: The proposed recurrent BERT can augment various V&L BERT architectures with a recurrent function and serve as a navigator network by itself.This differs from approaches that use pre-trained V&L BERT only for language encoding or instruction-path compatibility.
  • V&L Multi-Task Learning: A single VLN BERT network addresses navigation and remote referring expression tasks seamlessly in REVERIE.This connects the model to unified V&L multi-task learning.

3. Proposed Model

The proposed model turns a one-stream V&L BERT into a recurrent navigator by carrying a state token across time, combining language with new visual observations, and refining state using cross-modal and decision information.

  • 3.1. VLN Background: VLN requires an agent to map an instruction and sequential observations to actions that move between states until it stops at the target.The state includes navigational history and the current viewpoint, heading, and elevation.
  • 3.2. Revisit BERT: BERT is a multi-layer Transformer that uses multi-head self-attention and feed-forward processing to encode bidirectional language representations.V&L BERT extends this architecture by concatenating language and visual tokens and pre-training on image-text data.
  • Vision Processing: At each step, OSCAR receives the previous state token, language tokens, scene-view tokens, and, in REVERIE, object tokens.Visual features are projected into the BERT token space before being concatenated with state and language features.
  • Language Processing: At initialization, the [CLS] feature summarizes the instruction as the agent’s initial state representation.During navigation, language tokens provide keys and values while state and visual tokens attend to the input sequence.
  • State Representation: The state summarizes accumulated textual, visual, and decision information and is updated autoregressively as the leading token for the next step.This recurrence avoids an explicit memory buffer or additional recurrent network.
  • State Refinement: State refinement weights textual and visual features, applies cross-modal matching, and incorporates the selected action’s directional features into the new state.In REVERIE, only visual features are sent to the state because the instructions are high-level.
  • Decision Making: Action probabilities are obtained from mean visual-token attention relative to the state, while object-selection probabilities use mean attention over candidate objects.The same attention-based selection principle is used for navigation and remote referring expressions.
  • Training: Training combines reinforcement learning and imitation learning objectives, including A2C for reinforcement learning.The supplied description identifies the optimization framework but does not provide the complete objective equations.

4. Experiments

Experiments on R2R and REVERIE evaluate navigation, referring expression grounding, attention behavior, and ablations of V&L BERT components. The recurrent BERT achieves strong benchmark performance while using recurrence and selective language self-attention to reduce computational demands.

  • Main Results: 63% SR (+8%) and 57% SPL (+5%) are achieved on R2R with PREVALENT initialization, improving over the OSCAR-initialized model and previous methods.The OSCAR-initialized model performs better than previous methods across dataset splits, while PREVALENT initialization further improves performance.
  • Main Results: Validation-unseen SR improves by 11.13% for navigation and 6.36% for object grounding, indicating stronger generalization to unseen instructions and environments.On R2R, the performance gap between validation-unseen and test-unseen splits is greatly reduced relative to previous methods.
  • Main Results: 24.62% SR and 19.48% SPL for navigation, plus 12.65% RGS and 10.00% RGSPL for referring expressions, outperform the previous best on REVERIE.The same BERT-based model handles navigation and remote object grounding jointly; the previous method has higher OSR but lower SR.
  • Ablation Study: Ablations show that fine-tuning V&L BERT for navigation and using its original architecture to learn recurrence outperform using it only as text and image encoders.Further gains come from using averaged final-layer visual attention as action probabilities and adding visual-textual matching to the state representation.
  • Ablation Study: The model attends to language once at initialization rather than re-attending at every step, addressing the high memory cost of repeated self-attention during navigation.Emb-Attn, Init-Attn, and Re-Attn consume much more training memory per sample than the proposed approach under the reported setting.
  • Ablation Study: PREVALENT initialization converges faster and reaches better performance within fewer iterations than OSCAR initialization or random initialization.The training comparison evaluates models initialized from OSCAR, PREVALENT, and random weights.

5. Conclusion

The paper introduces recurrence into V&L BERT so it can process time-dependent inputs in partially observable navigation while benefiting from generic pre-trained V&L knowledge. Applied as the navigator network, it achieves state-of-the-art performance on R2R and REVERIE and supports navigation–referring-expression multitasking.

  • Recurrence enables V&L BERT to recognise time-dependent inputs in partially observable Markov decision processes.
  • The model’s training-time comparison includes randomly initialised network parameters as the “no init.” condition.
  • The recurrent model allows downstream VLN learning to benefit from pre-trained generic V&L knowledge.
  • BERT itself serves as the navigator network and achieves state-of-the-art performance on R2R and REVERIE.
  • The recurrent BERT is capable of solving VLN and referring expression tasks simultaneously.

A. Datasets

The evaluation uses two distinct VLN datasets with different environments, instructions, goals, and visual grounding requirements. R2R tests instruction-following navigation, while REVERIE combines navigation with target-object identification.

  • The study evaluates the proposed model on two distinct VLN datasets: R2R and REVERIE.
  • R2R requires navigation through photo-realistic Matterport3D environments to reach targets using low-level natural-language instructions.
  • R2R contains 61 training scenes, 11 validation scenes, and 18 test scenes in unseen environments.
  • REVERIE requires navigation to a viewpoint where a target object is visible, followed by identifying that object from candidate objects.
  • REVERIE provides high-level navigation instructions alongside specific object-grounding instructions across 4,140 target objects in 489 categories.

B. Implementation Details

The implementation represents navigable visual observations with direction-aware scene features and, for REVERIE, position- and direction-aware object features. Navigation decisions include movement or stopping in R2R, while REVERIE links stopping to object-grounding confidence.

  • Visual Features: Experiments use Matterport3D Simulator panoramas containing 36 single-view images across 12 headings and 3 elevation angles.
  • Visual Features: Scene features use navigable-direction grid features from ResNet-152 pre-trained on Places365, concatenated with directional encodings.
  • Visual Features: Directional encodings represent each image’s heading and elevation relative to the agent’s orientation.
  • Visual Features: REVERIE object features come from Faster-RCNN trained on Visual Genome and encode object position and viewing direction.
  • Decision Making: In R2R, the agent moves to the navigable direction with the largest attention score or stops when the stop representation receives the largest score.
  • Decision Making: In REVERIE, stopping occurs when an object token scores above all scene tokens, and the highest-scoring object becomes the grounded target.

B.3. Adaptation to PREVALENT (§3.3 & §3.5)

The PREVALENT adaptation uses recurrent state representations within the LXMERT-like architecture, combining prior state, encoded language, and new visual observations. Cross-modal state matching improves performance in unseen environments, while the recurrent design supplies navigation decisions without a downstream network.

  • State Adaptation: The PREVALENT adaptation initializes the agent state from the [CLS] feature produced by TRM-Lang1 encoding the instruction.
  • State Adaptation: At each navigation step, previous state, encoded language, and new visual observations produce updated language-aware state features and decisions.
  • State Adaptation: Unlike PREVALENT fine-tuning with EnvDrop, the proposed method uses TRM-Vis1 and TRM-Vis2 directly to learn state–language–vision relationships for decisions.
  • State Refinement: Cross-modal matching weights visual and language tokens to refine the agent state, improving performance in unseen environments.
  • State Refinement: The PREVALENT-based model is evaluated with and without cross-modal matching for the agent’s state.
  • Reinforcement Learning: The critic predicts an expected value from the updated state representation using a multilayer perceptron in A2C reinforcement learning.

B.5. Reward Shaping (§3.4)

The reward design combines progress, stopping, and path-fidelity signals to encourage successful, efficient, instruction-following navigation. Ablations show path-fidelity rewards improve performance, but the recurrent BERT structure has a larger influence.

  • Progress Reward: Progress rewards encourage approaching the target, while stopping rewards distinguish successful completion from stopping at a wrong viewpoint.The agent is rewarded for stopping within 3 meters of the target and penalized for moving away or stopping incorrectly.
  • Path Fidelity Rewards: Path-fidelity rewards use normalized dynamic time warping to encourage trajectories that follow the ground-truth path rather than longer or cyclic routes.The progress reward alone does not constrain route efficiency or instruction-following accuracy.
  • Path Fidelity Rewards: A negative stopping reward penalizes agents that first approach the target and then depart from it.This reward is triggered when the previous distance is at most 1.0 meter and the distance change is positive.
  • Ablation Study: Path-fidelity rewards produce higher Success Rate, lower Trajectory Length, and higher Success weighted by Path Length in both evaluated model variants.For the OSCAR-based models, the gap between Oracle Success Rate and Success Rate is reduced by about 1.45%.
  • Ablation Study: Reward shaping contributes only a slight improvement compared with the larger influence of the recurrent BERT structure.This comparison is made against results reported in Tables 1 and 3 of the main paper.

C.2. Self-Attended Language Features with Gradient Accumulation (§4.2)

The gradient-accumulation experiment examines language self-attention methods under a common effective batch size. Larger batch-size training improves the compared methods, leaving the need for language self-attention at every step unresolved when computation is sufficient.

  • Experimental Setup: Gradient accumulation gives Emb-Attn, Init-Attn, and Re-Attn an effective batch size of 16 while keeping the learning rate unchanged.Each method uses batch size 4 and accumulates gradients for 4 iterations.
  • Results: All three language self-attention methods improve significantly on the validation unseen split under the larger effective batch size.The comparison is made against the results reported in Table 4 of the main paper.
  • Discussion: Whether language self-attention is necessary at every navigational step when computational power is sufficient remains an open question.The authors leave this investigation for future work because larger batch sizes may further improve performance, potentially including their own method.

D. Visualisation (§4.1)

The visualisations examine how recurrent BERT attends to language, state, and visual inputs during navigation. Attention shifts from broad instruction context toward relevant sub-instructions and the predicted direction as processing deepens.

  • Language Self-Attention: Selected language-attention heads show different functions across layers, with shallow layers aggregating information from the sentence and deeper layers exhibiting distinct patterns.Figure 6 visualizes row-normalized language self-attention weights at initialization.
  • Language Self-Attention: Final-layer heads aggregate information around punctuation, suggesting implicit segmentation into sub-sentences for matching observations to relevant instruction parts.This pattern is linked to the sub-instruction concept for contextual instruction grounding.
  • State/Vision Step-wise Attention: From Step 3 onward, state and different visual views are aggregated to support choosing the correct navigation direction.Figure 7 shows a trajectory together with final-layer state and visual-token attention weights.
  • State/Vision Layer-wise Attention: At Step 4, candidate views first attend broadly to the instruction, then increasingly focus on its middle portion as visual-language signals propagate through deeper layers.This progression is shown in the layer-wise attention visualization of Figure 8.
  • State/Vision Layer-wise Attention: From Layer 6 onward, visual features dominate and attention concentrates on the token for the predicted direction, becoming more confident after deeper-layer reasoning.Layers 7–9 still show uncertainty among directions spatially close to the correct one before confidence increases.
Loading 2011.13922v2…