PaperlayerView exact revision on arXiv

Source-linked AI summary

Offline Visual Representation Learning for Embodied Navigation

Karmesh Yadav, Ram Ramrakhya, Arjun Majumdar, Vincent-Pierre Berges, Sachit Kuhar, Dhruv Batra, Alexei Baevski, Oleksandr Maksymets

arXiv:2204.13226v1cs.CVcs.LG

TL;DR

Embodied agents must learn visual representations for understanding and navigating environments, yet semantic-navigation results remain limited. OVRL pretrains visual representations offline on indoor images and finetunes them for task-specific navigation. It reports improved ImageNav and ObjectNav results, with gains sustained and increasing during long ImageNav training.

  • Problem

    How to learn visual representations that help embodied agents understand environments and navigate remains an open question, while semantic-navigation results have been weaker.

    Source: §1 · pp. 1–2

  • Method

    OVRL uses DINO self-supervised learning to pretrain visual representations offline on Omnidata indoor images, then finetunes them for individual tasks in Habitat.

    Source: §1 · pp. 2–3

  • Results

    54.2% ImageNav success (up from 29.2%) and 23.2% ObjectNav success (up from 18.1%) are the reported state-of-the-art comparisons.

    Source: §1 · p. 3, §6 · p. 15

  • Takeaways & Limitations

    The same pretrained vision backbone generalizes across scene datasets, and its ImageNav gains continue to increase over 2 billion training frames.

    Source: §1 · p. 3, §5.5 · p. 15

Abstract

from arXiv · show

How should we learn visual representations for embodied agents that must see and move? The status quo is tabula rasa in vivo, i.e. learning visual representations from scratch while also learning to move, potentially augmented with auxiliary tasks (e.g. predicting the action taken between two successive observations). In this paper, we show that an alternative 2-stage strategy is far more effective: (1) offline pretraining of visual representations with self-supervised learning (SSL) using large-scale pre-rendered images of indoor environments (Omnidata), and (2) online finetuning of visuomotor representations on specific tasks with image augmentations under long learning schedules. We call this method Offline Visual Representation Learning (OVRL). We conduct large-scale experiments - on 3 different 3D datasets (Gibson, HM3D, MP3D), 2 tasks (ImageNav, ObjectNav), and 2 policy learning algorithms (RL, IL) - and find that the OVRL representations lead to significant across-the-board improvements in state of art, on ImageNav from 29.2% to 54.2% (+25% absolute, 86% relative) and on ObjectNav from 18.1% to 23.2% (+5.1% absolute, 28% relative). Importantly, both results were achieved by the same visual encoder generalizing to datasets that were not seen during pretraining. While the benefits of pretraining sometimes diminish (or entirely disappear) with long finetuning schedules, we find that OVRL's performance gains continue to increase (not decrease) as the agent is trained for 2 billion frames of experience.

1 Introduction

The paper asks how embodied agents should learn visual representations for understanding and navigating new environments, where semantic-navigation results remain weaker than other navigation results. It proposes OVRL, which pretrains visual representations offline on indoor images and finetunes them for downstream tasks, improving ImageNav and ObjectNav results.

1 Introduction

  • Embodied agents must understand environmental structure and semantics while strategically exploring and navigating to goals.The paper frames this as important both for practical applications such as home-assistant robots and for studying visuomotor inductive biases.Source: §1 · p. 1
  • Semantic-navigation results remain comparatively weak, with prior ObjectNav challenge success at 18% and 24% and best-known single-RGB ImageNav success at 29.2%.The paper contrasts these results with strong performance on other navigation tasks.Source: §1 · p. 2
  • The paper poses visual representation learning as a central question for embodied scene understanding.It motivates this question by noting that representations matter in 2D image understanding as well as embodied scene understanding.Source: §1 · p. 2
  • OVRL separates offline self-supervised visual pretraining from downstream finetuning on individual navigation tasks.Pretraining uses DINO on large-scale pre-rendered indoor images from Omnidata; downstream tasks include ImageNav and ObjectNav in Habitat.Source: §1 · pp. 2–3
  • 54.2% ImageNav success (up from 29.2%) and 23.2% ObjectNav success (up from 18.1%) mark OVRL’s reported state-of-the-art results.The ImageNav comparison is in the single-RGB Gibson setting; the ObjectNav comparison is in the RGBD plus known-pose setting.Source: §1 · p. 3

2 Related Work

Prior work combines visual representation learning with reinforcement learning, uses temporal objectives, or relies on large-scale pretrained models and auxiliary losses. OVRL differs by learning from IID images without a temporal objective and focuses on RGB representations.

Self Supervised Learning (SSL) in RL

  • OVRL learns representations from IID images without a temporal objective, unlike methods that require image sequences or extra anti-collapse losses.The passage contrasts OVRL with temporal contrastive approaches and non-contrastive temporal methods such as PBL, SPR, and SGI.Source: §2 · p. 3

Visual Navigation

  • On single-camera ImageNav, OVRL combines a simpler model architecture with higher performance than Memory-Augmented RL, and ZER’s reward and view sampling further improve OVRL.The passage also notes end-to-end and modular approaches in ObjectNav, including the Habitat-Web imitation-learning approach OVRL builds upon.Source: §2 · pp. 3–4

SSL in Embodied AI

  • OVRL pretrains on the public 14.5M-image Omnidata Starter Dataset, unlike EmbCLIP’s unreleased 400M image-caption-pair dataset.The comparison concerns the scale and availability of the pretraining data.Source: §2 · p. 4
  • OVRL outperforms Ye et al. ’s results on ObjectNav without using any auxiliary losses, leaving open possible future improvements by combining the two ideas.Source: §2 · p. 4

3 Approach

OVRL has two stages: DINO-based self-supervised pretraining of a visual encoder, followed by downstream policy learning in Habitat. The approach uses image augmentations and task-specific policy architectures for ImageNav and ObjectNav.

3 Approach

  • OVRL first pretrains an encoder with DINO and then learns downstream policies in Habitat for ImageNav and ObjectNav.The overview figure depicts the same two-stage approach, with downstream finetuning for navigation tasks.Source: §3 · p. 4, Figure 2 · p. 5

3.1 Self-Supervised Pretraining

  • DINO trains a student to match a teacher across two global and eight local augmented image views, using temperature-scaled probability distributions.Both networks produce K-dimensional features; only the student processes local views, and the teacher is updated as an exponential moving average of student parameters.Source: §3.1 · pp. 4–5
  • The pretraining setup uses a modified ResNet50 with half-width layers and Group-Norm, plus a projection head discarded after training.The projection head is a three-layer MLP with 2048 hidden units, followed by L2 normalization and a K-dimensional weight-normalized layer.Source: §3.1 · p. 5

3.2 Downstream Learning

  • Policy learning uses image augmentations, including color jitter, random rotation, random resized crop, and translation, with augmentations kept consistent over time.The passage describes differing augmentation use during RL action sampling and during forward-backward passes in RL and IL.Source: §3.2 · pp. 5–6
  • ImageNav uses pretrained observation and goal encoders with a two-layer LSTM policy trained by DD-PPO, while ObjectNav combines pretrained, finetuned RGB features with frozen depth features in an imitation-learned GRU policy.ObjectNav also supplies GPS+Compass and goal embeddings to the policy.Source: §3.2 · p. 6

4 Experiments

The experiments evaluate OVRL’s self-supervised pretraining and downstream policy learning on ImageNav and ObjectNav. They compare OVRL with scratch-trained and prior approaches across differing datasets and evaluation settings.

4.1 Self-Supervised Pretaining

  • The Omnidata Starter Dataset used for pretraining contains approximately 14.5 million rendered images from six 3D-scene datasets; Gibson test-scene images and CLEVR images were excluded.Gibson test scenes were excluded to avoid contaminating ImageNav downstream experiments; CLEVR was excluded for visual and domain dissimilarity.Source: §4.1 · p. 7

4.2 Downstream Learning: ImageNav

  • ImageNav evaluation used Gibson’s 72 training and 14 testing scenes and two episode splits, as prior results varied across settings and were not directly commensurable.Agents used either one front-facing RGB camera or four directional RGB cameras, without depth or GPS+Compass sensors.Source: §4.2 · p. 7
  • 41.3% SR and 26.9% SPL on ImageNav split A versus 17.9% SR and 9.3% SPL for scratch; OVRL also exceeds ZER by 12.1% in success rate and 5.3% in SPL.The OVRL-versus-scratch methods differ only in pretrained encoder initialization.Source: §4.2 · pp. 8–9, Table 1 · p. 9
  • On four-RGB ImageNav, OVRL exceeds Mem-Aug RL by 10.8% in success rate and 6.5% in SPL; on split B, it exceeds NRNS by 21.5% in success and 16.0% in SPL without depth or egocentric pose.The four-camera result is reported despite OVRL not using an additional memory mechanism.Source: §4.2 · p. 9

4.3 Downstream Learning: ObjectNav

  • ObjectNav evaluation used Matterport3D with 61 training, 11 validation, and 18 testing scenes; success required stopping within 1 m of the goal object and viewing it from the final position.The task supplied RGBD, GPS+Compass, and the goal category ID, with a 500-step episode limit.Source: §4.3 · p. 10
  • 23.2% success rate and 7.6% SPL on MP3D ObjectNav test-std in RGBD; OVRL exceeds DDPPO by 17% in success rate and 5.5% in SPL, and Habitat-Web by 6.3% success rate and 1.9% SPL in RGB.Against EmbCLIP in RGB, OVRL gains 3.3% in success rate but is 1.0% worse in SPL.Source: Table 2 · p. 10, §4.3 · p. 11

5 Analysis

The analysis tests pretraining data, finetuning choices, SSL algorithms, model size, and long training schedules. OVRL’s gains persist and increase over a 2-billion-step ImageNav schedule, while the ablations identify conditions that affect performance.

5.1 How much does the pretraining image dataset matter?

  • On ImageNav, Gibson-ShortestPath pretraining reaches 36.9% success, while OSD-100% is 4.4% higher; ImageNet-1k reaches 20% success, only 2.1% above Scratch.The authors attribute OSD’s advantage over Gibson-ShortestPath to Omnidata’s diverse sampling scheme.Source: §5.1 · pp. 11–12, Table 3 · p. 12
  • OSD-10%, 25%, and 100% achieve essentially the same peak performance, with roughly 60–600 images per scene sufficient for useful representations.OSD images are uniformly sampled from about 2,400 scenes; the compared subsets contain 1%, 10%, and 25% of the original dataset.Source: §5.1 · p. 12, Table 3 · p. 12
  • Using only HM3D or Taskonomy for pretraining performs similarly to the full dataset, while combining Taskonomy and HM3D gives a slight increase.The authors hypothesize that removing Replica+GSO and BlendedMVG images focused on household objects and sculptures improves ImageNav representations.Source: §5.1 · p. 12, Table 3 · p. 12

5.2 How do augmentations and finetuning impact performance?

  • With augmentations, finetuning improves success by 11.2% and SPL by 9.9%; augmentations improve both metrics only when the encoder is finetuned.With a frozen encoder, augmentations reduce success by 2.9% and SPL by 2.0%; with a finetuned encoder, they improve success by 6.0% and SPL by 3.8%.Source: Table 4 · p. 13, §5.2 · p. 13

5.3 How does the choice of SSL algorithm affect performance?

  • On ImageNav, DINO surpasses MOCO-v2 and SimCLR in success and SPL.The authors note that OVRL can be coupled with improved SSL algorithms.Source: §5.3 · p. 13, Table 5 · p. 14

5.4 Does model size matter?

  • ResNet18 reduces ImageNav success by more than 7% and SPL by 5.0%, while ResNet50-64B offers only a slight performance improvement at higher compute cost.The comparison also includes ResNet101 with 32 baseplanes.Source: Table 6 · p. 14, §5.4 · p. 14

5.5 Is OVRL effective when the agent is trained for much longer?

  • Across 2 billion simulation steps, OVRL’s advantage over Scratch continues to increase on HM3D-trained, Gibson-tested ImageNav.The authors also report a high correlation between training and testing success rates, indicating positive transfer between the datasets.Source: §5.5 · pp. 14–15, Figure 4 · p. 14, Table 7 · p. 14

6 Conclusions

The conclusion presents OVRL as a two-stage self-supervised approach that improves ImageNav and ObjectNav results. It also identifies future work on DINO components, data collection, and interactive tasks.

6 Conclusions

  • OVRL achieves 54% ImageNav success in Gibson and 23% ObjectNav success in MP3D, improving state of the art by 25% and 5.1% absolute, respectively.The reported settings are single-RGB for ImageNav and RGB/RGBD-only for ObjectNav.Source: §6 · p. 15
  • Future work includes studying DINO components, improving data collection strategies, and extending OVRL to interactive tasks such as Rearrangement.Source: §6 · p. 15

A Approach

  • In the four-camera ImageNav architecture, current and goal image features are encoded separately, then combined and processed by a two-layer, 512-dimensional LSTM.The goal-camera features are added to keep the goal representation rotation-invariant.Source: §A.1 · p. 20

B Additional Experiments

B.1 Downstream Learning: Indoor Scene Classification

This appendix evaluates frozen visual encoders on indoor scene classification using a linear classifier. OVRL pretraining outperforms the alternative pretraining methods and datasets in several reported comparisons, though supervised ImageNet pretraining has higher top-1 accuracy.

B.1 Downstream Learning: Indoor Scene Classification

  • The evaluation freezes pretrained ResNet50 encoders and trains a linear classifier on pooled features from 55 indoor Places365 categories.Baselines include ImageNav Scratch and CRL encoders, plus ImageNet-DINO and supervised ImageNet encoders.Source: §B.1 · pp. 20–21
  • OVRL-Frozen achieves 82.9% top-5 accuracy and 50.9% top-1 accuracy on Places, with 18.9% higher top-1 accuracy than the ImageNav Scratch representations.Source: §B.1 · p. 21, Table 8 · p. 21
  • OVRL-Frozen has 8.8% higher top-1 accuracy than the CRL reimplementation and 2.6% higher top-1 accuracy than ImageNet-DINO.Source: §B.1 · p. 21
  • ImageNet supervised learning achieves 3.8% higher top-1 accuracy than OVRL-Frozen, while ImageNav finetuning reduces scene-classification performance by 8%.The authors suggest that some SSL features useful for scene classification may not help ImageNav.Source: §B.1 · p. 21

B.2 Downstream Learning: ImageNav

The appendix tests ImageNet-pretrained encoders on ImageNav under conditions without finetuning and augmentations. The reported results contrast their adaptation with OSD-pretrained OVRL.

B.2 Downstream Learning: ImageNav

  • ImageNet pretraining improves over Scratch by 2.1% success and 4.6% SPL, but the authors find these representations do not adapt readily to ImageNav through simple finetuning and augmentation.Source: §B.2 · pp. 21–22, Table 9 · p. 22
  • Freezing the ImageNet-pretrained encoder improves success by 3.7% but reduces SPL by 0.6%.Source: §B.2 · p. 21, Table 9 · p. 22
  • Removing augmentations from the frozen ImageNet baseline improves success by 6.1% and SPL by 1% over ImageNet-Finetuned; by contrast, OSD-pretrained OVRL improves with finetuning and augmentations.Source: §B.2 · pp. 21–22, Table 9 · p. 22

C.1 Does the scratch baseline catch up with OVRL on Gibson scenes?

On Gibson, Scratch did not catch OVRL even after 1.5 billion training steps. OVRL’s test performance at 500 million steps was approximately double Scratch’s after its longer run.

C.1 Does the scratch baseline catch up with OVRL on Gibson scenes?

  • At 500M steps, OVRL reaches 41.3% success and 26.9% SPL on Gibson test scenes, approximately double Scratch’s 22.9% success and 13.5% SPL after 1.5B steps.The comparison is reported alongside the Gibson training-and-test curves and the table comparing the methods after 1.5 billion steps.Source: Figure 5 · p. 22, §C.1 · p. 23, Table 10 · p. 23
  • After 1.5B training steps on Gibson, OVRL reaches 95% success and 89% SPL, while Scratch reaches 70% success and 60% SPL.Scratch’s success and SPL nearly double during the 1B steps after 500M.Source: §C.1 · pp. 22–23
  • Gibson has 72 training scenes, compared with 800 for HM3D; the authors expected faster convergence on Gibson.Source: §C.1 · p. 22

C.2 How do SSL representations compare against the representations learnt with RL on a given task?

The authors use an ImageNav-learned RL encoder as a pretrained encoder for the same downstream task, eliminating dataset distribution shift. RL pretraining improves training performance over Scratch, but has a test generalization gap relative to OVRL.

C.2 How do SSL representations compare against the representations learnt with RL on a given task?

  • On Gibson test scenes, RL-Pretrained methods trail OVRL by approximately 14% in success rate and 11% in SPL.The comparison is reported for the test set; Fig. 6 plots Success/SPL over steps for Scratch, OVRL, and RL-Pretrained methods on Gibson train and test scenes.Source: §C.2 · p. 23, Figure 6 · p. 24
  • Using the Scratch agent’s ImageNav encoder for the same downstream task eliminates the possibility of dataset distribution shift.Source: §C.2 · p. 23
  • RL-Pretrained methods significantly improve Gibson training success and SPL over Scratch, reducing the training gap to OVRL to 8%.The authors also report a significant generalization gap for RL-Pretrained methods on test scenes.Source: §C.2 · p. 23, Figure 6 · p. 24

D Compute Resources

The paper reports substantial compute use for pretraining, task runs, and analysis experiments.

D Compute Resources

  • Pretraining the vision encoder on 12 million Omnidata Starter Dataset images takes 65 hours on 64 GPUs, or 173 GPU days.Source: D Compute Resources · p. 24
  • A default OVRL ImageNav run takes 40 hours on 32 GPUs, equivalent to 53 GPU days.Source: D Compute Resources · p. 24
  • ImageNav experiments in Section 4.2 use approximately 960 GPU days, ObjectNav experiments 640 GPU days, and analysis experiments over 3500 GPU days.Source: D Compute Resources · p. 24

E Places Dataset

The scene-classification experiments use 55 indoor scene classes adapted from Places365.

E Places Dataset

  • The scene-classification experiments use 55 indoor scene classes from Places365, adapted from.Source: E Places Dataset · p. 24
  • The listed classes include bedrooms, kitchens, living rooms, offices, corridors, and classrooms.Source: E Places Dataset · p. 24
  • The class list also includes specialized settings such as engine rooms, fire escapes, cockpits, and swimming pools.Source: E Places Dataset · p. 24
Loading 2204.13226v1…