Zero Experience Required: Plug & Play Modular Transfer Learning for Semantic Visual Navigation

Ziad Al-Halah; Santhosh K. Ramakrishnan; Kristen Grauman

Source-linked AI summary

Zero Experience Required: Plug & Play Modular Transfer Learning for Semantic Visual Navigation

Ziad Al-Halah, Santhosh K. Ramakrishnan, Kristen Grauman

arXiv:2202.02440v2Feb 5, 2022cs.CVcs.AIcs.LG

TL;DR

Task-specific visual navigation policies require expensive, repeated interactions whenever tasks or goal modalities change. The paper introduces a modular transfer model that learns an image-goal policy, aligns other goal modalities offline, and transfers the full model to target tasks. Across diverse photorealistic environments and tasks, it reports strong state-of-the-art performance and zero-shot experience equivalent to 507 million scratch-training interactions, while retaining limitations for dissimilar or multi-goal tasks.

Problem
Task-specific navigation training requires massive interactions and repeated retraining for new tasks or goal modalities, while isolated policies miss shared structure.
Method
The model learns an image-goal semantic search policy, trains a joint goal embedding offline for other modalities, and transfers the full model plug-and-play.
Results
The approach outperforms state-of-the-art image-goal and transfer-learning methods, and its zero-shot performance on five semantic tasks equals 507 million scratch-training interactions.
Takeaways & Limitations
A single source-task policy can support multiple semantic navigation tasks with zero-shot deployment or limited target-specific fine-tuning.
Takeaways & Limitations
Transfer is not theoretically guaranteed to help, and target tasks substantially different from the source may not benefit from transferred experience.

Abstract

from arXiv · show

In reinforcement learning for visual navigation, it is common to develop a model for each new task, and train that model from scratch with task-specific interactions in 3D environments. However, this process is expensive; massive amounts of interactions are needed for the model to generalize well. Moreover, this process is repeated whenever there is a change in the task type or the goal modality. We present a unified approach to visual navigation using a novel modular transfer learning model. Our model can effectively leverage its experience from one source task and apply it to multiple target tasks (e.g., ObjectNav, RoomNav, ViewNav) with various goal modalities (e.g., image, sketch, audio, label). Furthermore, our model enables zero-shot experience learning, whereby it can solve the target tasks without receiving any task-specific interactive training. Our experiments on multiple photorealistic datasets and challenging tasks show that our approach learns faster, generalizes better, and outperforms SoTA models by a significant margin.

1. Introduction

Visual navigation spans distinct semantic tasks and goal modalities, but task-specific training requires costly repeated interactions. The paper proposes transferring one image-goal navigation policy across tasks, including zero-shot target-task operation.

1. Introduction: Task-specific navigation policies require repeated environment interactions, special-purpose priors, and substantial computation, limiting sample efficiency and scalability.Separate treatment also misses shared structure across tasks and may require manual 3D annotations.
1. Introduction: The paper challenges distinct policies by proposing that experience from one navigation task can transfer to multiple tasks and goal modalities.The motivating examples connect object finding with room finding, sketches, and sounds.
1. Introduction: The method learns a general-purpose image-goal semantic search policy, then trains a joint goal embedding offline for downstream modalities and tasks.Target tasks use either zero-shot transfer or limited target-specific fine-tuning.
1. Introduction: ZSEL removes interactive training on the new navigation task, extending zero-shot learning from supervised classes to reinforcement-learning behaviors.This setup targets lifelong learning in which deployed agents encounter novel tasks.
1. Introduction: The approach is evaluated across hundreds of environments, four navigation tasks, and five goal modalities, with ZSEL on five semantic navigation tasks equivalent to 507 million scratch-training interactions.The evaluation spans Matterport3D [13], Gibson [70], and HM3D [54].

2. Related Work

Prior visual navigation work includes task-specific mapping, learned representations, multitask learning, and meta-RL. This paper instead transfers a policy learned from one unannotated source task to new tasks without target interactions.

Visual Navigation: Semantic navigation includes PointNav, ObjectNav, RoomNav, AudioNav, and ImageNav, each pairing navigation with a distinct goal specification.Examples include coordinates, object names, rooms, sounds, and photographs.
Transfer Learning in Navigation: Transfer methods pretrain environment representations or primitive skills, often reducing downstream samples, while supervised and self-supervised approaches commonly pretrain observation encoders.These approaches motivate transferring reusable navigation knowledge.
Transfer Learning in Navigation: Multitask learning jointly trains tasks and meta-RL adapts a policy across task distributions, whereas this method learns from one task without manual annotations and transfers zero-shot.The distinction is the absence of interactive target-task training.
Zero-Shot Learning: Unlike conventional zero-shot learning for supervised classification, ZSEL transfers behaviors learned on a source navigation task to target tasks without new target interactions.The target is a set of navigation behaviors rather than a classifier for unseen classes.

3. Plug & Play Modular Transfer Learning

The model combines an image-goal semantic search policy, a joint embedding for multiple goal modalities, and plug-and-play transfer. It supports zero-shot deployment or few-shot fine-tuning while addressing semantic reward and sensor constraints.

3. Plug & Play Modular Transfer Learning: The architecture has observation, goal, and policy modules that encode observations and goals before producing navigation actions.These modules define the agent interface used for transfer.
3.1. Semantic Search Policy for Image Goals: The source policy learns image-goal navigation because image goals require no manual annotations, can be sampled freely, and encourage semantic spatial priors.The task uses an RGB goal image from an unknown pose and standard 90° field of view.
3.1. Semantic Search Policy for Image Goals: The proposed reward combines progress toward the goal with reduced angular error near the goal, plus an efficiency slack and a maximum success reward.The angle term encourages the agent to align its current view with the image goal; γ = 0.01, and success uses ds = 1 m and αs = 25°.
3.1. Semantic Search Policy for Image Goals: Randomizing the image view at the fixed goal location increases goal diversity, regularizes training, and encourages spatial semantic associations.The augmentation reduces overfitting to a single goal description and can capture nearby-object cues.
3.2. Joint Goal Embedding Learning: A joint goal embedding maps modalities such as audio, sketches, category names, images, and edgemaps into a space aligned with the image-goal encoder using offline paired data.The modality encoder is trained so similar image-goal pairs have similar embeddings, with similarity labels from annotations.
3.3. Transfer and Zero-Shot Experience Learning: Compatible modules enable plug-and-play zero-shot transfer without target interactions, while full-model fine-tuning supports faster convergence and better downstream performance.Unlike approaches transferring only the observation encoder, this method transfers the complete model.

4. Evaluation

The evaluation tests the semantic search policy on image-goal navigation and then measures modular transfer, zero-shot performance, scalability, and long-term task-expert training across diverse downstream tasks and modalities. Across these settings, the approach outperforms transfer baselines, transfers without target interactions, and reaches comparable task-expert performance substantially faster.

Image-Goal Navigation: In image-goal navigation, our model gains +6.6% Succ and +2.6% SPL over the best baseline in split A, and +9% Succ and +11.2% SPL over [32] in split B.The model uses a 90° FoV, whereas some comparison methods use more complete sensor configurations or panoramic sensing.
Transfer Learning: Our approach outperforms all transfer-learning baselines across downstream semantic navigation tasks, with success-rate gains of +5% to +14% over DD-PPO.The comparison includes self-supervised, supervised, reinforcement-learning, and task-expert baselines.
Zero-Shot Experience Learning: ZSEL outperforms the Task Expert in 4 of 5 tasks without target-task experience, while competing methods often require millions of target interactions to reach the same level.In ViewNav, no baseline reaches the model’s ZSEL performance; the text attributes this partly to the difficulty of estimating stopping distances for the goal modality.
Modular Transfer and Scalability: Combining transferred modules produces the best ObjectNav-Label performance and enables ZSEL, while downstream performance improves as source-task ImageNav experience increases.Transferring the observation encoder alone provides positive transfer, whereas transferring the goal encoder and policy alone does not improve over no transfer.
Long-Term Task Expert Training: The model reaches the best performance of the Task Expert 34.7× faster, while the Task Expert requires more than 22M steps on ObjectNav and RoomNav and up to 416M on ViewNav.Across five tasks, the Task Expert requires 507M steps to reach the model’s ZSEL performance and never reaches the model’s top performance after target-task finetuning.

5. Conclusion

The paper presents a plug-and-play modular transfer model that unifies semantic visual navigation across tasks and goal modalities. It also introduces zero-shot experience learning for new navigation tasks.

The model provides a unified transfer approach for diverse semantic visual navigation tasks with different goal modalities.It is designed as a plug-and-play modular system rather than separate task-specific policies.
Zero-shot experience learning enables the model to perform new navigation tasks without task-specific interactive training.The authors describe this as a new functionality for visual navigation and a stepping stone toward lifelong and open-world learning.

6. Supplementary Materials

The supplementary materials provide implementation and dataset details, expanded results, qualitative examples, ablations, failure cases, and discussion of societal impact and limitations.

Supplementary materials include shared implementation details and dataset descriptions for image-goal navigation, target tasks, and the goal embedding space.
They report detailed image-goal results across three episode-difficulty levels and target-task results with standard deviations.
Additional materials show visual goal modalities, qualitative results across tasks, performance curves, and sensor-configuration ablations.
The supplement includes examples of model failure cases and discussion of potential societal impact and limitations.

7. Shared Setup

All reinforcement-learning methods use a common training setup and are evaluated in photorealistic simulated environments with unseen test scenes.

The methods use random-crop and color-jitter augmentation, DD-PPO, fixed optimization settings, and equal computational resources.The shared configuration includes two PPO epochs, 128 forward steps, entropy coefficient 0.01, clipping 0.2, and end-to-end Adam optimization.
Experiments use Habitat with Gibson, Matterport3D, and HM3D scans, while test scenes remain disjoint from training scenes to measure generalization.

8. Image-Goal Navigation

Image-goal navigation is evaluated across three difficulty levels using Gibson episodes and unseen scenes. The proposed model performs better across both test splits and all difficulty levels.

The training split contains 9K episodes from 72 Gibson scenes, while test splits contain 4.2K and 3K episodes from 14 disjoint unseen scenes.Episodes are divided by geodesic goal distance into easy, medium, and hard levels.
Episodes use geodesic-distance ranges of 1.5–3 m, 3–5 m, and 5–10 m for easy, medium, and hard difficulty, respectively.The test splits also enforce minimum geodesic-to-Euclidean distance ratios to avoid trivial straight-line paths.
The supplementary materials include examples of image, sketch, and edgemap visual goals used in navigation tasks.
Better performance is reported across easy, medium, and hard episodes in both Gibson test splits A and B.Table 4 contains the detailed results, while Fig. 10A provides a qualitative example.

9. Transfer Learning to Downstream Tasks

The modular transfer approach is evaluated across downstream navigation tasks, goal modalities, sensor configurations, and transfer settings. It performs diverse tasks out of the box under zero-shot experience learning, while enhanced visual sensing improves performance and modality-specific failures remain.

Task Setup: The evaluation uses downstream episodes for ObjectNav, RoomNav, and ViewNav, with sketches, audio clips, and edgemaps included among the target goals.ObjectNav includes 24K training and 1K testing episodes, RoomNav 25K and 290, and ViewNav 24K and 1.5K; sketch and audio goals use separate train/test splits.
Goal Embedding Space: The joint goal embedding aligns labels, sketches, sounds, room labels, and edgemaps with image-goal representations using offline training while keeping the image encoder frozen.A sampled image view serves as a positive anchor for multiple target goal modalities.
Detailed Results for All Tasks: The approach performs a diverse set of downstream tasks and goal modalities effectively using plug-and-play modular transfer learning.The evaluation compares transfer methods, task experts, zero-shot experience learning, and fine-tuning across the downstream tasks.
Scalability (Sensors): Increasing vision resolution from 128 to 256 produces a larger ZSEL success-rate improvement than diversifying the sensor suite.Adding depth and pose improves performance, but the larger gain comes from the enhanced vision channel.
Failure Cases: Failures arise from ambiguous goal modalities and distance estimation, including viewpoint misestimation, texture-poor edgemaps, and early stopping.ImageNav can stop far from the photographed viewpoint, ViewNav can match a similar edge structure, and multiple tasks can stop before reaching the goal.

10. Potential Societal Impact

Semantic visual navigation could support assistive robots in daily-life tasks, but the study’s building datasets represent limited geographic and cultural settings. This creates biases in the architectures, rooms, and objects the agent encounters.

Potential Benefits: Semantic navigation could support elder-care robots with tasks such as finding keys or bringing medicine to people.The stated application domain involves agents finding semantic objects in 3D environments.
Potential Risks: The datasets contain 3D scans of western-style buildings and houses from relatively well-off areas, limiting architectural and object diversity.This may restrict the technology’s availability to a smaller section of the population.
Mitigation: More diverse datasets and robust adaptation to shifts in building layouts and object types are identified as needed to mitigate these effects.

11. Discussion and Limitations

The paper presents modular transfer learning as a way to solve multiple semantic navigation tasks zero-shot or through faster fine-tuning. Its scope remains limited to semantic navigation and transfer may not help when target tasks differ substantially from the source.

Contributions: The model handles multiple semantic navigation tasks with diverse goal modalities and can solve downstream tasks without task-specific interactive training.Fine-tuning is also reported to learn faster, generalize better, and reach higher performance than baselines.
Scope: The approach is positioned as a first step toward broader task transfer, but multi-goal tasks requiring planning remain future work.The discussion specifically identifies tasks involving a series of goals, such as VLN, as requiring additional research.
Limitations: Transfer is not theoretically guaranteed to help when target tasks differ substantially from the source task.This is stated as a general limitation of the transfer-learning setting.