PaperlayerView exact revision on arXiv

Source-linked AI summary

Building Generalizable Agents with a Realistic and Rich 3D Environment

Yi Wu, Yuxin Wu, Georgia Gkioxari, Yuandong Tian

arXiv:1801.02209v2cs.LGcs.AI

TL;DR

Agents trained for specific environments can struggle to navigate unseen 3D scenes with varied appearances and layouts. The paper introduces House3D and RoomNav, combining rich scene diversity with pixel-, object-, and scene-level augmentation. On unseen environments, the final agent outperforms baselines by over 8% in navigation success rate.

  • Problem

    Agents trained on specific environments and tasks often fail to generalize to unseen environments with low- and high-level visual and structural variation.

  • Method

    The paper builds House3D from 45K diverse labeled houses and evaluates gated reinforcement-learning agents on RoomNav using multiple augmentation levels.

  • Results

    Over 8% higher navigation success rate on unseen environments is achieved relative to raw-RGB baseline methods.

  • Takeaways & Limitations

    House3D and its augmentation techniques support studying and building reinforcement-learning agents that generalize across diverse unseen scenes.

  • Takeaways & Limitations

    Domain randomization requires substantially more training samples, while segmentation-mask augmentation assumes access to segmentation output from a vision subsystem.

Abstract

from arXiv · show

Teaching an agent to navigate in an unseen 3D environment is a challenging task, even in the event of simulated environments. To generalize to unseen environments, an agent needs to be robust to low-level variations (e.g. color, texture, object changes), and also high-level variations (e.g. layout changes of the environment). To improve overall generalization, all types of variations in the environment have to be taken under consideration via different level of data augmentation steps. To this end, we propose House3D, a rich, extensible and efficient environment that contains 45,622 human-designed 3D scenes of visually realistic houses, ranging from single-room studios to multi-storied houses, equipped with a diverse set of fully labeled 3D objects, textures and scene layouts, based on the SUNCG dataset (Song et.al.). The diversity in House3D opens the door towards scene-level augmentation, while the label-rich nature of House3D enables us to inject pixel- & task-level augmentations such as domain randomization (Toubin et. al.) and multi-task training. Using a subset of houses in House3D, we show that reinforcement learning agents trained with an enhancement of different levels of augmentations perform much better in unseen environments than our baselines with raw RGB input by over 8% in terms of navigation success rate. House3D is publicly available at http://github.com/facebookresearch/House3D.

1 INTRODUCTION

Reinforcement-learning agents often specialize in their training environments, while existing generalization techniques use simplified settings. House3D addresses this gap with diverse, labeled indoor scenes and RoomNav, where augmented agents achieve higher success on unseen houses.

  • Generalizability is a prominent reinforcement-learning challenge because agents trained on specific environments and tasks often fail on new environments.
  • Existing approaches vary pixels, environment parameters, or rewards, but commonly study simplified environments lacking real-world diversity and perception challenges.
  • House3D converts 45K human-designed SUNCG houses into an efficient, extensible environment with diverse layouts, objects, and visual appearances.
  • RoomNav asks agents to navigate from random house locations to rooms specified by high-level semantic concepts.
  • 35.8% success on 50 unseen environments exceeds the baseline’s 25.7% by 10%.

2 RELATED WORK

Prior work spans generalization, navigation, language grounding, and interactive environments, but often evaluates simplified tasks or limited scene variation. House3D extends this direction with richer environments and multimodal inputs for semantic generalization.

  • Prior simulated environments often target one intelligence aspect, whereas House3D supports more comprehensive problems through diverse interactive scenes.
  • 3D Navigation: Navigation research includes SLAM-based planning, end-to-end pixel policies, mapping, and reinforcement learning with auxiliary tasks or spatial memory.
  • 3D Navigation: Recent 3D-navigation studies often test generalization only on pixel variations or small mazes, rather than richer unseen scenes.
  • Gated Modules: Gated attention encodes a high-level instruction as an embedding that modulates the agent’s visual signal for RoomNav.
  • Generalization: House3D uses visually and structurally diverse environments and reports improved generalization with depth and segmentation masks added to raw visual input.

3 HOUSE3D: AN EXTENSIBLE ENVIRONMENT OF 45K 3D HOUSES

House3D is a realistic, extensible platform built from a large, richly annotated collection of indoor scenes. Its renderer and flexible interaction model provide diverse modalities and high-throughput simulation for reinforcement learning.

  • House3D provides thousands of realistic indoor scenes and supports navigation, visual understanding, language grounding, and concept learning.
  • DATASET: SUNCG contributes 45,622 human-designed scenes, over 20 room types, over 80 object categories, and millions of annotated rooms and object instances.
  • DATASET: Scene annotations map 3D locations to object instances or free space and room types, enabling detailed spatial and semantic representations.
  • OBSERVATIONS: Agents receive RGB, semantic or instance segmentation, and depth signals, with additional support for occupancy maps, connectivity, and shortest paths.
  • INTERACTION: House3D permits movement throughout scenes subject to collision constraints and exposes a flexible API for adding more complex interactions.

4 ROOMNAV: A BENCHMARK TASK FOR CONCEPT-DRIVEN NAVIGATION

RoomNav benchmarks concept-driven navigation: agents must interpret semantic goals, explore unfamiliar houses, and reach target rooms under navigation constraints. The task uses disjoint house splits, multimodal observations, fixed or continuous actions, and shaped rewards.

  • RoomNav asks agents to reach a room specified by a semantic concept while generalizing across unseen layouts and furniture arrangements.
  • TASK: The benchmark defines goals as “go to X,” where X is a predefined room or object type interpreted across visually distinct scenes.
  • ENVIRONMENT STATISTICS: Experiments use 270 houses split into 20 small-training, 200 large-training, and 50 test houses for generalization evaluation.
  • TASK FORMULATION: Each episode randomly places the agent in an environment, supplies a semantic concept, and requires actions based on its visual history and goal.
  • TASK FORMULATION: Policies train on environments disjoint from those used for evaluation, formalizing unseen-environment generalization.
  • OBSERVATIONS: Agents can use raw pixels, segmentation masks, and depth, with concepts represented as one-hot vectors.
  • ACTION SPACE: The action space contains 12 discrete rotation and movement actions, with continuous velocities also explored; collisions leave the agent stationary.
  • SUCCESS MEASURE AND REWARD FUNCTION: Success requires entering the target room and consecutively seeing an associated object category, while shortest-distance shaping supplements sparse rewards.

5 GATED-ATTENTION NETWORKS FOR MULTI-TARGET LEARNING

RoomNav is modeled as multi-target learning, using gated-CNN and gated-LSTM policies to condition navigation on visual state and target concepts across continuous and discrete actions.

  • Multi-Target Learning: RoomNav policies condition on both the input state and target concept, using gated-CNN for continuous actions and gated-LSTM for discrete actions.The models are trained with DDPG and A3C, respectively.
  • State Encoding: The state encoder stacks recent frames, extracts image features, embeds the target, and fuses both representations into a final state encoding.The fusion module produces hs = M(x, y|θ).
  • Gated-Attention for Feature Fusion: Gated fusion transforms the target embedding, applies sigmoid gating elementwise to visual features, and produces an attention-like representation.The operation is M(x, y|θ) = x ⊙sigmoid(fmlp(y|θ)).
  • Gated-CNN for Continuous Policy: The continuous policy applies an MLP and softmax to the state representation, then uses Gumbel-Softmax to introduce stochasticity for exploration and robustness.Its policy is µ(st|θ) = Gumbel-Softmax(fmlp(hs|θ)).
  • Deep Deterministic Policy Gradient: The Q-function fuses visual features with the action vector before an MLP produces Q(s, a), while DDPG balances policy and critic losses.The shared actor-critic loss is LDDPG(θ) = −Lµ(θ) + αDDPGLQ(θ).
  • Gated-LSTM for Discrete Policy: The discrete-action model applies gated fusion over time, feeds target-conditioned representations to an LSTM, and uses separate MLPs for policy and value outputs.CNN parameters are shared across time.

6 EXPERIMENTAL RESULTS

Experiments evaluate RoomNav generalization across input modalities, model architectures, and augmentation levels using held-out houses. Diverse training environments, semantic signals, and complementary augmentations consistently improve test success and reduce overfitting.

  • Experimental Setup: Test success rates are measured on held-out houses, with Esmall containing 20 training houses, Elarge 200, and Etest 50.Agents fail after 100 steps; evaluation uses fixed random seeds and thousands of episodes per setting.
  • Baselines: RGB-trained models on Esmall overfit strongly, with test performance far below training performance and gated-LSTM underperforming concat-LSTM on test.The models learn spurious color patterns that do not transfer to unseen environments.
  • Baselines: Depth information improves performance in both training and test, motivating RGB+Depth and Mask+Depth inputs in subsequent experiments.Segmentation masks provide semantic information, while depth supplies additional scene structure.
  • Augmentation: Pixel-level, task-level, and scene-level augmentations are complementary, with their combination producing the best test performance.Domain randomization or segmentation masks address visual variation, auxiliary targets reduce overfitting, and diverse environments improve scene-level generalization.
  • Scene-Level Augmentation: Training on Elarge largely resolves overfitting, narrows training–test gaps, and generally favors LSTM models over CNN models because of their higher capacity.The larger and more semantically diverse environment improves generalization across model types and augmentation settings.
  • Overall Results: The best result comes from gated-attention models with semantic signals, exceeding RGB channels and Esmall-trained counterparts by over 8% in generalization.Segmentation masks can approximate RGB with domain randomization on Esmall while requiring fewer training samples, although domain randomization supports RGB-only operation.

7 CONCLUSION

The paper introduces House3D and RoomNav to study semantic navigation and generalization in diverse unseen 3D houses. Training with multiple augmentation levels yields substantially higher unseen-environment performance than baseline methods.

  • House3D contains 45K houses with diverse objects and natural layouts resembling the real world.
  • RoomNav requires agents to understand semantic concepts, interpret visual input, navigate to targets, and succeed in unseen environments.
  • Over 8% higher performance on unseen environments is achieved than baseline methods through domain randomization, multi-target training, and environmental diversity.

A ROOMNAV TASK DETAILS

RoomNav specifies agent state, movement and rotation actions, collision handling, and reward components for semantic navigation in House3D.

  • RoomNav represents the agent with 3D position and viewing rotation, while fixing height and leaving three degrees of freedom.
  • Actions update horizontal position and viewing rotation, with collisions preventing movement and velocity bounded by |δx|, |δy| ≤0.5 and |δρ| ≤30.
  • Continuous actions encode movement and rotation as probability distributions that produce bounded position and angle changes.
  • Discrete control provides 12 movement or rotation actions, without allowing movement and rotation simultaneously.
  • Rewards combine shortest-distance shaping with obstacle penalties, success reward +10, and a time penalty outside the target room.

B.1 NETWORK ARCHITECTURES

The network architectures combine visual convolutional processing with semantic concept embeddings through gated fusion, using either CNN or recurrent LSTM policy designs.

  • The CNN module applies batch normalization and ReLU activations, with concept instruction embeddings of dimension 25.
  • Both architectures use four convolutional layers and transform concept embeddings for gated fusion with visual features.
  • Gated-CNN: Gated-CNN uses a 512-unit fully connected layer, while its policy and Q-function use multilayer perceptrons.
  • Gated-LSTM: Gated-LSTM uses a 256-unit visual representation and a 256-dimensional LSTM module for recurrent processing.
  • Gated-LSTM: The Gated-LSTM policy and value networks use separate multilayer perceptrons with hidden layers sized 128, 64, and 32.

B.2 TRAINING PARAMETERS

Training uses normalized inputs, regularization, discounting, and separate implementation settings for DDPG and A3C across House3D environments.

  • Inputs are normalized to [0, 1], with weight decay of 10^-5 and discounted factor γ = 0.95 used throughout training.
  • DDPG: DDPG stacks k = 5 frames, trains with batch size 128, updates every 10 time steps, and uses a replay buffer of 7 × 10^5.
  • DDPG: DDPG training runs for 80000 episodes across settings, with entropy coefficients varying by environment and input signal.
  • A3C: A3C uses reward clipping to [−1, 1], learning rate 1e −3, batch size 64, and back-propagation through 30-step unrolls.
  • A3C: A3C training uses 120 processes on Esmall and 200 on Elarge, with gradient clipping and adaptive learning-rate decay.

B.3 GENERALIZATION OVER DIFFERENT CONCEPTS

Across the five navigation concepts, dining room and living room are easiest, while bathroom is hardest; auxiliary room-type prediction does not improve performance. Successful trained agents also reach targets after reasonable exploration rather than only starting nearby.

  • Concept difficulty: Dining room and living room are easiest across concepts, whereas bathroom is hardest for both gated-CNN and gated-LSTM models.The paper attributes this pattern to larger spaces and better connectivity for dining and living rooms, versus smaller, harder-to-find bathrooms.
  • Auxiliary supervision: Auxiliary prediction of the current room type does not improve either training or test performance.The authors believe reward shaping already provides strong supervision signals.
  • Navigation efficiency: Successful trained agents can explore the environment and reach the target after reasonable numbers of steps, unlike random agents that succeed mainly when spawned nearby.The comparison uses averaged steps toward the target over successful episodes.
Loading 1801.02209v2…