Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scale

Ram Ramrakhya; Eric Undersander; Dhruv Batra; Abhishek Das

Source-linked AI summary

Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scale

Ram Ramrakhya, Eric Undersander, Dhruv Batra, Abhishek Das

arXiv:2204.03514v2Apr 7, 2022cs.AIcs.CVcs.RO

TL;DR

Embodied object-search requires agents to explore unseen environments, but prior human demonstration datasets were too small for large-scale imitation learning. The paper builds a browser-to-Mechanical-Turk teleoperation infrastructure and collects demonstrations for ObjectNav and Pick&Place. Human-demonstration IL outperforms RL on ObjectNav and achieves nonzero Pick&Place success where RL does not, supporting investment in large-scale imitation learning.

Problem
Human demonstration datasets for embodied visual-search tasks were typically too small, limiting evidence about large-scale imitation learning compared with reinforcement learning.
Method
The authors connect Habitat to Amazon Mechanical Turk through a browser-based teleoperation system and train agents with behavior cloning and inflection weighting.
Results
Across ObjectNav and Pick&Place, IL outperforms RL: 70k human demonstrations beat 240k agent trajectories on ObjectNav, while Pick&Place reaches ∼18% success versus 0%.
Takeaways & Limitations
The results provide compelling evidence for investing in large-scale imitation learning from human demonstrations.
Takeaways & Limitations
The approach uses vanilla behavior cloning with inflection weighting and remains constrained by the amount of human demonstrations that can be gathered.

Abstract

from arXiv · show

We present a large-scale study of imitating human demonstrations on tasks that require a virtual robot to search for objects in new environments -- (1) ObjectGoal Navigation (e.g. 'find & go to a chair') and (2) Pick&Place (e.g. 'find mug, pick mug, find counter, place mug on counter'). First, we develop a virtual teleoperation data-collection infrastructure -- connecting Habitat simulator running in a web browser to Amazon Mechanical Turk, allowing remote users to teleoperate virtual robots, safely and at scale. We collect 80k demonstrations for ObjectNav and 12k demonstrations for Pick&Place, which is an order of magnitude larger than existing human demonstration datasets in simulation or on real robots. Second, we attempt to answer the question -- how does large-scale imitation learning (IL) (which hasn't been hitherto possible) compare to reinforcement learning (RL) (which is the status quo)? On ObjectNav, we find that IL (with no bells or whistles) using 70k human demonstrations outperforms RL using 240k agent-gathered trajectories. The IL-trained agent demonstrates efficient object-search behavior -- it peeks into rooms, checks corners for small objects, turns in place to get a panoramic view -- none of these are exhibited as prominently by the RL agent, and to induce these behaviors via RL would require tedious reward engineering. Finally, accuracy vs. training data size plots show promising scaling behavior, suggesting that simply collecting more demonstrations is likely to advance the state of art further. On Pick&Place, the comparison is starker -- IL agents achieve ${\sim}$18% success on episodes with new object-receptacle locations when trained with 9.5k human demonstrations, while RL agents fail to get beyond 0%. Overall, our work provides compelling evidence for investing in large-scale imitation learning. Project page: https://ram81.github.io/projects/habitat-web.

1. Introduction

The paper studies learning object-search behavior from large-scale human demonstrations in unseen environments. It shows that human-trained agents can acquire exploration strategies that are difficult to engineer through reinforcement learning or shortest-path imitation.

Learned search behavior: Human demonstrations include sophisticated exploration, whereas shortest-path demonstrations omit the exploration required by object-search tasks.The paper highlights peeking into rooms, checking corners, and turning in place for panoramic views.
Large-scale data collection: 92k human demonstrations were collected for ObjectNav and Pick&Place, far exceeding prior human demonstration datasets.The collection includes 80k ObjectNav and 12k Pick&Place demonstrations.
IL versus RL: IL using 70k human demonstrations outperforms RL using 240k agent-gathered trajectories on ObjectNav.The comparison uses vanilla imitation learning without additional bells or whistles.
Scaling: Accuracy improves with more human demonstrations, indicating promising scaling behavior for large-scale imitation learning.The reported data-size trends suggest that additional demonstrations may further improve performance.
IL versus RL: On Pick&Place, IL reaches ∼18% success on new object-receptacle locations, while RL remains at 0%.The IL result uses 9.5k human demonstrations.

2. Related work

Prior embodied demonstration datasets primarily use planner-generated shortest paths or relatively small human-provided trajectories. Habitat-Web addresses this scale limitation with a web-based infrastructure for collecting human navigation and interaction demonstrations.

Existing demonstration paradigms: Prior embodied datasets commonly contain either shortest-path trajectories generated with privileged information or human-provided trajectories.The paper distinguishes planner-generated demonstrations from human navigation demonstrations.
Dataset scale: Human-provided navigation datasets are typically small, with most listed datasets containing fewer than 10k episodes.EmbodiedQA is cited as having approximately 700 human-provided episodes.
Paper contribution: Habitat-Web provides scalable, extensible collection of human navigation and interaction demonstrations in Habitat.The infrastructure supports tasks including language-based tasks situated in the Habitat simulator.
Paper contribution: The authors collected approximately 13 times more demonstrations than prior publicly available datasets.The total collection comprises 92k demonstrations.

3. Habitat-WebGL Infrastructure

Habitat-Web connects the Habitat simulator to Amazon Mechanical Turk through a browser-based interface, enabling remote users to teleoperate virtual agents and submit demonstrations at scale. The system records interaction data while supporting multiple Habitat-compatible tasks.

Infrastructure: The web-based setup connects Habitat to Amazon Mechanical Turk users for large-scale human demonstration collection.The application renders Habitat-Sim assets in the browser using WebGL.
User interface: The interface presents users with a first-person view, task instruction, keyboard controls, and a submission control.For ObjectNav, the instruction is formatted as “Find and go to the <goal_object_category>”.
Object interaction: For Pick&Place, the interface highlights graspable objects with a 3D bounding box and tracks grabbed objects in an inventory.The highlighting was introduced to improve the user experience when selecting objects.
Data recording: The backend records keyboard actions, agent state, and object states during data collection.These records support reconstruction of the demonstrated interaction sequences.
Simulation integration: The application exposes Habitat simulator functionality through Javascript bindings and executes user actions every 50ms.The interface renders at 20 frames per second and uses Habitat physics APIs.

4. Tasks and Datasets

The paper studies ObjectNav and Pick&Place, both of which require searching unseen environments without maps. Human demonstrations are longer and use more varied actions than shortest paths, reflecting the exploration and interaction demands of these tasks.

ObjectNav: ObjectNav requires navigating to an instance of a specified object category in an unseen environment without a map.The agent uses RGBD, GPS+Compass, and the goal object category ID.
ObjectNav datasets: Shortest-path ObjectNav demonstrations follow the geodesic path to the nearest navigable viewpoint and average 67 steps versus 243 for human demonstrations.More shortest paths are generated to approximately match the action count of human data.
ObjectNav metrics: ObjectNav exploration is evaluated with Occupancy Coverage and Sight Coverage, measuring visited area and visible navigable area.Coverage is computed using voxel grids and iteratively updated field-of-view projections.
Pick&Place: Pick&Place requires finding and picking up an object, finding a receptacle, and placing the object there in a new environment.The task extends object navigation by performing search twice within one episode.
Demonstration behavior: Human demonstrations are longer and use all nine actions, while shortest-path demonstrations have a less varied action distribution.Humans often use MOVE_BACKWARD to backtrack, unlike shortest-path agents by design.

5. Imitation Learning from Demonstrations

The paper trains imitation-learning policies from human demonstration trajectories using behavior cloning, inflection weighting, and CNN–RNN architectures tailored to ObjectNav and Pick&Place.

Behavior cloning learns a parameterized policy that maps observations to action distributions from a dataset of human demonstration trajectories.The objective minimizes the negative log-likelihood of demonstrated actions.
Inflection weighting upweights timesteps where consecutive actions change, improving learning over vanilla behavior cloning for long repetitive navigation sequences.The coefficient is based on the ratio of total actions to inflection points.
The base policy embeds sensory inputs with feed-forward modules, concatenates features, and uses a GRU to predict the next action distribution.RGB uses a randomly initialized ResNet18, while depth uses a PointGoal-pretrained ResNet50.
OBJECTNAV: The ObjectNav policy combines RGBD, GPS+Compass, semantic features, and the goal category before processing them with a two-layer, 512-dimensional GRU.It is trained for approximately 400M steps on approximately 70k demonstration episodes.
PICK&PLACE: The Pick&Place policy combines RGBD observations with encoded language instructions and feeds them into a two-layer, 512-dimensional GRU.It is trained for approximately 90M steps on approximately 9.5k demonstration episodes.

6. Experiments & Results

Experiments compare imitation learning with reinforcement learning across ObjectNav and Pick&Place, while also examining behavior, data efficiency, and scaling with demonstrations.

ObjectNav: 20.0% success and 6.5% SPL are achieved by a prior reward-engineered RL policy on ObjectNav.The policy separates exploration from navigation and rewards area coverage before the goal is seen.
ObjectNav: 27.8% success and 9.9% SPL are achieved by IL with 70k demonstrations on the Habitat ObjectNav Challenge TEST-STD.This outperforms prior RL-trained counterparts on the reported benchmark.
Scaling: Increasing the human-demonstration dataset from 4k to 70k consistently improves validation success without saturation.The experiments use fixed hyperparameters across dataset-size splits, suggesting further demonstrations may yield additional gains.
Sample Efficiency: IL requires 7x fewer unique experience steps than RL to outperform RL on validation success.IL trains for approximately 21 epochs on a static dataset of approximately 70k demonstrations, whereas RL gathers unique trajectories online.
Pick&Place: 17.5% success and 9.8% SPL are achieved on new object-receptacle initializations after training on 9.5k human demonstrations.RL fails to exceed 0% success on the same split, while performance continues improving with larger Pick&Place datasets.

7. Characterizing Learned Behaviors

The learned IL agents exhibit search strategies that resemble human exploration and are more prominent than in RL or shortest-path agents.

Observed Strategies: IL agents use peeking, corner checking, panoramic turns, beelines, exhaustive search, and looping back during ObjectNav.These behaviors were categorized from 300 validation episodes per method and visualized in Figure 6.
IL–RL Comparison: IL agents trained on human demonstrations achieve higher occupancy and sight coverage, peeking, panoramic turns, beelines, and exhaustive search than RL agents.RL agents spend more time in the goal room on average but show significantly higher variance across scenes.

8. Conclusion

The paper scales human-demonstration collection and finds that imitation learning outperforms reinforcement learning while acquiring sophisticated search behavior.

Main Findings: IL using 70k human demonstrations outperforms RL using 240k agent-gathered trajectories on ObjectNav.On Pick&Place, IL reaches approximately 18% success while RL remains at 0%.
Main Findings: Human demonstrations implicitly convey sophisticated object-search behavior more prominently than RL agents acquire it.The conclusion frames these results as evidence supporting large-scale imitation learning.

A.1. Pick&Place

Pick&Place requires agents to locate and manipulate an object and receptacle without being given their locations, making exploration central to task completion.

Task Definition: The agent must find the object, pick it up, find the receptacle, and place the object on it.The instruction specifies the object and receptacle identities but not their locations in a new environment.
Experimental Scope: The section studies human-demonstration statistics, IL scaling with dataset size, and an RL baseline for Pick&Place.These experiments address learning performance as the amount of training data increases.

A.1.1 Dataset Stats

Human demonstrations for PICK&PLACE are longer and use a broader, more uniform action distribution than shortest-path demonstrations.

932 vs 342 steps per demonstration: human PICK&PLACE demonstrations are substantially longer than shortest paths.Humans also use all 9 actions, whereas shortest paths use only 6.

A.1.2 RL Baseline

The RL baseline uses a CNN+RNN policy trained with rewards for success, speed, exploration, object visibility, manipulation, and dropping penalties. Despite this reward structure, it fails to solve PICK&PLACE.

RL Baseline: The RL agent is trained with sparse success, slack, exploration, object-seen, grab/release, and drop-penalty rewards.Exploration is encouraged through visitation-based coverage, with exploration decayed over time.
RL Baseline: 0% success: after 100M training steps, the PICK&PLACE policy never solves the task.It learns to pick up the object when visible but fails to search for the receptacle and place the object on it.
Generalization: 49.4% success and 16.4% SPL: human-demonstration IL transfers zero-shot from Matterport3D to Gibson without finetuning.The transferred IL agent outperforms an RGBD-plus-semantics RL baseline by 33.6% success and 11.5% SPL, but trails SemExp by 4.9% success and 3.5% SPL.
Human Demonstrations: 25.2% to 68.2%: human OBJECTNAV annotators show substantial variation in average SPL.The best annotator reaches 68.2% averaged over 6 episodes, close to shortest paths.