Source-linked AI summary
Socially Compliant Navigation through Raw Depth Inputs with Generative Adversarial Imitation Learning
Lei Tai, Jingwei Zhang, Ming Liu, Wolfram Burgard
TL;DR
The paper addresses socially compliant navigation around pedestrians without requiring explicit pedestrian localization and velocity estimates. It combines behavior cloning with GAIL on raw depth inputs, and reports safer, efficient navigation in simulation and real-world deployment. It also releases pedestrian-simulation resources based on the social force model.
Problem
Existing socially compliant navigation methods depend on precise pedestrian location and velocity estimates, specialized sensors, and potentially time-consuming state extraction.
Method
The approach pretrains behavior cloning, then applies GAIL to raw depth inputs while incorporating temporal correlations and social-force information.
Results
Experiments report safer and more efficient behavior than pure behavior cloning, with socially compliant navigation demonstrated in simulation and real-world deployment.
Takeaways & Limitations
Raw-depth GAIL can guide mobile robots toward socially compliant behavior without relying on explicit pedestrian-state inputs.
Takeaways & Limitations
Real-world performance is influenced by the Turtlebot waffle's limited maximum speed and field of view.
Abstract
from arXiv · showhide
We present an approach for mobile robots to learn to navigate in dynamic environments with pedestrians via raw depth inputs, in a socially compliant manner. To achieve this, we adopt a generative adversarial imitation learning (GAIL) strategy, which improves upon a pre-trained behavior cloning policy. Our approach overcomes the disadvantages of previous methods, as they heavily depend on the full knowledge of the location and velocity information of nearby pedestrians, which not only requires specific sensors, but also the extraction of such state information from raw sensory input could consume much computation time. In this paper, our proposed GAIL-based model performs directly on raw depth inputs and plans in real-time. Experiments show that our GAIL-based approach greatly improves the safety and efficiency of the behavior of mobile robots from pure behavior cloning. The real-world deployment also shows that our method is capable of guiding autonomous vehicles to navigate in a socially compliant manner directly through raw depth inputs. In addition, we release a simulation plugin for modeling pedestrian behaviors based on the social force model.
I. INTRODUCTION
Dynamic pedestrian navigation requires robots to avoid collisions while following socially acceptable behaviors. Existing methods often depend on precise pedestrian state estimates and specialized sensors, motivating direct navigation from raw depth inputs.
- Dynamic environments require mobile robots to avoid collisions and respond appropriately to changing human behavior.
- Model-based approaches require scenario-specific force-parameter tuning, while learning-based approaches recover expert policies or latent costs.
- Previous approaches require precise nearby-pedestrian localization and velocity information, often restricting them to robots with high-precision sensors.
- The paper targets socially compliant indoor navigation through raw depth images.
- Estimating pedestrian states from raw sensory input is generally time-consuming, so direct learning from raw sensor inputs remains important.
2) Perception from visual input:
Visual sensing offers a lower-cost alternative to high-precision lidar, and depth inputs are chosen for better simulation-to-reality consistency. The paper combines raw-depth feature learning with GAIL-based policy improvement and releases simulation resources.
- 2) Perception from visual input:: Vision sensors are more affordable than 3D lidars, making raw-visual-input navigation more feasible for mobile agents.
- 2) Perception from visual input:: The model uses neural networks to extract useful features directly from onboard depth-camera inputs.
- 2) Perception from visual input:: Simulated depth images are preferred over RGB because they better match real-world data and ease model transfer.
- 2) Perception from visual input:: Behavior cloning is simple to deploy but ignores temporal correlations, limiting generalization to scenarios that differ substantially from training data.
- 2) Perception from visual input:: The approach first learns an initial policy with behavior cloning, then applies GAIL to incorporate temporal correlations in the demonstrations.
- 2) Perception from visual input:: The paper introduces a raw-depth GAIL approach and releases a socially compliant pedestrian-simulation plugin plus a dataset of 10,000 state-action pairs.
II. BACKGROUND
The paper formulates navigation as a Markov decision process in which an agent selects actions from states, receives rewards, and transitions through environment dynamics according to a policy.
- An MDP represents an agent interacting through sequences of observations, actions, rewards, and state transitions.
- At each time step, the agent chooses an action from its current state according to policy π and receives a reward before transitioning.
A. Imitation learning
Imitation learning derives policies from expert trajectories through behavior cloning or inverse reinforcement learning. GAIL directly matches generated and expert state-action pairs, while this work further uses WGAN-based discrimination.
- A. Imitation learning: Behavior cloning directly maps recorded states to expert actions, but covariate shift and ignored temporal correlations cause compounding error and poor generalization.
- A. Imitation learning: Inverse reinforcement learning estimates a latent reward or cost from expert demonstrations while accounting for entire trajectories, but its reinforcement-learning inner loop is expensive.
- A. Imitation learning: GAIL directly learns a policy from expert demonstrations without first learning an intermediate reward function.
- A. Imitation learning: GAIL trains a generator to match expert state-action pairs while a discriminator distinguishes generated samples from expert data.
- A. Imitation learning: The learning procedure alternates discriminator updates with TRPO updates, treating discrimination scores as costs for policy optimization.
- A. Imitation learning: The proposed framework extends GAIL with WGAN, replacing discriminator classification with regression to improve training stability.
C. Social force model
The social force model represents pedestrian acceleration as the sum of forces associated with goals, nearby pedestrians, obstacles, and fluctuations. The paper uses this model to simulate socially compliant pedestrians and releases a Gazebo plugin for that purpose.
- C. Social force model: The social force model computes pedestrian acceleration from the sum of several applied forces.These forces are described as components of the pedestrian dynamics model.
- C. Social force model: Desired force drives an agent toward its navigation goal, while social force measures nearby-pedestrian influence.The desired force can be represented using the agent pose and target pose.
- C. Social force model: Obstacle force supports collision avoidance, while fluctuation force represents environmental randomness and stochastic pedestrian behavior.The paper omits estimating obstacle and fluctuation forces to prioritize social aspects.
- C. Social force model: The released Gazebo plugin simulates pedestrians behaving according to the social force model.The plugin provides the socially compliant pedestrian simulation environment used in the paper.
III. METHODS
The method formulates pedestrian navigation from depth images and desired-force inputs, using a behavior-cloning generator refined by asynchronous GAIL across varied social scenarios. It also contributes a social-force pedestrian simulator and a 10,000-state-action-pair dataset.
- III. METHODS: Navigation is formulated as an MDP whose state combines a depth image with the force toward the desired target.The policy generator and discriminator are updated in an interleaved training procedure.
- III. METHODS: Training and evaluation cover six scenarios, including passing, overtaking, crossing, and interactions with pedestrian groups.The scenarios are divided into three relatively easy and three difficult cases.
- III. METHODS: The released dataset contains 10,000 state-action pairs with depth, RGB, social-force, desired-force, and velocity recordings.The dataset is intended for further benchmarking and uses varied social scenarios.
- III. METHODS: The generator uses depth images and desired force to predict expert actions while jointly performing social-force prediction.Shared multitask features are intended to form an effective compact representation and improve generalization.
- III. METHODS: Asynchronous GAIL initializes its policy from behavior cloning, samples trajectories from social simulations, and alternates discriminator and policy updates.The discriminator is randomly initialized, while policy optimization uses the discriminator-derived cost under TRPO.
- III. METHODS: The approach supports transfer to different motion planners because the social-force prediction component can remain unchanged across mobile platforms.The paper links this design to improved generalization and reduced overfitting.
B. Algorithm
The paper addresses limited generalization in standard GAIL by training across different social tasks rather than one fixed task. Its modified procedure combines behavior-cloning initialization, multi-scenario trajectory sampling, and adversarial policy optimization.
- B. Algorithm: Original GAIL trains one model for each specific task, while similar environment trajectories limit generalization across episodes.The paper motivates a modified training procedure inspired by asynchronous deep reinforcement learning.
- B. Algorithm: Figure 6 compares behavior-cloning trajectories in the upper panels with GAIL-optimized trajectories in the lower panels across the considered scenarios.Robot trajectories are red, pedestrian trajectories are blue, and path thickness increases from start to endpoint.
- B. Algorithm: Each environment instance in the proposed approach corresponds to a different social task during training.This contrasts with treating all sampled environments as repetitions of the same task.
- B. Algorithm: The generator is initialized with behavior-cloning weights, and the discriminator is initialized randomly before iterative training begins.This establishes the starting points for the two networks in Algorithm 1.
- B. Algorithm: The procedure samples trajectories from different social scenario simulations and feeds generated and expert trajectories to the discriminator.The discriminator distinguishes generated policies from expert policies using the sampled trajectories.
IV. EXPERIMENTS
The experiments train behavior cloning from social-force-model trajectories, then improve it with GAIL across multiple social scenarios. The resulting policy accounts for trajectory-level temporal structure and supports socially compliant planning.
- Training setup: 10,000 state-action pairs were collected from randomized social scenarios, with 2,000 samples reserved to evaluate the pre-trained model.The data included trajectories viewed from both the robot and pedestrian perspectives.
- Training setup: The GAIL training procedure samples trajectories from six randomly chosen scenarios, totaling approximately 1,000 samples per training step.The generator is initialized with behavior-cloning weights.
- Optimization: The model is trained for 300 iterations over 10 hours using a Tesla P100 GPU, with RMSprop for the discriminator and TRPO for the policy network.The discriminator learning rate is 5e−5.
- Policy behavior: Behavior cloning makes frame-wise reactive decisions, whereas GAIL uses trajectory temporal correlations to plan over a whole trajectory.The GAIL policy is initialized from the pre-trained behavior-cloning policy.
A. Evaluation in simulated environments
The simulated evaluation compares behavior cloning and GAIL on controlled Turtlebot3 episodes across social scenarios using depth images and desired forces.
- Evaluation setup: Ten test episodes are executed in each scenario under both policies, with randomness removed to ensure fair comparisons.The simulated Turtlebot3 waffle receives depth images and desired forces as inputs.
1) Qualitative evaluation:
Qualitative and quantitative evaluation shows that GAIL produces safer, more socially compliant navigation and generally efficient paths compared with behavior cloning.
- Qualitative evaluation: GAIL guides the robot farther from pedestrians in scenarios (a), (b), (c), and (f), and succeeds in group-navigation cases where behavior cloning fails.Specifically, GAIL passes between pedestrians in (d) and exits a group in (e).
- Qualitative evaluation: The small field of view limits pedestrian perception to nearby people, producing sudden turns especially in the crossing scenario.This limitation affects the observed trajectories.
- Quantitative evaluation: The evaluation uses min-dist, the episode’s minimal robot–pedestrian distance, and travel-time, the time from start to target.Both statistics are reported in Fig. 7.
- Quantitative evaluation: GAIL has larger average min-dist than behavior cloning in all six scenarios and more efficient travel-time in most scenarios, though it is slightly slower in two easy scenarios.The two metrics indicate safer and efficient path planning overall.
B. Real world experiments
The approach is deployed on a Turtlebot waffle in an indoor office using onboard raw depth images, and the paper reports validation in both simulated and real-world experiments.
- Real-world deployment: The real-world platform autonomously navigates randomly selected collision-free office targets using cropped Intel RealSense R200 depth images.A low-cost laser range sensor localizes the robot, and a Jetson TX2 supports onboard processing.
- Real-world deployment: Fig. 8 depicts indoor-office experiments on a Turtlebot waffle using raw depth images and desired forces as inputs.The figure concerns the real-world platform and its input modalities.
- Validation and releases: The approach is validated in simulated and real-world experiments, alongside a released social-force pedestrian plugin and simulation dataset.These releases support modeling pedestrian behavior and provide 10,000 socially compliant navigation state-action pairs.
- Limitations: Real-world performance is influenced by the Turtlebot waffle’s limited maximum speed and field of view.The authors identify evaluation on more compatible platforms as future work.