PaperlayerView exact revision on arXiv

Source-linked AI summary

Robot Navigation in Crowds by Graph Convolutional Networks with Attention Learned from Human Gaze

Yuying Chen, Congcong Liu, Ming Liu, Bertram E. Shi

arXiv:1909.10400v1cs.ROcs.AIcs.CV

TL;DR

Dense crowds make safe and efficient mobile-robot navigation challenging, while existing deep reinforcement-learning methods deteriorate as crowd size grows. The paper combines a graph-based reinforcement-learning policy with human-gaze-trained attention, outperforming state-of-the-art methods on real-world dense pedestrian data.

  • Problem

    Safe and efficient mobile-robot navigation remains challenging in dense crowds, and existing deep reinforcement-learning methods deteriorate as crowd size grows.

  • Method

    The method represents crowd states with graphs and incorporates attention weights learned by a graph convolutional network trained on human gaze data into graph-based reinforcement learning.

  • Results

    The proposed method achieves 18.4% higher success rate and 16.4% shorter navigation time than state-of-the-art methods on real-world dense pedestrian data.

  • Takeaways & Limitations

    Graph-based feature aggregation and learned human attention together improve crowd-navigation performance while supporting varying crowd sizes and interpretable agent weighting.

Abstract

from arXiv · show

Safe and efficient crowd navigation for mobile robot is a crucial yet challenging task. Previous work has shown the power of deep reinforcement learning frameworks to train efficient policies. However, their performance deteriorates when the crowd size grows. We suggest that this can be addressed by enabling the network to identify and pay attention to the humans in the crowd that are most critical to navigation. We propose a novel network utilizing a graph representation to learn the policy. We first train a graph convolutional network based on human gaze data that accurately predicts human attention to different agents in the crowd. Then we incorporate the learned attention into a graph-based reinforcement learning architecture. The proposed attention mechanism enables the assignment of meaningful weightings to the neighbors of the robot, and has the additional benefit of interpretability. Experiments on real-world dense pedestrian datasets with various crowd sizes demonstrate that our model outperforms state-of-art methods by 18.4% in task accomplishment and by 16.4% in time efficiency.

I. INTRODUCTION

Crowd navigation remains difficult as density grows and conventional approaches may ignore human behavior or struggle with varying crowd sizes. The paper proposes graph-based reinforcement learning with human-gaze-derived attention to represent and prioritize agents.

  • Motivation: Dense-crowd mobile-robot navigation must be both safe and efficient but remains challenging in real environments.Applications include hospitals, shopping malls, and canteens.
  • Motivation: Traditional methods can produce unnatural or shortsighted behavior, while separate prediction and planning may cause freezing as crowd density increases.These approaches either simplify pedestrians as dynamic obstacles or model prediction and planning as separate stages.
  • Motivation: Deep reinforcement learning learns efficient interaction policies, but existing models compress variable-sized crowds into fixed-dimensional representations.The supplied passage identifies varying crowd size as a challenge for current models.
  • Approach: The proposed method represents robot-crowd interactions as a graph whose nodes and edges vary over time, with attention modulating GCN aggregation.The graph captures relations among agents, while the adjacency matrix controls feature processing and aggregation.
  • Approach: A supervised two-layer GCN learns human attention from navigation gaze data, and the learned weights are incorporated into a second GCN-based reinforcement-learning architecture.A deep neural network then estimates the corresponding value function.
  • Contributions: The architecture handles varying neighbor counts naturally and is reported to outperform state-of-the-art methods on real-world pedestrian trajectories.The authors identify interpretability and extensibility as additional properties of the approach.

II. RELATED WORK

Prior crowd-navigation research includes model-based avoidance, imitation learning, and deep reinforcement learning, while graph learning provides a flexible representation for variable agent interactions. This work combines graph-based aggregation with gaze-derived attention to weight crowd interactions.

  • Robot navigation in crowds: Model-based navigation methods require scenario-specific parameter tuning and may produce unnatural behavior because they do not fully capture human behavior.Examples include Social Force Model, RVO, and ORCA-based approaches.
  • Robot navigation in crowds: Imitation learning derives navigation policies from human demonstrations, including raw 2D laser, depth, and inverse-reinforcement-learning inputs.These methods model navigation behavior through supervised or maximum-entropy approaches.
  • Robot navigation in crowds: Deep reinforcement learning learns policies through trial and error, using either raw sensors or agent-level environmental representations.Agent-level representations can encode pedestrian intent more richly than raw sensor information.
  • Robot navigation in crowds: Distance-based ordering is not always reasonable because a nearby follower may matter less than a farther pedestrian ahead of the robot.This motivates learning relative importance rather than assigning it solely by distance.
  • Graph representation learning: GCNs aggregate node features over graph neighborhoods, and weighted adjacency matrices support adaptive, dynamic aggregation.Graph attention variants assign different importances to nodes or interactions.
  • Graph representation learning: This paper uses human-gaze-derived attention weights to determine the adjacency matrix and modulate interactions in crowd graphs.The approach applies attention to graph-based crowd representation rather than relying only on generic self-attention.

C. Human attention in visuomotor learning

The paper uses graph representations to model robot–crowd interactions and incorporates learned human attention into reinforcement learning. This design assigns weighted importance to neighboring agents while aggregating crowd information through graph convolutions.

  • Attention-based weighting: Human gaze has improved policy learning in prior visuomotor tasks, motivating its use for robot navigation.The paper incorporates attention into deep reinforcement learning rather than treating gaze only as an additional input.
  • Graph representation: Graph convolution represents robot–human interactions through nodes and adjacency-matrix connections.The adjacency matrix encodes connection strength and separates robot–human attention, robot visibility, and crowd-interaction areas.
  • Graph representation: Graph representations handle varying and dense crowds by allowing the robot node to aggregate human features naturally.This avoids a separate sequential aggregation step for filtering or combining human features.
  • Graph representation: The graph convolutional layer generates node embeddings by aggregating information from related nodes.The adjacency matrix defines how features are combined, with normalized rows and ReLU activation.
  • Attention-based weighting: Learned human attention modulates the red adjacency-matrix area that represents the robot’s importance weighting of humans and itself.The method, G-GCNRL, is compared with distance-weighted D-GCNRL and uniform-weight U-GCNRL baselines.

B. Graph-based deep V-learning

The graph-based V-learning architecture encodes robot and human states as a crowd graph, extracts crowd features at the robot node, and uses them in value-based action selection.

  • Crowd feature extraction: The crowd graph connects robot and human nodes so graph convolutions can aggregate their interaction features.The topology is expressed in robot-centric coordinates and uses graph edges to connect the agents.
  • Crowd feature extraction: Robot and human node states include positions, velocities, radii, distances, and robot-centric motion information.The robot state additionally includes goal distance, preferred velocity, moving direction, footprint radius, and velocities.
  • Value learning: The graph network extracts the output features of the robot node as crowd features for value estimation.These crowd features are concatenated with robot status and passed to the value network for V-learning.
  • Value learning: The adjacency matrix captures how node states influence neighboring nodes and can use attention-derived weights rather than fixed binary connections.The attention weights are trained to mimic human gaze estimates, unlike self-attention in a standard graph attention network.
  • Value learning: The value network is initialized through ORCA imitation learning and then trained further with reinforcement learning.During reinforcement learning, actions are sampled, the highest-valued action is selected, and state-value pairs are stored in a replay buffer.

C. GCN to predict human attention

A separate GCN predicts attention weights for robot, goal, and human nodes from their graph-structured states. These weights are supervised by human gaze data and used to configure the navigation graph.

  • Attention network design: The attention network models attention prediction as node regression over robot, goal, and human nodes.Its star topology connects the robot to humans and the goal, while each node receives an attention weight.
  • Attention network design: Attention-network inputs represent node positions and velocities in robot-centric coordinates.The coordinates are explicitly defined relative to the robot’s frame.
  • Attention network design: The attention network uses two GCN layers with output sizes of 128 and 1 to produce one weight per node.Its outputs are passed through softmax so the attention weights sum to one.
  • Integration with V-learning: The learned attention network supplies the adjacency matrix used by the graph-based value network.The resulting framework connects the attention network’s lower branch with graph-based V-learning through the policy adjacency matrix.
  • Training data: Supervised training uses gaze collected from people steering through reproduced real-world crowd scenes toward a goal.The training data contain state–attention pairs gathered across varying crowd sizes and densities.

IV. EXPERIMENTAL RESULTS

The experiments evaluate the attention network and the graph-based reinforcement-learning model in dynamic crowd scenes. A test scene shows that learned attention concentrates on pedestrians relevant to the robot’s avoidance and following behavior.

  • Evaluation: The experiments evaluate both the attention network and the G-GCNRL model in dynamic crowd-navigation settings.The authors direct readers to accompanying video material because of the experiment’s dynamic nature.
  • Attention interpretation: Learned attention weights are largest for pedestrians near the human operator’s gaze trajectory.In the illustrated scene, those pedestrians include people the robot avoids and the person it follows.

A. Human attention modelling

The learned attention network identifies humans relevant to navigation by matching predicted weights to gaze-derived ground truth, outperforming distance-based and self-attention alternatives.

  • A. Human attention modelling: The learned attention weights were largest for humans near the operator’s gaze, corresponding to pedestrians avoided or followed by the human operator.In the illustrated scene, two humans blocked the robot’s path, while another moving right was followed.
  • A. Human attention modelling: The attention network’s predicted weights closely matched the ground-truth weights in quantitative evaluation.Similarity was evaluated using KL divergence and Correlation Coefficient, where smaller KL and larger CC indicate better similarity.
  • A. Human attention modelling: 50.5% smaller KL divergence and 21.3% larger CC were achieved versus distance-related weights.Distance-based weights decay with the distance between each human and the robot.
  • A. Human attention modelling: 46.7% smaller KL divergence and 17.5% larger CC were achieved versus self-attention weights.The self-attention baseline used weights obtained from SARL’s self-attention module.

B. Graph represented reinforcement learning with attention

The evaluation compares the proposed graph-based model with SARL and uses ablations to isolate the contributions of graph structure and gaze-learned attention.

  • B. Graph represented reinforcement learning with attention: The experiments compare G-GCNRL with state-of-the-art SARL and conduct ablations of graph structure and learned attention.Tests used environments from Fig. 6 with randomly selected starting frames and robot routes between black-triangle positions.
  • B. Graph represented reinforcement learning with attention: SA-GCNRL isolates the graph contribution by using a GCN for value estimation while replacing learned gaze attention with SARL self-attention.The model was implemented to demonstrate the advantages of using the GCN alone.

1) Comparing with SARL:

Compared with SARL, graph-based aggregation improves navigation, and adding gaze-learned attention yields further gains in success rate and navigation time.

  • 1) Comparing with SARL:: 7.9% higher success rate and 6.0% shorter navigation time were achieved by SA-GCNRL than SARL.This comparison isolates the benefit of graph-based value estimation while retaining SARL self-attention.
  • 1) Comparing with SARL:: 9.7% higher success rate and 11.1% shorter navigation time were achieved by G-GCNRL than SA-GCNRL.This comparison measures the additional effect of attention weights learned from human gaze data.
  • 1) Comparing with SARL:: 18.4% higher success rate and 16.4% shorter navigation time were achieved by combining graph-based feature aggregation with learned human attention.The combined result is reported as the overall improvement over the compared approach.
  • 1) Comparing with SARL:: In the simple environment, G-GCNRL reached the goal in 12.4 s rather than SARL’s 14 s.The G-GCNRL trajectory was more constant, while SARL nearly stopped between 6 and 8 seconds.

2) Ablation study for learned human attention:

The ablation compares gaze-learned attention with distance-based and uniform weighting, showing higher success rates but no shorter successful-navigation time for G-GCNRL.

  • 2) Ablation study for learned human attention:: G-GCNRL uses gaze-learned attention, whereas D-GCNRL uses distance-based weights and U-GCNRL assigns equal weights to all nodes.The ablation is designed to evaluate the benefit of learned human attention.
  • 2) Ablation study for learned human attention:: Fig. 7 displays robot and human trajectories for a simple Zara2 environment and a hard NYC-GC environment, with timestamps marked along the paths.The robot trajectory is highlighted in green, while other colors represent human trajectories.
  • 2) Ablation study for learned human attention:: 10.5% higher success rate than D-GCNRL and 5.0% higher success rate than U-GCNRL were achieved by G-GCNRL.The passage reports that the success-rate improvement was significant.
  • 2) Ablation study for learned human attention:: G-GCNRL did not achieve shorter navigation time when the goal was reached, while U-GCNRL generally achieved the shortest navigation time.Thus, the learned attention benefit in this ablation was reported for success rate rather than successful-navigation time.

3) Ablation study for graph structure:

The ablation results show that graph structure improves crowd-state estimation, while attention further improves performance, especially in dense crowds. The learned-attention graph model achieves the highest success rate across environments.

  • Graph structure: 23.7% higher success rate and 23.2% shorter navigation time were achieved by U-GCNRL than UARL.This comparison uses equal attention weights in both models, isolating the benefit of graph structure.
  • Graph structure with attention: 7.9% higher success rate and 6.0% shorter navigation time were achieved by SA-GCNRL than SARL.SA-GCNRL uses a graph structure with attention weights determined by SARL.
  • Crowd density: In the most complex environment, averaging 30 humans per frame, uniform-attention models performed much worse than self-attention or learned-human-attention models.Performance differences were smaller in low-density crowds, where tracking all humans is relatively easy.
  • Overall result: G-GCNRL achieved the highest success rate in all environments.The result supports combining graph-based crowd representation with learned human attention.
Loading 1909.10400v1…