Source-linked AI summary
Intention Aware Robot Crowd Navigation with Attention-Based Interaction Graph
Shuijing Liu, Peixin Chang, Zhe Huang, Neeloy Chakraborty, Kaiwen Hong, Weihang Liang, D. Livingston McPherson, Junyi Geng, Katherine Driggs-Campbell
TL;DR
Crowd navigation is challenging because human goals and preferred walking styles are unobservable, and prior methods may omit future intentions or heterogeneous interactions. The paper uses a graph-based, attention-equipped model-free RL policy with trajectory predictions to represent interactions and penalize intrusions into intended paths. Its experiments report navigation improvements from prediction and human–human attention, with larger prediction gains in randomized scenarios.
Problem
In dense, interactive crowds, prior methods may omit human–human interactions or future intentions, while pedestrian goals and preferred walking styles are not observable to the robot.
Source: §I · p. 1Method
The method uses a spatio-temporal graph with separate attention for robot–human and human–human interactions, and incorporates multi-step trajectory predictions into model-free RL through a reward that discourages intrusions.
Source: §I · pp. 1–2, C. Network Architecture · p. 4Results
Around 20% higher success rate with randomized scenarios and around 12% without randomization were reported for prediction-aided models; they also had at least 7% lower ITR and 0.02 m larger SD.
Source: 2) Intention awareness · p. 5Takeaways & Limitations
The authors conclude that prediction-aided planning enables proactive, long-sighted decisions, especially in challenging crowd-navigation tasks.
Source: 2) Intention awareness · p. 6Simulated human behaviors are far from realistic, and the method assumes humans never react to the robot.
Source: §VI · p. 6
Abstract
from arXiv · showhide
We study the problem of safe and intention-aware robot navigation in dense and interactive crowds. Most previous reinforcement learning (RL) based methods fail to consider different types of interactions among all agents or ignore the intentions of people, which results in performance degradation. To learn a safe and efficient robot policy, we propose a novel recurrent graph neural network with attention mechanisms to capture heterogeneous interactions among agents through space and time. To encourage longsighted robot behaviors, we infer the intentions of dynamic agents by predicting their future trajectories for several timesteps. The predictions are incorporated into a model-free RL framework to prevent the robot from intruding into the intended paths of other agents. We demonstrate that our method enables the robot to achieve good navigation performance and non-invasiveness in challenging crowd navigation scenarios. We successfully transfer the policy learned in simulation to a real-world TurtleBot 2i. Our code and videos are available at https://sites.google.com/view/intention-aware-crowdnav/home.
I. INTRODUCTION
Crowd navigation is difficult because pedestrian goals and walking styles are unobservable, while prior methods may omit future intentions and human–human interactions. The paper proposes a spatio-temporal interaction graph with separate attention for robot–human and human–human interactions, plus predicted trajectories to guide safer, more socially aware navigation.
I. INTRODUCTION
- Without explicitly predicting human trajectories, earlier methods may produce unsafe or shortsighted behavior; prediction-based alternatives are often limited to one-step prediction, discrete actions, or few intentions.The figure contrasts safety circles centered on current pedestrian positions with the proposed circles centered on predicted future positions.Source: §I · p. 1, Figure 1 · p. 1
- Prior methods that omit human–human interactions can lose performance in dense, highly interactive crowds, where robot–human and human–human interactions affect decisions differently.The authors also note that some earlier approaches hardcode human–human attention scores or fail to distinguish the two interaction types.Source: §I · p. 1
- The method models robot–human and human–human interactions separately with attention, and uses predicted intentions in model-free RL to guide the robot away from current and intended human positions.The reward design is described as compatible with any pedestrian trajectory predictor.Source: §I · pp. 1–2
A. Reaction-based methods
The related work identifies limitations in reaction-based, learning-based, and trajectory-prediction approaches, then motivates separately learned attention for robot–human and human–human interactions. The methodology section introduces the reinforcement-learning formulation, prediction reward, interaction graph, and derived network architecture.
A. Reaction-based methods
- ORCA and Social Force use reaction rules but require carefully tuned parameters and can fail when reciprocal avoidance assumptions do not hold.ORCA assumes agents avoid each other under a reciprocal rule; Social Force represents interactions with attractive and repulsive forces.Source: A. Reaction-based methods · p. 2
B. Learning-based methods
- Deep V-Learning and DS-RNN learn interaction rules, but both use only past and current human states and therefore can produce shortsighted robot behavior.Deep V-Learning also inherits issues from its ORCA expert, while DS-RNN uses model-free RL without assumptions on state transitions.Source: B. Learning-based methods · p. 2
C. Trajectory prediction-based methods
- Trajectory-prediction methods face trade-offs: tree search can grow exponentially with action-space size, while one-step observations capture instantaneous velocity rather than pedestrian intent.The paper instead predicts several timesteps ahead and modifies the reward; it also notes that finite, small goal sets limit some alternatives.Source: C. Trajectory prediction-based methods · p. 2
D. Attention mechanism for crowd interactions
- Prior attention approaches either hardcode human–human scores or fail to distinguish interaction types; this method learns robot–human and human–human attention separately.The cited related work notes that interactions among humans can influence the robot without being explicitly modeled.Source: D. Attention mechanism for crowd… · p. 2
III. METHODOLOGY
- The methodology formulates crowd navigation as reinforcement learning, introduces a prediction reward, and derives a network from a spatio-temporal interaction graph.The section proceeds from the RL formulation and prediction reward to the graph representation and network architecture.Source: §III · p. 2
A. Preliminaries
The preliminaries represent crowd navigation as an MDP in which the robot acts under a policy while people move according to unknown transition probabilities. The state includes the robot and observed humans, with human history and predicted positions represented over time.
A. Preliminaries
- The MDP state combines the robot state with observed humans’ previous and predicted positions, and the number of observed humans can vary across timesteps.Source: 1) MDP formulation · pp. 2–3
- The network diagram represents robot–human and human–human interactions with separate attention mechanisms and connects adjacent timesteps with a GRU.Source: Figure 2 · p. 3
- At each timestep the robot selects an action under policy π, receives a reward, and transitions under unknown dynamics; observed humans also move under unknown transition probabilities.An episode ends when the robot reaches its goal, exceeds the maximum episode length, or collides with a human.Source: 1) MDP formulation · p. 3
2) Trajectory predictor:
A trajectory predictor maps observed human histories to future trajectories, whose predicted positions approximate intended paths. The reward penalizes intrusions into those predicted zones and also guides goal approach while assigning a collision penalty.
2) Trajectory predictor:
- The predictor uses observed human trajectories from t−M through t to predict future trajectories from t+1 through t+K.A circle centered at each predicted position approximates a human’s intended path; a learnable predictor is pretrained and used only for inference.Source: 2) Trajectory predictor · p. 3
3) Reward function:
- The prediction reward penalizes collisions with predicted human positions, with less penalty for intrusions farther into the future.The collision penalty is rc = −20, and intrusions are weighted differently across prediction timesteps.Source: 3) Reward function · p. 3
- The reward includes a potential-based term that guides the robot toward the goal using the change in distance to it.The goal distance is the L2 distance between the robot and goal positions.Source: 3) Reward function · p. 3
- The reward assigns +10 on reaching a goal state, a collision penalty in failure states, and otherwise combines the potential and prediction rewards.Source: 3) Reward function · p. 3
B. Spatio-Temporal Interaction Graph
The sti-graph represents visible agents and their spatial interactions at each timestep, separating human-human from robot-human edges. It links graphs over time and shares parameters by edge type as the crowd grows.
B. Spatio-Temporal Interaction Graph
- The sti-graph connects visible agents with pairwise edges, separated into human-human and robot-human interactions with indirect and direct effects on robot decisions, respectively.Humans outside the robot’s field of view are excluded because their motions cannot be predicted or integrated into planning.Source: B. Spatio-Temporal Interaction Graph · p. 3
- A temporal function connects graphs at adjacent timesteps, integrating temporal correlations to support long-term rather than reactive robot decision-making.The node and edge sets may change as agent movements change visibility.Source: B. Spatio-Temporal Interaction Graph · pp. 3–4
- Sharing parameters among edges of the same type keeps the number of parameters constant as the number of humans increases.Source: B. Spatio-Temporal Interaction Graph · p. 4
C. Network Architecture
The network derives from the sti-graph, using separate attention modules for human-human and robot-human interactions and a recurrent unit for temporal processing.
C. Network Architecture
- The architecture represents human-human and robot-human interaction functions as feedforward attention networks and the temporal function as a GRU.A trajectory predictor predicts humans’ personal zones.Source: C. Network Architecture · p. 4
1) Attention mechanisms:
- Attention modules weight each edge connected to an agent so the network can focus on important interactions.The attention computes scores from queries and keys and applies normalized scores to values, following scaled dot-product attention.Source: 1) Attention mechanisms · p. 4
- For human-human attention, current and predicted future human states are concatenated and linearly transformed into queries.The multi-head scaled dot-product attention uses eight heads; d is the query and key dimension.Source: 1) Attention mechanisms · p. 4
HHW Q
The robot-human attention stage reweights human features using the robot state and human-human attention output; visibility masks exclude invisible people.
HHW Q
- Robot-human queries are computed from the robot state and human-human weighted features, then used to reweight those human features again.The robot-human attention uses one head.Source: HHW V · p. 4
- Binary visibility masks prevent attention to invisible humans and provide unbiased gradients compared with filling invisible humans with dummy values.The authors report that the masks stabilize and accelerate training.Source: HHW V · p. 4
- The robot state is embedded with linear layers before being combined with the twice-weighted human features for recurrent processing.Source: 2) GRU · p. 4
2) GRU:
The GRU processes the robot-state embedding alongside twice-weighted human features, and its hidden state feeds the policy and value outputs. Training separates trajectory prediction from policy learning and uses PPO.
2) GRU:
- The GRU receives concatenated robot-state and twice-weighted human features, and its hidden state feeds the value and policy outputs.Source: 2) GRU · p. 4
3) Training:
- The trajectory predictor is trained separately and frozen during RL because joint training is unstable and less efficient given the tasks’ different objectives.During RL training, the predictor is run only for inference.Source: 3) Training · p. 4
- PPO trains the policy and value function, using 16 parallel environments; each policy update uses 30 steps of six episodes.Source: 3) Training · p. 4
IV. SIMULATION EXPERIMENTS
The simulations test robot navigation in dense crowds under randomized and fixed human behaviors, then compare prediction variants and established navigation methods. The setup uses limited sensing and continuous-velocity actions.
A. Simulation environment
- The 12 m × 12 m simulation uses a 5 m sensor range and crowds of up to 20 humans.The paper presents the environment as denser and more realistic than prior setups that assume unlimited detection range.Source: A. Simulation environment · p. 4
- Agents use holonomic actions specified as desired x- and y-velocities, assumed immediately achievable and held for the next timestep.The robot's continuous action space has a maximum speed of 1 m/s.Source: A. Simulation environment · p. 4
- Randomized episodes vary human goals, speeds, and radii; humans follow ORCA and react to one another but not to the robot.A second environment uses fixed human speeds and radii with no random goal changes; the invisible setting discourages learning a policy that forces humans to yield.Source: A. Simulation environment · pp. 4–5
1) Baselines and Ablation Models:
- Prediction ablations compare no predictor, constant velocity, GST, and an oracle, while broader baselines include ORCA, SF, and DS-RNN.The prediction models forecast five timesteps; the GST ablation without human-human attention replaces that attention with fully connected layers.Source: 1) Baselines and Ablation Models · p. 5
2) Training and Evaluation:
In simulation, prediction-aided methods generally improve navigation and social metrics, while human-human attention improves results over its ablation. The reported comparisons also show that predictor advantages depend on crowd-motion complexity.
2) Training and Evaluation:
- RL methods are trained for 2 × 10^7 timesteps and tested on 500 unseen cases using navigation and social metrics.ITR measures time colliding with humans' ground-truth future positions; SD is the closest-human distance during intrusions.Source: 2) Training and Evaluation · p. 5
1) Effectiveness of HH attention:
- GST with human-human attention has 12% higher success rates, lower ITR, and longer SD than the GST ablation without that attention.Without prediction, GST with human-human attention also outperforms ORCA, SF, and DS-RNN in most cases, especially in the more challenging randomized environment.Source: 1) Effectiveness of HH attention · p. 5
2) Intention awareness:
- Predictions improve success rate by around 20% in randomized scenarios and around 12% without randomization, with at least 7% lower ITR and 0.02 m larger SD.Navigation time and path length are longer because the reward penalizes short but impolite paths.Source: 2) Intention awareness · p. 5, Table II · p. 5
- In the illustrated episode, prediction-aided robots keep social distance from future human paths, unlike the more aggressive methods without prediction.The figure marks predicted trajectories in orange and ground-truth future trajectories and personal zones in gray.Source: Figure 3 · p. 6, 2) Intention awareness · p. 6
- GST with attention performs closest to the oracle across metrics, followed by constant-velocity prediction, but predictor choice depends on crowd-motion complexity.The margins between GST and constant velocity are small, especially without randomized humans, whose ORCA-controlled motion tends to be linear when no other agents are nearby.Source: 2) Intention awareness · p. 6
V. REAL-WORLD EXPERIMENTS
The simulator-trained policy was tested with real humans on a TurtleBot 2i, achieving an 83.33% success rate across 18 trials. The authors report promising results in simulation and the real world, while identifying unrealistic simulated behavior and non-reactive humans as limitations.
V. REAL-WORLD EXPERIMENTS
- The simulator-trained model was deployed on a TurtleBot 2i, using a T265 tracking camera and an RPLIDAR-A3 scanner with a 2D LIDAR people detector.The scanner was used to remove non-human obstacles from a map before human positions were estimated.Source: §V · p. 6
- 83.33% success across 18 trials in a 5m × 5m indoor space with 1 to 4 humans; the robot interacted proactively despite perception noise and robot dynamics.Humans walked slowly to compensate for the robot’s maximum speed of 0.5m/s.Source: §V · p. 6
- Wall collisions in failures resulted from excessive turning angles, indicating that environmental constraints need to be incorporated in future work.Source: §V · p. 6
VI. CONCLUSION AND FUTURE WORK
- The authors identify unrealistic simulated human behavior and the assumption that humans never react to the robot as limitations; future work needs better pedestrian models or real-trajectory data and mutual influence modeling.They describe reasoning about past and future spatio-temporal interactions as essential toward smooth human-robot interaction.Source: §VI · p. 6