Source-linked AI summary
Human-Human & Human-Robot Interaction Transformer (H2INT) for Robot Navigation in Dense and Uncertain Crowds
Ao Shen, Kaixi Chen, Shiwei Liu, Fang Deng, Chen Chen
TL;DR
Safe navigation in dense crowds requires handling pedestrian behavior that changes in response to the robot and varies across individuals. H2INT keeps this interaction uncertainty in the learning environment, encodes relations hierarchically with gated Transformers and recurrent memory, and reports safer, more robust navigation across response conditions and crowd densities.
Problem
Robot navigation is challenged by pedestrian motion that can change in response to the robot, while many approaches assume uniform reciprocity or robot-independent motion.
Method
H2INT uses a behavior-level crowd formulation with latent heterogeneous responsiveness, a two-stage gated Transformer, recurrent policy memory, and curriculum training.
Results
H2INT maintains the highest success and lowest collision rates across response regimes and remains most robust as crowd density increases.
Takeaways & Limitations
The framework supports navigation under interaction uncertainty, with evidence from simulations, ablations, unseen crowd-flow transfer, and a LiDAR-based robot trial.
Abstract
from arXiv · showhide
Safe robot navigation in dense crowds requires reasoning about pedestrian motion and how it may change in response to a robot. However, many learning-based approaches generate pedestrian motion independently of the robot or assume uniform reciprocity, omitting an important source of interaction uncertainty. This paper presents a Human-Human & Human-Robot Interaction Transformer (H2INT), a reinforcement learning framework that retains robot-conditioned changes in pedestrian motion during policy learning while allowing responsiveness to vary across pedestrians. Responsiveness affects the crowd dynamics when the robot is visible but is not supplied as a policy input; the policy must instead infer its consequences from robot-centered relative positions. A two-stage gated Transformer progressively encodes human-human and human-robot relations, while a recurrent policy captures their temporal evolution. A curriculum gradually reduces pedestrian responsiveness to increase interaction difficulty. Simulation experiments demonstrate improved navigation safety and robustness over representative baselines across response conditions and crowd densities, and show transfer without retraining to structurally distinct crowd-flow layouts. Ablations support the hierarchical relational encoding and gated updates. Real-robot deployment further verifies that the learned policy can operate with sparse observations in a physical environment.
I. INTRODUCTION
H2INT addresses interaction uncertainty in crowd navigation by modeling robot-conditioned, heterogeneous pedestrian responses and inferring their consequences from partial observations. It combines a behavior-level crowd formulation with gated hierarchical relational encoding and evaluates robustness across varied navigation conditions.
- Robot motion can change pedestrian behavior, creating a partially observable feedback loop that complicates safe and efficient navigation.
- Many existing approaches treat crowds as external obstacles, assume fixed reciprocity, or generate pedestrian trajectories independently of the robot.
- H2INT models robot-conditioned and robot-independent pedestrian motion with heterogeneous, temporally persistent response tendencies that remain latent to the policy.
- A two-stage gated Transformer progressively encodes human-human and human-robot relations from relative positions, while recurrent context supports inference under partial observability.
- Training progressively reduces pedestrian cooperation, and evaluation includes distinct crowd-flow modes plus LiDAR-based robot deployment.
- The framework targets interaction feedback rather than claiming self-attention or cognitive attention estimation as novel.
III. PROBLEM FORMULATION
The problem formulation represents navigation as partially observable interaction over a variable-size pedestrian crowd. The robot observes its own state and robot-centered geometric relations, while latent heterogeneous responsiveness changes pedestrian motion without being supplied to the policy.
- A. Environment Modeling: The environment contains a mobile robot and a variable number of pedestrians in a planar, partially observable setting.
- A. Environment Modeling: The robot state is a 7-dimensional vector encoding position, radius, goal coordinates, preferred speed, and heading angle.
- A. Environment Modeling: Each pedestrian is represented by a robot-relative edge feature rather than a full dynamic state, capturing spatial configuration without uncertain motion estimates.
- A. Environment Modeling: Pedestrians receive episode-level responsiveness probabilities, and response is possible only when the robot is within their field of view and sensing range.
- A. Environment Modeling: A temporal debouncing filter requires K consecutive confirmations before changing a pedestrian’s latent binary responsiveness state.
- A. Environment Modeling: Responsiveness is a behavioral propensity rather than a calibrated cognitive-attention probability, and robot motion affects trajectories after response is enabled.
- A. Environment Modeling: When responsiveness is active, the robot enters a pedestrian’s ORCA neighbor set; otherwise, a dummy state replaces it outside the interaction range.
- A. Environment Modeling: Observations include robot velocity, robot state, and relative pedestrian positions forming a robot-centered star-shaped graph whose latent responsiveness affects transitions but is hidden from the policy.
B. Reward Function Design
The reward design encourages goal progress, collision avoidance, interpersonal distance maintenance, and efficient navigation, while PPO with GAE trains the policy under diverse crowd conditions.
- The per-timestep reward encourages goal reaching, collision avoidance, and interpersonal distance maintenance.Its main component balances these objectives.
- The reward uses distance to the goal and minimum distance to any human, with ρrobot = 0.3 as the success threshold.
- A constant time penalty is applied at every step to promote efficient navigation.
- The policy πθ(at|ot) is trained to maximize expected return using PPO with GAE.
- Latent pedestrian responsiveness, crowd density, and motion are randomized to produce a diverse policy-learning distribution.
IV. METHODOLOGY
The methodology combines relational interaction encoding with an actor-critic reinforcement-learning backbone for navigation.
- The policy combines relational interaction encoding with an actor-critic reinforcement-learning backbone.This combination is illustrated in Fig. 2.
- Relational interaction encoding supplies the policy with interaction features for navigation decisions.
- The actor-critic backbone is the reinforcement-learning component of the navigation policy.
A. Input Encoding and Embedding
The input encoder embeds robot state, velocity, and robot-centered human relations into a structured token sequence whose attention captures crowd context.
- Input Encoding and Embedding: The robot’s 7-dimensional state and measured velocity are concatenated into a 9-dimensional vector and embedded by an MLP.The resulting robot embedding is φr ∈ R^d.
- Input Encoding and Embedding: Each relative human-robot edge [∆x(i),∆y(i)] is processed by a shared MLP into a d-dimensional edge embedding.
- Input Encoding and Embedding: Dynamic masking and zero-padding support a variable number of observed pedestrians up to Nmax.
- Input Encoding and Embedding: The token sequence places the robot first and represents remaining tokens as relative human-robot relations with type and positional embeddings.
- Input Encoding and Embedding: Self-attention conditions each pedestrian representation on the surrounding crowd, providing implicit human-human context without estimated pedestrian velocities.
B. Hierarchical Gated Transformer Encoder
The hierarchical encoder progressively aggregates and refines robot-centered interaction representations through two gated Transformer stages with state-dependent residual updates.
- Hierarchical Gated Transformer Encoder: Two stacked gated Transformer stages progressively aggregate interactions in dense crowds.Both stages operate on the same robot-centered token set.
- Hierarchical Gated Transformer Encoder: The first stage constructs an initial interaction representation, while the second refines it using first-stage relational features.
- Hierarchical Gated Transformer Encoder: Gate1 regulates relational-message updates and Gate2 regulates nonlinear feature updates after attention and the FFN.Both are GRU-style residual gates.
- Hierarchical Gated Transformer Encoder: The gates are state-dependent rather than unweighted residual additions, and they differ from the policy core’s temporal GRU.
- Hierarchical Gated Transformer Encoder: The framework embeds robot and pedestrian relative spatial edges into a shared latent space before actor-critic navigation processing.
- Hierarchical Gated Transformer Encoder: The encoder’s first stage constructs an interaction representation and its second stage refines encoded relational features for robot decision making.
2) Stage-II Relational Refinement:
Stage II applies gated relational refinement to first-stage interaction tokens, producing a crowd-level representation for decision making while masking invalid or invisible agents.
- Stage-II Relational Refinement: The second gated stage refines first-stage interaction tokens into a crowd-level representation for decision making.The tokens already contain first-stage interaction information.
- Stage-II Relational Refinement: Attention masking excludes padded tokens and invisible agents from aggregation.Valid token pairs receive zero mask values, while invalid pairs receive −∞.
- Stage-II Relational Refinement: The updated robot token is extracted as the context-aware state representation summarizing interaction information at timestep t.This representation is used after the two-stage encoding.
C. Actor-Critic Policy Network
The recurrent Actor-Critic policy uses the encoded context to maintain temporal memory and directly map robot-centered observations to actions, with computation bounded by the padded crowd size.
- Actor-Critic Policy Network: A recurrent Actor-Critic policy receives context z_t, and a GRU maintains temporal memory through hidden state h_t.The hidden state is shared by the policy and value branches.
- Actor-Critic Policy Network: The policy outputs the mean action of a Gaussian distribution, while the value head estimates expected return V(o_t).The action distribution uses fixed diagonal covariance Σ.
- Actor-Critic Policy Network: The Transformer encoder and policy core are trained end-to-end with PPO and generalized advantage estimation.
- Actor-Critic Policy Network: For sequence length S = N + 1, attention and feed-forward operations determine the encoder’s main computational cost.The robot is an additional token, and dense attention scales with the padded sequence length despite masking.
- Actor-Critic Policy Network: Responsiveness annealing training performance is shown with dashed reductions of ρ_resp from 0.8 to 0.0 and variability bands of ± one standard deviation.The annealed policy converges faster and more stably than the non-annealed baseline.
E. Training Strategy with Responsiveness Annealing
Responsiveness annealing gradually increases interaction difficulty by reducing pedestrian cooperation during training, and the resulting policy is evaluated across response regimes with safety-oriented performance.
- Training Strategy with Responsiveness Annealing: Responsiveness annealing starts with a more responsive crowd and progressively reduces mean response probability ρ_resp during training.The curriculum exposes the policy to different levels of human responsiveness.
- Training Strategy with Responsiveness Annealing: Each annealing phase increases the proportion of non-responsive pedestrians and interaction difficulty.
- Training Strategy with Responsiveness Annealing: The annealed policy improves average return faster and with lower variance than abrupt fixed-difficulty training.This finding suggests that the curriculum stabilizes policy optimization.
- Training Strategy with Responsiveness Annealing: The response-regime evaluation uses 500 test episodes per setting and reports success, collision, timeout, navigation time, path length, and maximum velocity change.It covers different pedestrian response regimes, crowd densities, and unseen layouts.
- Training Strategy with Responsiveness Annealing: H2INT maintains the highest success rate and lowest collision rate across all response regimes.In the Low regime, success remains 94% and collision rate is 6%; harder settings moderately increase navigation time and path length.
B. Evaluation under Different Crowd Scales
Under non-responsive crowds, H2INT remains robust as density increases, while ablations and unseen-layout tests examine the contribution and transferability of its interaction representation.
- Evaluation under Different Crowd Scales: With ρ_resp = 0 and crowd sizes from 10 to 30, all methods degrade as density increases, but H2INT remains the most robust.
- Evaluation under Different Crowd Scales: At 30 pedestrians, H2INT achieves 88% success and 12% collision, outperforming classical and learning-based baselines.It also records the shortest navigation time and smallest velocity change among learning-based methods.
- Evaluation under Different Crowd Scales: The density study removes cooperative yielding, so the widening gap at N = 30 supports hierarchical interaction representation under dense conflicts without human assistance.
- Ablation Studies: Ablations replace H2INT with a single-stage Transformer, mean pooling, or standard Transformer residual connections.Mean pooling degrades most in the Low-response regime, while Standard TF degrades most sharply at the highest density.
- Evaluation under Different Crowd Scales: In three structurally distinct unseen layouts, H2INT retains the highest success rate and records no timeouts without retraining.Its advantage persists as responsiveness decreases, beyond the original radial Circle Crossing geometry.
E. Real-World Deployment
The learned policy was deployed on a mobile robot using sparse LiDAR-based relative pedestrian positions and reached its goal without collision in an indoor exhibition venue. This trial demonstrates deployability under noisy, partially occluded, and unmodeled motion, but not statistical superiority; broader quantitative field studies remain necessary.
- Deployment setup: LiDAR-based DR-SPAAM supplied relative pedestrian positions, while the policy received no velocity, gaze estimate, response label, or latent state.The policy’s planar velocity output was converted into bounded linear and angular commands through ROS cmd_vel.
- Observed run: The robot reached its goal without collision while naturally moving pedestrians crossed its path.The illustrated run provides a physical-world complement to simulation results.
- Observed run: The trial exercised LiDAR detection and policy inference under noise, partial occlusion, and unmodeled motion.It therefore tested the sparse observation interface in a physical environment.
- Scope: The deployment verifies a deployable observation interface rather than statistical superiority.The paper states that broader quantitative field studies are still required.
- Scope: Future work will evaluate success, pedestrian clearance, motion comfort, and runtime through quantitative field trials across multiple venues and crowd densities.The proposed extensions also include interaction-dependent cues and robustness to missing or noisy measurements.