PaperlayerView exact revision on arXiv

Source-linked AI summary

End-to-End Safe Social Navigation via Multi-Task Reinforcement Learning and Probabilistic Perception

Tommaso Van Der Meer, Andrea Garulli, Antonio Giannitrapani, Renato Quartullo, Alberto Vaglio, Alexandre Alahi

arXiv:2610.05733v2cs.ROcs.HC

TL;DR

Social navigation must balance efficiency, physical safety, and social compliance, while many RL methods assume exact human positions and velocities. JESSI combines raw-LiDAR control with probabilistic human perception and constrained actions; across two simulated setups, its variants rank among the strongest on success rate, and JESSI-MT has the highest space compliance among them.

  • Problem

    Social navigation must balance efficiency, physical safety, and social compliance, but many approaches assume exact human states despite noisy, limited, and occluded onboard perception.

    Source: §I · p. 1

  • Method

    JESSI maps raw LiDAR to control through an actor-critic policy and probabilistic human-state perception, with Dirichlet actions and deterministic bounding for safety.

    Source: §I · p. 2, A. Robot kinematics and action space… · p. 2, B. Neural network architecture · p. 3

  • Results

    Across two simulated setups, JESSI policies are among the top success-rate performers, and JESSI-MT has the highest space compliance among JESSI variants.

    Source: G. Experimental results · pp. 6–7

  • Takeaways & Limitations

    The intermediate probabilistic representation grounds the RL policy in a physically meaningful latent space.

    Source: §V · p. 8

Abstract

from arXiv · show

Autonomous social navigation requires balancing efficiency, physical safety, and social compliance. Reinforcement Learning (RL) methods provide a viable and effective solution but often rely on unrealistic assumptions, such as the knowledge of humans' position and velocity. In this paper, we introduce JESSI (JAX-based E2E Safe Social Interpretable navigation), a lightweight end-to-end RL framework that maps raw LiDAR scans directly to kinematically feasible control commands. JESSI enhances safety via Dirichlet-parameterized continuous action spaces and deterministic bounding, while an integrated attention-based perception module extracts probabilistic human states for interpretable, socially aware decision-making. Through extensive simulations and real-world deployment on a differential-drive robot, we demonstrate that jointly optimizing the RL policy with a supervised perception signal in a multi-task paradigm enhances social behavior. Ultimately, JESSI is able to balance high navigation success rates and superior social behaviors compared to state-of-the-art baselines.

I. INTRODUCTION

Social navigation must balance efficiency, physical safety, and social norms, yet existing approaches can assume perfect human-state knowledge or neglect obstacles and human semantics. JESSI combines raw-LiDAR end-to-end control with probabilistic human perception, kinematic action constraints, and a differentiable architecture.

I. INTRODUCTION

  • Existing approaches may assume exact human states, omit static obstacles, treat people as geometric obstacles, or generate kinematically infeasible commands.Onboard perception also faces sensor noise, limited fields of view, and occlusions.Source: §I · p. 1
  • JESSI brings multi-task end-to-end learning to safe social navigation, following work that reports benefits from jointly optimizing auxiliary perception tasks.Its lightweight JAX implementation uses vectorization and just-in-time compilation for efficient inference on embedded systems.Source: §I · pp. 1–2
  • JESSI cascades perception and actor-critic networks to map raw LiDAR directly to control, without an external oracle for human positions and velocities.The architecture also considers static obstacles and is agnostic to the specific LiDAR setting.Source: §I · p. 2
  • JESSI estimates pedestrian positions and velocities and feeds structured human states into scene attention to encourage socially compliant behavior.Its intermediate representation includes confidence scores and bivariate Gaussian human-state distributions, providing insight into robot decisions.Source: §I · p. 2
  • Dirichlet action distributions and deterministic bounding provide a safety layer for differential-drive kinematics and static-obstacle collision avoidance.The framework's hybrid structure preserves differentiability for joint optimization.Source: §I · p. 2

II. METHOD

The method combines kinematically constrained continuous actions, probabilistic human perception, and multi-stage training. The stated training goals are sample efficiency and stable convergence.

II. METHOD

  • The continuous action parameterization inherently respects the robot's kinematic constraints.An attention-based perception module extracts probabilistic human-state representations from raw sensor data.Source: §II · p. 2
  • Attention-based perception extracts probabilistic representations of surrounding human states from raw sensor data.The passage identifies this as part of the method's core approach.Source: §II · p. 2
  • A multi-stage training pipeline is used to promote sample efficiency and stable convergence.Source: §II · p. 2

A. Robot kinematics and action space parameterization

JESSI models a differential-drive robot’s kinematically feasible actions and maps network outputs into that feasible space with collision-aware bounding. Its integrated network combines perception and actor-critic modules for safe navigation commands.

A. Robot kinematics and action space parameterization

  • The discrete-time kinematic model describes a differential-drive robot as a disk with position and orientation, whose bounded wheel speeds define a triangular feasible action space.The action is a = [v, ω]T, with forward motion constrained by v ≥ 0.Source: A. Robot kinematics and action space… · p. 2
  • Dirichlet samples over action-space vertices produce feasible continuous commands, while LiDAR-scaled deterministic bounding prevents intersections with measured point clouds.The action space is triangular because wheel speeds are bounded and the robot moves forward only.Source: A. Robot kinematics and action space… · p. 2

B. Neural network architecture

  • The proposed architecture integrates a perception module and actor-critic policy to process raw sensory data and output safe navigation commands.Fig. 3 distinguishes the perception module, actor-critic module, and network inputs; its dimensions label the LiDAR stack, beams, embedding, angular sectors, and predicted objects.Source: B. Neural network architecture · p. 3, Figure 3 · p. 3

1) Perception Module:

The perception module processes stacked LiDAR scans with attention layers to estimate nearby pedestrians’ probabilistic states. Its outputs encode confidence and uncertainty for downstream navigation.

1) Perception Module:

  • The module processes a stack of recent LiDAR scans aligned to the latest robot-centric frame to predict nearby pedestrians’ dynamic states.Its beam inputs include normalized distance, range validity, hit-point coordinates, beam-angle sine and cosine, and measurement-time difference.Source: 1) Perception Module · p. 3
  • Angular local cross-attention maps beam embeddings to robot-fixed angular sectors, and spatio-temporal self-attention aggregates features across scans while supporting variable observation-history lengths.A later learned-query cross-attention layer reduces features to the desired number of output components.Source: 1) Perception Module · p. 3
  • Each predicted pedestrian component includes a confidence score and bivariate Gaussian estimates of position and velocity, representing spatial uncertainty from LiDAR occlusions and sparsity.The module detects humans rather than tracking identities, since the policy uses their probabilistic states rather than identities.Source: 1) Perception Module · p. 3

2) Actor-Critic policy:

The actor-critic policy combines robot state, probabilistic human estimates, and LiDAR features to select actions. A three-stage training pipeline pretrains perception, warms up the policy through imitation, and then fine-tunes the full architecture with reinforcement learning and an auxiliary perception loss.

2) Actor-Critic policy:

  • Scene attention selects humans using whole-scene context, and confidence-weighted pooling emphasizes reliable detections before the actor outputs Dirichlet concentration parameters and the critic estimates state value.The policy also receives robot proprioception and LiDAR-derived obstacle features, along with human position and velocity uncertainties.Source: 2) Actor-Critic policy · pp. 3–4

C. Multi-stage training pipeline

  • Perception is first pretrained with supervised data, using Hungarian matching and a weighted loss for position, velocity, and confidence predictions.The position and velocity terms are negative log-likelihoods under predicted Gaussians; confidence classification uses binary cross-entropy, with random rotations and Gaussian noise for augmentation.Source: C. Multi-stage training pipeline · p. 4
  • Behavioral cloning then warms up actor-critic networks from expert planner trajectories by matching expert actions and predicted state values to their targets.The action target is the mean of the predicted Dirichlet distribution, and the value target is the empirical discounted return.Source: C. Multi-stage training pipeline · p. 4
  • The full architecture is fine-tuned with PPO, while an auxiliary perception loss is included in the total objective to support robust representation learning.Policy and value gradients may or may not flow back through the perception backbone, depending on the training paradigm.Source: C. Multi-stage training pipeline · p. 4

III. NUMERICAL RESULTS

The paper evaluates its approach through a campaign of simulated experiments and introduces the simulation environment and scenario design.

III. NUMERICAL RESULTS

  • An extensive campaign of simulated experiments evaluates the proposed approach, with the simulation environment and scenario design described first.The supplied passage does not report experimental outcomes.Source: §III · p. 4

A. Simulation environment and scenario design

The simulator models varied pedestrian interactions, randomized layouts, and noisy LiDAR, while the reward combines goal progress, collision penalties, social comfort, and trajectory smoothness.

A. Simulation environment and scenario design

  • Simulated pedestrians follow the Headed Social Force Model, which produces yielding, decelerating, and collision-avoiding trajectories; the robot is hidden from them in training but visible in testing.The simulator randomizes human and obstacle placement and adds Gaussian noise and range outliers to 360° LiDAR scans.Source: A. Simulation environment and scenario… · p. 4

B. Reward function

  • The reward combines goal reaching, collision penalties, and social comfort with weighted terms for goal progress and reduced angular velocity.The supplied weights are wb = 1, wp = 0.03, and wr = 0.0075; the social discomfort distance is 0.2 m.Source: B. Reward function · p. 5

C. Training paradigms

The study compares three PPO training paradigms that differ in whether reinforcement-learning gradients update the perception module.

C. Training paradigms

  • JESSI-MT jointly trains the policy and perception module end-to-end, allowing both reinforcement-learning and supervised-perception gradients to shape shared representations.The perception module may learn task-aware representations focused on navigation-critical humans.Source: C. Training paradigms · p. 5
  • JESSI-MD trains perception only with supervised loss while a stop-gradient prevents reinforcement-learning gradients from changing its state estimates.It requires an additional fine-tuning phase during policy training.Source: C. Training paradigms · p. 5
  • JESSI-PO freezes perception after supervised pretraining and updates only the actor-critic networks using the reinforcement-learning loss.This is described as the most computationally lightweight training approach.Source: C. Training paradigms · p. 5

D. Experimental setup

The evaluation standardizes raw-LiDAR inputs and feasible continuous actions, then tests navigation across varied human and obstacle scenarios against model-based and learning-based baselines.

D. Experimental setup

  • All policies receive raw LiDAR scans and return continuous commands bounded to the differential-drive robot’s triangular feasible action space.State-based baselines use JESSI’s pretrained perception, with only human tracks above 0.5 confidence passed to them.Source: D. Experimental setup · p. 5
  • The first setting covers six training scenarios and three unseen scenarios, with success requiring reaching the goal without collision before the time limit.The second setting models static obstacles as stationary humans in a wider, wall-free scene.Source: D. Experimental setup · pp. 5–6, Figure 5 · p. 6

1) Model-based baselines:

2) End-to-end learning baselines:

Across both experimental setups, the three JESSI variants lead in success rate, with JESSI-MT showing the strongest space compliance among them; the evaluation also compares efficiency, safety, and social behavior.

2) End-to-end learning baselines:

F. Evaluation Metrics

  • The evaluation tracks success, timeout, human and obstacle collision rates, time to goal, linear and angular jerk, and the fraction of trajectory kept more than 0.5 m from humans.These metrics cover navigation efficiency, safety, smoothness, and space compliance.Source: F. Evaluation Metrics · p. 6

G. Experimental results

  • DRA-MPPI attains the highest space compliance but also one of the highest timeout rates, while noisy inputs make SARL* action filtering less effective against obstacles.The reported obstacle-bounding algorithm is naturally conservative for DIR-SAFE.Source: G. Experimental results · p. 7
  • JESSI variants rank highest in success rate in both setups, and JESSI-MT has the highest space compliance among the three; vanilla E2E methods lead in time to goal and path smoothness.In the first setup, BV-E2E has a much lower obstacle-collision rate than V-E2E; Table I reports the first-setup results.Source: G. Experimental results · pp. 6–7, Table I · p. 7

IV. REAL WORLD EXPERIMENTS

JESSI's evaluation reports improved social performance alongside high navigation success, and the trained policy was tested on two robot platforms. The conclusion links probabilistic human-state perception and deterministic action bounding to the reported performance and safety benefits.

IV. REAL WORLD EXPERIMENTS

  • The trained JESSI-MT policy was fine-tuned and tested on Turtlebot4 and Loomo, with Loomo depth converted to a roughly 60° 2D LiDAR scan.Inference averaged 20 ms on an Intel Core i5-7200U CPU.Source: §IV · p. 7, Figure 6 · p. 7

V. CONCLUSIONS

  • JESSI's numerical evaluation associates explicit human-state semantics with higher success and slightly higher space compliance, while deterministic bounding reduces static-obstacle collisions.Source: §V · p. 8
  • JESSI maps raw LiDAR scans to kinematically feasible continuous commands through a sensor-agnostic architecture.The conclusion describes this as bridging idealized state-based planning and practical sensor-based autonomy.Source: §V · pp. 7–8
Loading 2610.05733v2…