PaperlayerView exact revision on arXiv

Source-linked AI summary

Social NCE: Contrastive Learning of Socially-aware Motion Representations

Yuejiang Liu, Qi Yan, Alexandre Alahi

arXiv:2012.11717v3cs.LGcs.AIcs.CVcs.RO

TL;DR

Closed-loop motion prediction remains vulnerable to collisions, with safe-only training data offering few examples of dangerous interactions. Social-NCE uses a contrastive loss and prior-informed negative sampling to train motion representations, reducing collision rates across forecasting and navigation tasks and achieving new state-of-the-art results on several benchmarks.

  • Problem

    Neural motion models can produce colliding trajectories in closed-loop prediction, while training data often contains safe behaviors but few dangerous negative examples.

    Source: §1 · p. 1, §3 · p. 3

  • Method

    Social-NCE uses an auxiliary contrastive loss to distinguish ground-truth positive events from synthetic negatives sampled using prior knowledge about socially unfavorable states.

    Source: §1 · pp. 1–2

  • Results

    Across forecasting, behavioral cloning, and robot-navigation reinforcement learning, Social-NCE reduces collision rates and achieves new state-of-the-art results on several benchmarks; on ETH & UCY, collision rate falls by 37.0% on average for Social-STGCNN.

    Source: §1 · p. 2, §4.2.1 · p. 5

  • Takeaways & Limitations

    The results suggest negative data augmentation can be an alternative to interactive data collection for robust sequential prediction.

    Source: §5 · p. 8

Abstract

from arXiv · show

Learning socially-aware motion representations is at the core of recent advances in multi-agent problems, such as human motion forecasting and robot navigation in crowds. Despite promising progress, existing representations learned with neural networks still struggle to generalize in closed-loop predictions (e.g., output colliding trajectories). This issue largely arises from the non-i.i.d. nature of sequential prediction in conjunction with ill-distributed training data. Intuitively, if the training data only comes from human behaviors in safe spaces, i.e., from "positive" examples, it is difficult for learning algorithms to capture the notion of "negative" examples like collisions. In this work, we aim to address this issue by explicitly modeling negative examples through self-supervision: (i) we introduce a social contrastive loss that regularizes the extracted motion representation by discerning the ground-truth positive events from synthetic negative ones; (ii) we construct informative negative samples based on our prior knowledge of rare but dangerous circumstances. Our method substantially reduces the collision rates of recent trajectory forecasting, behavioral cloning and reinforcement learning algorithms, outperforming state-of-the-art methods on several benchmarks. Our code is available at https://github.com/vita-epfl/social-nce.

1. Introduction

Neural motion models can produce unsafe closed-loop predictions, in part because training data often lacks near-accident examples. Social-NCE uses prior-informed negative events to train more robust representations, reducing collision rates across several tasks and benchmarks.

1. Introduction

  • Training data often contains only safe human behaviors, leaving models without near-accident examples to learn social norms and contributing to errors that accumulate in closed-loop prediction.The resulting discrepancy between training and test state distributions can lead to catastrophic errors.Source: §1 · p. 1
  • Interactive data collection can broaden training distributions, but it is costly and often impractical for forecasting because human behavior is difficult to intervene on at scale.The paper therefore asks whether prior knowledge can improve representations through self-supervision on a fixed dataset.Source: §1 · p. 1, Figure 2 · p. 2
  • Social-NCE is an auxiliary contrastive loss that trains motion representations to distinguish ground-truth positive events from synthetic negative events.It is paired with negative data augmentation to promote robustness in multi-agent settings.Source: §1 · pp. 1–2, Figure 1 · p. 1
  • The social sampling strategy uses the primary agent’s ground-truth future location as a positive event and neighboring agents’ regions as negatives, exploiting that agents cannot occupy the same location.This self-supervision informs the model about unfavorable low-density states without collecting dangerous state-action pairs.Source: §1 · p. 2
  • Social-NCE consistently reduces collision rates and achieves new state-of-the-art results on several benchmarks across trajectory forecasting, behavioral cloning, and robot-navigation reinforcement learning.The method is model-agnostic and can be used as a generic component for neural motion models.Source: §1 · p. 2

2. Related Work

Prior approaches to socially-aware motion representations include hand-crafted rules and neural models, while sequential prediction methods address covariate shift through interventions or other objectives. Social-NCE applies contrastive learning with informed negative sampling to this multi-agent motion setting.

2. Related Work

  • Hand-crafted distance- and rule-based models have been used to study social motion, but can struggle to capture strong interactions in complex scenes.Earlier data-driven methods also faced performance drops in densely populated spaces due to limited modeling capacity.Source: §2 · p. 2
  • Neural architectures for social motion include feature pooling, attention, spatio-temporal graphs, latent variables, and injected prior knowledge; this work is orthogonal to architecture design.It focuses instead on the learning approach for robust representations.Source: §2 · p. 2
  • Interactive expert queries can help behavioral cloning but are difficult to apply to forecasting, while inverse reinforcement learning typically requires extensive exploration for an RL subproblem.Other losses penalize undesirable predicted states, but can become ineffective late in training once the model fits the dataset well.Source: §2 · p. 2
  • Related negative-sample methods learn value or cost functions, whereas this work aims to improve representations without modifying the main task loss and uses informed rather than random negatives.Social-NCE applies contrastive learning to multi-agent motion and investigates sampling specific to socially-aware representation learning.Source: §2 · pp. 2–3

3. Method

Socially aware motion models often train on safe, positive-only data, while sequential prediction can produce socially unacceptable trajectories. The method combines contrastive representation learning with negative data augmentation to learn from a fixed dataset.

3.1. Contrastive Representation Learning

  • InfoNCE trains an encoder by bringing a query close to a positive key and pushing it away from negative keys in embedding space.Its effectiveness often depends on the informativeness of the positive and negative samples.Source: §3.1 · p. 3

3.2. Social NCE

  • Social-NCE contrasts an agent’s history embedding with a future-event key, adding an auxiliary loss to the conventional task loss.The future-event key is sampled at a chosen horizon, and the loss weight controls its contribution to training.Source: §3.2 · p. 4

3.3. Multi-agent Contrastive Sampling

  • Social sampling places negatives near other agents’ future positions, using comfort-area offsets to represent unfavorable events rather than randomly scattered locations.The method synthesizes 8(M −1) negatives and adds small random perturbations; this sampling is intended to help representations identify scenarios that could lead to catastrophic outcomes.Source: §3.3 · pp. 4–5

4. Experiment

The experiments compare vanilla training, random-negative contrastive training, and Social-NCE across trajectory forecasting, imitation learning, and robot navigation. Social-NCE lowers collision rates on both forecasting benchmarks and imitation learning, while low-data evaluations report stronger safety or reward than the corresponding baselines.

4. Experiment

  • The experiments compare vanilla models, contrastive models with random negatives, and Social-NCE models using social negative sampling across forecasting, imitation, and reinforcement learning.Implementation uses two-layer MLP projection and event encoders, with 8-dimensional embeddings and task-specific distance hyperparameters.Source: §4 · p. 5, §4.1 · p. 5

4.2. Trajectory Forecasting

  • Forecasting evaluations use ETH & UCY and Trajnet++ models, with 8 observed time steps as input to predict 12 future time steps for all agents.Performance is measured by final displacement error and collision rate.Source: §4.2 · p. 5

4.2.1 ETH & UCY Benchmark

  • 37.0% for Social-STGCNN and 45.7% for Trajectron++ are the average collision-rate reductions on ETH and UCY, with competitive Top-20 FDE.The table compares Social-NCE with vanilla predictive learning on both models.Source: §4.2.1 · p. 5, Table 1 · p. 6

4.2.2 Trajnet++ Benchmark

  • 9.5%-37.5% lower collision rates than vanilla predictive learning occur across Trajnet++ interacting categories, including for the previously most robust Directional-LSTM.Prediction accuracy is not significantly affected, and the authors conjecture that output trajectories are adjusted locally rather than changing global patterns.Source: §4.2.2 · p. 6, Table 2 · p. 6

4.3. Imitation Learning

  • Imitation learning evaluates robot navigation through five simulated pedestrians using navigation time, collision rate, and accumulated reward.The experiment follows the open-source crowd-navigation simulator and evaluation protocol.Source: §4.3 · p. 6

4.3.1 Behavioral Cloning

  • Approximately 69% lower collision rate and an average reward of 0.323 are reported for Social-NCE versus vanilla behavioral cloning on 5k demonstrations.Random-negative sampling fails to outperform the vanilla baseline and even worsens the learned policy.Source: §4.3.1 · p. 7, Table 3 · p. 7

4.3.2 Low-data Regime

  • For crowd navigation, Rainbow DQN with Social-NCE is more sample efficient and achieves higher final reward than vanilla and random-negative counterparts.Offline RL results characterize Social-NCE as facilitating near-optimal policy recovery, particularly in the low-data regime.Source: Figure 6 · p. 7, Table 4 · p. 7
  • With 2k demonstrations, Social-NCE has a collision rate comparable to vanilla training with 5k, and its 5k-trained policy earns higher reward than the 10k counterpart.The figure describes vanilla performance as dropping sharply as demonstration amounts decrease, while Social-NCE retains higher reward and safety.Source: §4.3.2 · p. 7, Figure 5 · p. 7

4.4. Reinforcement Learning

Social-NCE improves off-policy Rainbow learning efficiency and narrows offline RL's performance gap with online methods. Its imitation-learning ablation finds that contrastive sampling across multiple horizons performs best.

4.4. Reinforcement Learning

  • The RL evaluation uses Rainbow with a Social-NCE encoder component and a dense reward function, testing both replay-buffer learning and static logged-data learning.The off-policy experiment averages results over eight random seeds; the offline evaluation uses ten.Source: §4.4 · pp. 7–8, §4.4.1 · p. 8, §4.4.2 · p. 8

4.4.1 Off-policy Reinforcement Learning

  • Social-NCE reaches reward 0.6 in less than 2,000 episodes, versus more than 4,000 for vanilla Rainbow, and quickly obtains a collision-free policy.It also slightly improves final performance; random negative sampling provides no significant gain.Source: §4.4.1 · p. 8

4.4.2 Offline Reinforcement Learning

  • Offline Social-NCE narrows the gap to online RL and achieves performance comparable to the best vanilla baseline using only 25% of collected data.No offline method reaches online RL reward levels, attributed to distributional shift.Source: §4.4.2 · p. 8

4.5. Ablation: Event Encoder

  • Using all four sampling horizons yields the best imitation-learning result, while the single-step horizon δt = 1 provides no improvement over baseline.Among single-step choices, δt = 3 improves both reward and collision metrics; the passage attributes δt = 1's lack of improvement to short-sightedness.Source: §4.5 · p. 8, Table 5 · p. 8

5. Conclusion

The paper presents Social-NCE with a sampling strategy as a contrastive approach to socially-aware motion representations. It reports gains on forecasting and crowd-navigation algorithms and points to negative data augmentation as a potential alternative to interactive data collection for robust sequential prediction.

5. Conclusion

  • Social-NCE combines a contrastive loss with a sampling strategy and boosts recent human trajectory forecasting and crowd-navigation algorithms.The authors suggest negative data augmentation as a promising alternative to traditional interactive data collection for robust sequential predictions.Source: §5 · p. 8

A.1. Trajectory Forecasting Details

The appendix describes evaluation on ETH & UCY and Trajnet++ using established protocols. Collision-rate evaluation differs by model prediction setup, with some models measured only over their first four prediction steps.

A.1. Trajectory Forecasting Details

  • Evaluation uses ETH & UCY and Trajnet++, a curated meta-dataset of interacting scenarios drawn from publicly available subsets.ETH & UCY contains pedestrian trajectories from varied scenarios.Source: §A.1 · p. 13
  • ETH & UCY uses leave-one-out evaluation across its five sub-datasets, while Trajnet++ uses its official training and test split.The protocol follows previous work.Source: §A.1 · p. 13
  • Collision rates for Social-STGCNN and Trajectron++ are evaluated over the first four prediction steps, whereas fully joint recurrent Trajnet++ models are evaluated over the full horizon.The first group conditions predictions only on neighboring-agent states up to observation time, unlike the fully joint recurrent models.Source: §A.1 · p. 13

A.2. Reinforcement Learning Details

The RL details use a dense reward for the model-free Rainbow algorithm and report that Social-NCE produces a smaller training–test state-distribution gap in imitation learning.

A.2. Reinforcement Learning Details

  • Rainbow uses a dense reward function to isolate the proposed method's effect from SARL's hand-crafted linear motion model and imitation pre-training.The dense reward is described as eliminating the need for imitation pre-training.Source: §A.2 · p. 13
  • The reward uses the robot's Euclidean distance to its goal, with α = 0.08 as a control parameter.Other settings are kept the same as Section 4.3.Source: §A.2 · p. 13
  • Figure 7 shows a much smaller gap between training and test state distributions with Social-NCE than with the vanilla method on crowd-navigation imitation learning.The figure presents a histogram of human–robot distance.Source: Figure 7 · p. 13

B. Covariate Shift

The analysis compares test-time human–robot distance distributions across policies, finding that our method visits dangerous states less often than vanilla imitation learning and more closely matches the training distribution there.

B. Covariate Shift

  • The analysis collects minimum robot–agent distances at each frame from 500 crowd-navigation test episodes.Source: B. Covariate Shift · p. 13
  • Given the same initial-state distribution, our method’s test distribution nearly overlaps training over dangerous distances, and visits dangerous states much less often than vanilla imitation learning.The analysis uses minimum robot–agent distance at each frame across 500 test episodes.Source: B. Covariate Shift · p. 13
  • Vanilla imitation learning yields lower test density around 1.0 m but higher density below 0.5 m than the training distribution.Source: B. Covariate Shift · p. 13
  • The expert demonstrator has near-zero density of short-distance states, reflecting social awareness and the lack of dangerous occurrences in its demonstrations.Source: B. Covariate Shift · p. 13

C. Qualitative Results

Across three interacting Trajnet++ scenarios, the vanilla Directional-LSTM produces collisions, while our method generates collision-free, socially compliant trajectories over the prediction horizon.

C. Qualitative Results

  • Vanilla Directional-LSTM collides at the 4th, 12th, and 10th predicted steps in Avoidance, Group, and Other, respectively; our method remains collision-free throughout.Source: C. Qualitative Results · p. 13, Figure 8 · p. 14
  • In Group, our primary trajectory stays between two neighbors; in Avoidance, our method cooperatively adjusts both agents’ trajectories.Source: C. Qualitative Results · p. 14
  • In Other, our method jointly twists the trajectories of pedestrians approaching from almost orthogonal directions so they pass the crowded spot smoothly.Source: C. Qualitative Results · p. 14

Related papers

Loading 2012.11717v3…