PaperlayerView exact revision on arXiv

Source-linked AI summary

Trajectory Prediction Method for person under WSN with Hierarchical Envelope Adversarial Domain Adaptation

Wenqian Jiang, Qifeng Sun, Xiaosong Gao, Xin Zou, Xingrui Ni, Yuan Hu, Yongming Li

arXiv:2610.21132v1cs.CV

TL;DR

Existing pedestrian trajectory models often simplify social coupling and rely on predefined metrics for domain alignment. HEADA-LTCTPM combines multi-level envelope samples, loosely-tightly coupled modeling, and adversarial adaptation. It reports optimal ADE in 18 of 20 cross-domain tasks, with lower ADE and FDE than MESC-HEDA.

  • Problem

    Many models use only one social-coupling type and domain-adaptation methods mainly rely on predefined statistical measures, leaving heterogeneous interactions and domain shifts insufficiently addressed.

    Source: §1 · p. 1

  • Method

    HEADA-LTCTPM constructs multi-level envelope samples, models loosely and tightly coupled interactions, and uses hierarchical adversarial adaptation across local, global, and dual-level samples.

    Source: §1 · p. 2, §4.2.3 · p. 13

  • Results

    HEADA-LTCTPM achieves optimal ADE in 18/20 cross-domain tasks, with 17.6% lower ADE and 19.1% lower FDE than MESC-HEDA.

    Source: §6 · p. 22

  • Takeaways & Limitations

    The reported results describe a balanced precision-efficiency profile with real-time capability in challenging dense scenarios.

    Source: §6 · p. 22

Abstract

from arXiv · show

Pedestrian trajectory prediction holds significant applications across autonomous driving, intelligent surveillance, and smart cities. However, the complexity and unpredictability of pedestrian interactions make this task a persistent challenge. Current models face two common limitations: 1) Consider only a single type of social coupling. This introduces redundant interactions. More critically, it fails to account for the varying nature of trajectory coupled across different scenarios and between different pedestrians. Consequently, models cannot deeply adapt to diverse scenes. 2) Inadequate domain shift handling. Although very few methods address domain shift, they rely on statistical criterion-based domain distribution alignment. Such approaches exhibit strong dependency on predefined statistical metrics. This leads to significant limitations in complex and dynamic environments. To address these problems, a Hierarchical Envelope Adversarial Domain Adaptation with Loosely-Tightly coupled model (HEADA-LTCTPM) is proposed. First, an envelope sample construction mechanism was proposed. It constructs envelope samples and extends them into graph structures. Secondly, an adversarial domain adaptation module was proposed. This integrates both local and global domain adaptation strategies. It dynamically adapts to diverse coupled relationships across scenarios. These innovations collectively enhance prediction accuracy and robustness. In the experimental section, two representative public datasets were employed for validation and comprehensive comparisons with six relevant baseline algorithms were conducted. Results demonstrate that our model achieves significantly higher accuracy compared to existing methods, with the ADE and FDE metrics reduced by 17.6% and 19.1% respectively. The time overhead meets practical requirements, which verifies the effectiveness of our key innovations.

1 Introduction

Pedestrian trajectory prediction must account for heterogeneous social interactions and domain shifts, but existing models often use a single coupling type and metric-based alignment. HEADA-LTCTPM combines multi-level envelope samples, loosely-tightly coupled modeling, and adversarial adaptation.

1 Introduction

  • Existing models often use a single social-coupling type and predefined statistical measures for domain alignment, limiting their handling of heterogeneous interactions and domain shifts.The introduction also identifies modeling only individual pedestrians as a granularity limitation.Source: §1 · p. 1
  • The proposed envelope mechanism constructs individual-, local-, and global-level samples to represent trajectory data at multiple levels.The stated aim is to extract richer spatiotemporal feature representations.Source: §1 · p. 2
  • The loosely-tightly coupled mechanism combines interaction types and uses envelope samples of varying coupling strengths followed by decision fusion.It is intended to adaptively model heterogeneous social interactions.Source: §1 · p. 2
  • The hierarchical envelope adversarial adaptation module uses generator-discriminator learning to extract domain-invariant features without relying only on manually designed distance metrics.The introduction describes this framework as dynamically adapting across complex scenarios.Source: §1 · p. 2
  • Experiments include ablations, algorithm comparisons, parameter analysis, and trajectory and feature-distribution visualizations to assess the method.The paper frames the work as advancing from single social-coupled domain adaptation to envelope-based adversarial adaptation with loosely-tightly coupling.Source: §1 · p. 2

2 Related works

Related work spans non-adaptive trajectory models and a smaller set of domain-adaptive approaches. The section reviews their modeling strategies alongside reported strengths and limitations.

2 Related works

  • Non-adaptive approaches include physics-based and statistical models as well as deep-learning RNN, GAN, and GNN architectures for sequential data and interactions.The section notes that physics-based and statistical models face representation challenges as interactions grow more complex.Source: §2 · p. 2
  • RNN-based methods model pedestrian interactions and group or scene information, but recursive computation makes parallelization difficult and limits training efficiency and prediction timeliness.Examples include Social-LSTM and multi-scale LSTM encoders.Source: §2 · p. 2
  • GAN-based methods use interaction, attention, or group information, while reported challenges include training instability, computational cost, and poor interpretability.The section cites mode collapse as an example of instability.Source: §2 · p. 2
  • GNN-based methods represent interactions with graph structures, and the review describes their capacity for dynamic spatial relationship modeling and complex group behavior.The reviewed models include spatiotemporal graph convolution, edge attention, and interpretable interaction features.Source: §2 · pp. 2–3
  • Domain-adaptive methods address distribution differences through approaches including domain-invariant GNNs, MMD alignment, pseudo-labeling, and feature alignment by behavior.The review characterizes this research area as limited and notes that earlier mixed-distribution training did not account for domain differences.Source: §2 · p. 3

3 Motivation

The motivation frames envelope samples as an information layer between raw trajectory data and prediction. It proposes multi-level envelopes that encode inter-sample relationships, together with loosely-tightly coupling and adversarial adaptation.

3 Motivation

  • The Knowledge Pyramid frames data processing as a progression from raw inputs through information to knowledge, with information bridging raw data and actionable knowledge.The cited account describes raw inputs as abundant and potentially redundant, with preprocessing filtering useful patterns.Source: §3 · p. 3
  • Conventional deep-learning approaches model raw samples, while envelope transformation incorporates hidden inter-sample relationships and maps raw data into multi-level envelope samples.The passage reports comparative studies finding higher accuracy for hierarchical envelope modeling than conventional raw-sample modeling.Source: §3 · p. 4
  • The proposed construction forms local-, individual-, and global-level envelope samples to mine correlation information and build information hierarchies.The motivation presents this as converting raw data into information before prediction.Source: §3 · p. 4
  • The proposed approach also combines loosely and tightly coupled relationships and introduces adversarial domain adaptation to address domain shift.These components are presented as addressing relational-information mining and domain-shift problems, respectively.Source: §3 · p. 4

4 Proposed Model

HEADA-LTCTPM predicts future pedestrian trajectories from observed trajectories using hierarchical graph-based envelope representations and loosely and tightly coupled features. Its trajectory predictors are fused, while an adversarial domain adaptation module aligns local-, global-, and dual-level features across domains using attention and adversarial learning.

4.1 Problem Definition

  • The model maps observed trajectories of N pedestrians to predicted future trajectories using learnable parameters.The problem definition distinguishes true two-dimensional coordinates from predicted coordinates over the future prediction period.Source: §4.1 · p. 5

4.2.1 Overall structure of the proposed model

  • HEADA-LTCTPM constructs local-, individual-, dual-, and global-level envelopes, then uses adversarial domain adaptation and decision fusion across coupled types.Generator-discriminator training learns domain-invariant features without manual metric design; fusion is intended to improve robustness across varying scenarios.Source: §4.2.1 · p. 5, Figure 3 · p. 6
  • Local-level envelopes select neighbors using velocity, distance, and direction, then apply spatial graph convolution to represent interactions.The local-level mapping gives less influence to stopped pedestrians and greater influence to faster pedestrians in the group.Source: §4.2.1 · pp. 6–9
  • Individual-level envelopes use attention to select temporal neighbors and time graph convolution; their features are spliced and fused with local-level samples.The individual-level construction adds positional encoding before applying self-attention.Source: §4.2.1 · pp. 7, 9, 10
  • The model forms individual-level envelope samples by threshold-screening pedestrian adjacency relationships, normalizing the adjacency matrices, and applying graph convolution.It concatenates local-level and individual-level feature representations to produce dual-level envelope samples.Source: §4.2.1 · p. 10

4.2.3 Loosely-tightly coupled mechanism

  • The loosely-tightly coupled mechanism trains the same model architecture separately on loose and tight hierarchical samples, then fuses their predictions.The loose model captures relatively weak pedestrian interactions suited to sparse scenes, while the tight model depicts complex interactions suited to dense scenes; dynamically adjusting their weights balances the two relationships.Source: §4.2.3 · p. 11
  • The loosely-tightly coupled construction module separately builds loose and tight local- and individual-level envelopes, then fuses each pair into dual-level samples.The same construction procedure is applied to target-domain data.Source: §4.2.3 · pp. 11–12
  • The global envelope sample concatenates each pedestrian’s velocity vectors across the period from the observed trajectory to the predicted trajectory.The predicted trajectory length is denoted in the passage as L_pre.Source: §4.2.3 · p. 13
  • HEADA aligns local-, global-, and dual-level envelope samples and uses adversarial learning to automatically learn domain-invariant features.Attention-based feature extraction adapts to varying sample scales, mitigating scale-mismatch inaccuracies in local-level adaptation; Fig. 7 presents the module.Source: Figure 7 · p. 14

N L    F  of the target domain dual-level envelope sample are obtained

The method forms dual-level envelope features for source and target domains, then uses attention-based representations for domain adaptation and fuses loosely and tightly coupled trajectory predictions.

N L    F  of the target domain dual-level envelope sample are obtained

N L    F  from both the source and target domains,

  • Local-level domain adaptation aligns source and target feature distributions using an MMD loss.The passage introduces the loss after describing attention-based local-level feature representations.Source: N L    F  from both the source and… · pp. 15–16
  • Global-level adaptation uses the source–target linear velocity difference ratio and global envelope features to define a distribution-alignment loss.The global-level loss is represented using MMD.Source: (3) Global-level domain adaptation · p. 16
  • Weighted fusion combines loosely and tightly coupled trajectory predictions, drawing on their complementary envelope samples.The method assumes predicted trajectories follow a bivariate normal distribution when generating the two prediction types.Source: (3) Global-level domain adaptation · p. 16

5 Experiments

The experiments evaluate cross-scene trajectory prediction on ETH and UCY, using 20 source-to-target tasks and ablations of the proposed modules. The paper reports superior accuracy against six state-of-the-art models across five public datasets.

5 Experiments

  • The paper reports superior accuracy against six state-of-the-art models across five public datasets.Experiments also include trajectory and feature-distribution visualizations and computational-efficiency comparisons.Source: §5 · p. 16

5.1 Experiment settings

  • 20 cross-scene tasks train on one of five ETH and UCY scenes and test separately on each of the other four.Observed trajectories from the other four scenes' validation sets are accessed for domain adaptation during training.Source: §5.1 · p. 17
  • Experiments use eight observed frames (3.2 seconds) to predict the following 12 frames (4.8 seconds).ADE and FDE assess trajectory-point and endpoint prediction errors, respectively.Source: §5.1 · pp. 16–17

5.2 Ablation experiments

  • The ablation sequence adds hierarchical envelope samples, adversarial domain adaptation, and loosely-tightly coupling to a local-envelope baseline.The ablation table compares module-use conditions, with bold data marked as optimal.Source: §5.2 · p. 17, Table 1 · p. 17

HEADA-LTCTPM

HEADA-LTCTPM combines envelope-based social coupling with adversarial domain adaptation, then evaluates its components and full model across parameter settings and cross-domain scenarios. It outperforms the compared methods overall, with gains reported across sparse and crowded scenes.

HEADA-LTCTPM

  • 4.6% lower ADE and 6.2% lower FDE were reported for HESC versus raw-sample LESC, alongside improved relational-information mining at individual, local, and global levels.The result is attributed to the sample-envelope transformation mechanism.Source: HEADA-LTCTPM · p. 18
  • 24.2% lower ADE and 22.6% improvement in FDE were reported for HESC-HEADA versus HESC without domain adaptation.The paper attributes the cross-domain gain to learning domain-invariant feature representations.Source: HEADA-LTCTPM · p. 18
  • 10.6% lower ADE and 12.2% improvement in FDE were reported for HEADA-LTCTPM versus single-coupled HESC-HEADA across tested scenarios.The loosely-tightly coupled module generates diverse interaction samples and fuses decisions; the paper also associates the integrated modules with reduced source-target distribution discrepancies and improved adaptability.Source: HEADA-LTCTPM · p. 18

5.3 Parameter Analysis

  • Local thresholds 0.10 and 0.05 retained about 10% and 20% of social relationships with the smallest ADE/FDE; individual thresholds 0.8 and 0.7 retained about 10% and 50% of interactions with the smallest indicators.The paper reports that overly permissive tightly coupled thresholds retain redundant information, while overly restrictive loosely coupled thresholds limit interaction learning.Source: Table 2 · p. 18, §5.3 · pp. 18–19, Table 3 · p. 19

5.3.3 The different weight hyperparameters

  • The reported best loss weights were λ=0.2 for adversarial adaptation alone (ADE 0.53, FDE 0.94), σ=6 with local and adversarial adaptation (ADE 0.45, FDE 0.72), and β=2 after adding global adaptation (ADE 0.43, FDE 0.72).Each setting is reported for a different adaptation configuration.Source: Table 4 · p. 19, Table 5 · p. 19, Table 6 · p. 19

5.3.4 The impact of training sample size on model

  • ADE and FDE generally decreased as training data increased from 20% to 100%, but this trend was not clear for hotel-source to univ-target transfer.The paper notes that the trend did not hold across all datasets.Source: §5.3.4 · pp. 19–20, Figure 8 · p. 20

5.4 Algorithm comparisons

  • ADE 0.42 was 17.6% better than MESC-HEDA, and FDE 0.72 was 19.1% lower; the model ranked first in ADE on 18/20 tasks and FDE in 17 scenarios.The comparison covers six baselines across cross-domain scenarios.Source: Table 7 · p. 20, Table 8 · p. 20, §5.4 · p. 20
  • The model remained strong across sparse and dense scenarios, ranking optimal on all four HOTEL-target tasks and three of four ZARA1-source and UNIV-source tasks.The paper reports target- and source-domain comparisons separately, with MESC-HEDA or Transformer as comparators.Source: §5.4 · p. 21

6 Conclusion

The conclusion presents HEADA-LTCTPM as a combination of loosely-tightly coupled envelope samples and adversarial domain adaptation. It reports higher accuracy than MESC-HEDA, real-time capability, and two extensions for future work.

6 Conclusion

  • The method combines loosely- and tightly-coupled envelope samples with decision fusion and adversarial training that learns domain-invariant features.The conclusion describes generator-discriminator interaction as the mechanism for feature-level cross-domain alignment.Source: §6 · p. 21
  • Qualitative analyses report closer trajectory alignment with ground truth than Social-STGCNN and T-GCN in crowded scenarios involving parallel movement, encounters, stationary states, and abrupt stops.The conclusion also reports increased feature-distribution overlap between domains after adversarial adaptation.Source: §6 · p. 21
  • In 18/20 cross-domain tasks, HEADA-LTCTPM outperformed MESC-HEDA by 17.6% in ADE and 19.1% in FDE, while meeting real-time requirements.The conclusion also reports 36.3% ADE and 35.1% FDE improvements over T-GCN.Source: §6 · pp. 21–22
  • Future work proposes shifting from overhead to first-person prediction and modeling interactions with heterogeneous agents beyond pedestrians.These are presented as extensions for broader scenario applicability and complex real environments.Source: §6 · p. 22

Authors contribution statement

The contribution statement assigns research, software, visualization, investigation, and writing roles across the listed authors. The authors declare no known competing financial interests or personal relationships that could have influenced the work.

Authors contribution statement

  • Qifeng Sun and Yongming Li are credited with supervision, conceptualization, methodology, software, and original-draft preparation, as well as review and editing.The statement also credits them with supervision-related contributions.Source: Authors contribution statement · p. 22
  • Wenqian Jiang is credited with visualization, software, investigation, data curation, methodology, and original-draft preparation.Source: Authors contribution statement · p. 22
  • Xiaosong Gao and Xin Zou are credited with supervision, methodology, review, and editing; Xingrui Ni and Yuan Hu are credited with writing contributions.The statement lists original-draft preparation and review/editing for the latter authors.Source: Authors contribution statement · p. 22

Conflict of interest

  • The authors declare no known competing financial interests or personal relationships that could have influenced the reported work.Source: Conflict of interest · p. 22

Related papers

Loading 2610.21132v1…