Towards Robust and Adaptive Motion Forecasting: A Causal Representation Perspective

1. Introduction

• The learning paradigm struggles to discover physical laws under spurious shifts and can require many observations to adapt, shortcomings rooted in correlation-based statistical learning.The authors note that larger models do not remove these issues.Source: §1 · p. 1
• The formalism separates domain-invariant causal variables, domain-specific style confounders, and non-causal spurious features whose correlations with future motion can change across conditions.The variables correspond to universal physical laws, site-varying motion styles, and features that are not causal parents of future motion.Source: Figure 1 · p. 1, §1 · p. 2
• An invariance penalty on empirical-risk variation across environments encourages models to suppress spurious features while exploiting causally invariant ones.The method keeps location-specific subsets distinct rather than merging them, using their unstable correlations to promote robustness across seen and unseen environments.Source: §1 · p. 2
• A two-encoder modular architecture separates invariant mechanisms from style confounders, approximating a sparse causal graph and localizing style adaptation to fewer parameters.The invariant encoder and style encoder separately represent the two mechanisms.Source: §1 · p. 2
• A style contrastive loss organizes style embeddings without prespecifying style-class counts and can provide self-supervision for test-time refinement.The loss uses distances between scenes to capture style relations and support incremental transfer to new styles.Source: §1 · p. 2
• Experiments on synthetic and controlled real-world settings report stronger OOD generalization with invariant loss and improved low-shot transfer from the modular architecture and style loss.The framework is presented as improving robustness and reusability of learned motion representations.Source: §1 · p. 2

Motion forecasting

• Motion forecasting models trained by maximum likelihood can perform strongly in-domain yet struggle under covariate shift; negative data augmentation is an alternative, but high-dimensional interventions can be difficult to design.The paper presents its causality-inspired approach as avoiding hand-engineered training-data interventions.Source: §2 · p. 2
• The counterfactual approach [13] estimates dataset biases before subtracting them, whereas this method suppresses biased features and preserves separate location subsets to exploit unstable correlations.The paper also characterizes the counterfactual problem as generally difficult to solve and its own interventional view of spurious correlations as easier to tackle in practice.Source: §2 · p. 2

Causal learning

• Causal representation work includes disentangled learning and invariant representations from varied observational setups, but unsupervised factor separation can be difficult without strong assumptions.This paper describes its contribution as integrating invariant representations with structural architectural design informed by domain knowledge.Source: §2 · p. 3

Distribution shifts

• Domain generalization targets unseen related test distributions, while domain adaptation uses unlabelled test samples and transfer learning uses limited data.The paper notes that some domain-generalization methods rely on strong test-distribution assumptions, and that domain adaptation is poorly suited to motion forecasting where sample efficiency matters.Source: §2 · p. 3

3.1. Formalism of Motion Forecasting

• The framework treats motion forecasting as a causal process with invariant variables, site-varying style confounders, and noncausal spurious features.Style variation may impair prediction accuracy, while spurious correlations can change across environments and lead to catastrophic errors.Source: §3.1 · p. 4

3.2. Causal Invariant Forecasting

• Invariant forecasting seeks a predictor that is equally optimal across training environments, using an objective that discourages average effects of spurious features.The objective instead encourages the model to seek universal mechanisms behind motion behaviors.Source: §3.2 · pp. 4–5

3.3. Modular Forecasting Model

• A two-encoder modular network separates domain-invariant features from style features, then incorporates style into the prediction representation.Style observations may be long sequences in which the style is fully observable, unlike past trajectories that may not reveal it.Source: §3.3 · p. 5

3.4. Style contrastive loss

• The style contrastive task pairs observations from the same environment and separates those from different environments, without assuming a fixed number of style classes.This supports incremental transfer to additional styles without changing the projection head or learning parameters from scratch.Source: §3.4 · p. 6
• At test time, the style contrastive loss refines each instance's feature activations by comparing predicted outputs with observed samples of the same style.The refinement adjusts feature activations rather than model weights and reduces embedding distances to same-style reference examples.Source: §3.4 · p. 6

4. Experiments

• Models forecast 12 future time steps from eight observed steps, with ADE and FDE evaluated over five random-seed experiments.Source: §4 · p. 6

4.1. Spurious Shifts

• At 8× the maximum training spurious strength, vanilla ERM reaches ADE 3.0 and counterfactual [13] about 2.5, while invariant learning is less sensitive.Figure 5 reports that methods perform strongly in training domains, with invariant learning substantially outperforming both baselines for α ∈ (8, 64].Source: Figure 5 · p. 7, §4.1 · p. 7
• The controlled shift test trains on four ETH-UCY locations at α ∈ {1, 2, 4, 8} and tests on ‘eth’ through α = 64.The spurious-feature strength is controlled by the domain-specific parameter α.Source: §4.1 · p. 7

4.2.1 Out-of-distribution Generalization

• The modular network achieves superior performance across style-shift environments, with an invariant first encoder yielding the best OOD robustness while remaining competitive in training domains.The invariant method alone does not yield clear advantages under style changes because it tends to ignore domain-specific style confounders.Source: §4.2.1 · p. 7, Table 1 · p. 8

4.2.2 Low-shot Transfer

• Two batches of new-style observations fine-tuning f match the prediction accuracy of five batches fine-tuning the whole model.Updating f while keeping most parameters fixed performs better than fine-tuning all style-related parameters in the low-data regime.Source: Figure 6 · p. 8, §4.2.2 · p. 8
• With a target-style scene as reference, test-time refinement closes trajectory error; a mismatched-style reference steers output toward that false style.The predicted trajectory moves gradually toward the ground truth when conditioned on a scene observation of the true style.Source: Figure 7 · p. 8, §4.2.2 · p. 8

• The invariant loss improves generalization under spurious shifts, while the modular architecture and style contrastive loss improve robustness and transferability under style shifts.Source: §5 · p. 8

Robustness on the original ETH-UCY dataset

• On the original ETH-UCY dataset, training STGAT on four subsets and testing on ‘hotel’ confirms the method’s strength on real-world data.Source: A. Additional Experiments · p. 9

Low-shot transfer on the SDD dataset

• On SDD, modular adaptation is evaluated for low-shot transfer across domains defined by agent type and average speed, using Y-Net.The reported Figure 8 results demonstrate scalability to real-world style shifts.Source: A. Additional Experiments · p. 9

Larger style shifts

• For OOD-Extra domains, the full invariant-plus-modular method yields larger performance gains as style shifts grow.Table 3 summarizes ADE scores on these domains.Source: Table 3 · p. 9, A. Additional Experiments · p. 9

Style contrastive pre-training

• Style-contrastive pretraining makes the modular model learn significantly faster than the model without pretraining in both domains.Source: Figure 9 · p. 9, A. Additional Experiments · p. 9

• The added input σ_t simulates observation uncertainty as a linear function of trajectory curvature; changing α controls the spurious-shift strength.The setting models co-occurring observation noise and nonlinear trajectories, whose correlation can be exploited by statistical models.Source: §B.1 · p. 9
• The experiments use standard STGAT with a three-stage training procedure, and predict one trajectory per instance to reduce comparison costs.The first LSTM takes x, y, and σ_t as input; the remaining STGAT configuration follows the original.Source: §B.1 · pp. 9–10
• At α = 64, invariant learning remains robust while the two baselines abruptly slow their predicted trajectories under stronger spurious features.The visualization holds past trajectories and ground-truth futures fixed across methods.Source: Figure 10 · p. 10

B.2. Style Shift Experiments

• The PECNet-like base model uses MLPs and is trained in four stages, ending with end-to-end training of the full model.The stages train the invariant encoder with the decoder, pre-train the style encoder and projection head, train the style modulator, then train the entire model.Source: §B.2 · p. 10
• Learning rates are tuned separately for each module, while most hyperparameters match the Appendix B.1 setup.Training, adaptation, and refinement settings are summarized in Table 5.Source: §B.2 · p. 10
• Table 4 covers hyperparameters for spurious-shift experiments, while Table 5 covers hyperparameters for style-shift experiments.Source: Table 5 · p. 10, Table 4 · p. 10

B.3. Other Details

• All models are trained on a single NVIDIA Tesla V100 GPU.Source: §B.3 · p. 10
• Each run takes around one hour.Source: §B.3 · p. 10
• The authors provide source-code repositories for their method and baselines.Source: §B.3 · p. 10

C. Additional Discussions

• The authors describe their work as the first attempt, to their knowledge, to incorporate causal invariance and structure into motion-forecasting models.Source: C. Additional Discussions · p. 10
• The representation emphasizes three high-level latent groups but largely overlooks fine-grained feature structure.Future work could separate traffic rules from social-distance conventions within style confounders and encourage sparse interplay between submodules.Source: C. Additional Discussions · pp. 10–11
• The empirical demonstrations cover two human-motion datasets and two base models, which the authors call proofs of concept.They identify evaluation in additional motion contexts as future work.Source: C. Additional Discussions · p. 11
• The authors caution that deficits under significantly changing conditions remain relevant when using the approach in real-world applications.Source: C. Additional Discussions · p. 11