PaperlayerView exact revision on arXiv

Source-linked AI summary

TransferTraj: A Vehicle Trajectory Learning Model for Region and Task Transferability

Tonglong Wei, Yan Lin, Zeyu Zhou, Haomin Wen, Jilin Hu, Shengnan Guo, Youfang Lin, Gao Cong, Huaiyu Wan

arXiv:2505.12672v1cs.LG

TL;DR

TransferTraj addresses the difficulty of transferring trajectory models across regions and tasks without retraining, where spatial-context variation and task-specific input-output structures limit existing approaches. It uses a multimodal region-transferable encoder and a unified masking-and-recovery scheme, and experiments report strong task and zero-shot or few-shot region transfer. The paper’s stated limitation is weaker coverage of cross-region classification tasks.

  • Problem

    Trajectory models need to transfer across regions and tasks without retraining, but spatial-context variation and differing task input-output structures make this difficult.

  • Method

    TransferTraj combines RTTE, which integrates spatial, temporal, POI, and road-network modalities with TRIE and SC-MoE, and masking-based pretraining that unifies task inputs and outputs.

  • Results

    TransferTraj consistently outperforms baselines across task-transfer, zero-shot region-transfer, and few-shot region-transfer experiments.

  • Takeaways & Limitations

    A single pretrained TransferTraj model can perform different generative tasks and transfer across regions without retraining.

  • Takeaways & Limitations

    Cross-region classification tasks such as trajectory-user linking and destination road-segment prediction remain challenging because user and road-network counts vary across regions.

Abstract

from arXiv · show

Vehicle GPS trajectories provide valuable movement information that supports various downstream tasks and applications. A desirable trajectory learning model should be able to transfer across regions and tasks without retraining, avoiding the need to maintain multiple specialized models and subpar performance with limited training data. However, each region has its unique spatial features and contexts, which are reflected in vehicle movement patterns and difficult to generalize. Additionally, transferring across different tasks faces technical challenges due to the varying input-output structures required for each task. Existing efforts towards transferability primarily involve learning embedding vectors for trajectories, which perform poorly in region transfer and require retraining of prediction modules for task transfer. To address these challenges, we propose TransferTraj, a vehicle GPS trajectory learning model that excels in both region and task transferability. For region transferability, we introduce RTTE as the main learnable module within TransferTraj. It integrates spatial, temporal, POI, and road network modalities of trajectories to effectively manage variations in spatial context distribution across regions. It also introduces a TRIE module for incorporating relative information of spatial features and a spatial context MoE module for handling movement patterns in diverse contexts. For task transferability, we propose a task-transferable input-output scheme that unifies the input-output structure of different tasks into the masking and recovery of modalities and trajectory points. This approach allows TransferTraj to be pre-trained once and transferred to different tasks without retraining. Extensive experiments on three real-world vehicle trajectory datasets under task transfer, zero-shot, and few-shot region transfer, validating TransferTraj's effectiveness.

1 Introduction

Trajectory learning models must support diverse generative tasks while transferring across regions without retraining, but spatial-context variation and task-specific input-output structures make this difficult.

  • Motivation: Dedicated models for each region and task incur substantial training costs and require abundant trajectory data, motivating transfer without retraining.Region transfer means applying a model trained in one region to another, while task transfer means reusing a model across tasks such as prediction and travel-time estimation.
  • Challenges: Region transfer is hindered by region-specific spatial features and POI or road distributions that alter turning, acceleration, and movement-direction patterns.Common normalization and grid-discretization methods can produce inconsistent spatial scales across regions.
  • Challenges: Task transfer is difficult because trajectory prediction and travel-time estimation require different input-output structures and learn different correlations.Prediction generates future points from historical sequences, whereas travel-time estimation predicts arrival time from origin to destination.
  • Proposed approach: TransferTraj combines the Region-Transferable Trajectory Encoder with a task-transferable masking-and-recovery scheme to transfer across regions and generative tasks.RTTE integrates spatial, temporal, POI, and road-network modalities and uses TRIE and SC-MoE, while the task scheme unifies task structures through masking and recovery.

2 Related Works

Trajectory learning research distinguishes models specialized to particular regions and tasks from transferable models based primarily on pretrained trajectory embeddings.

  • Categorization: Trajectory learning models are categorized as non-transferable or transferable according to whether they generalize across regions or tasks.
  • Non-transferable models: Non-transferable models are trained end-to-end for specific tasks and regions, using architectures such as RNNs or Transformers for prediction and recovery.Examples include DeepMove, HST-LSTM, ACN, PreCLN, TrImpute, and DHTR.
  • Transferable models: Transferable models mainly pretrain trajectory encoders with auto-encoding or contrastive learning to map trajectories into embedding vectors.Examples include trajectory2vec, t2vec, Trembr, START, and MMTEC.

3 Preliminaries

The paper represents trajectories as timestamped geographic points, supplements them with POI and road-segment context, and defines transferability as cross-region and cross-task generation without parameter retraining.

  • Vehicle trajectory: A vehicle trajectory is a sequence of points, each containing longitude, latitude, and a timestamp.The trajectory is written as T = ⟨p1, p2, . . . , pn⟩, with pi = (lngi, lati, ti).
  • Spatial context: A POI is represented by coordinates and a textual description containing its name, type, and address.
  • Spatial context: A road segment is represented by its midpoint coordinates and textual information about its name, type, and length.
  • Problem definition: Region- and task-transferable learning seeks a pretrained model that generates task-specific outputs across trajectory datasets from different regions without retraining its parameters.The objective is formalized through a trajectory model fθ with learnable parameters θ.

4 Methodology

TransferTraj combines a region-transferable encoder with a task-transferable masking-and-recovery scheme. RTTE integrates multiple trajectory modalities, relative spatial information, and context-conditioned experts, while pre-training masks modalities and trajectory points to support task transfer without retraining.

  • TransferTraj comprises RTTE and a task-transferable input-output scheme for region and task transferability.
  • Region-Transferable Trajectory Encoder: RTTE represents each trajectory point with spatial, temporal, POI, and road-network modalities, then mixes them into latent trajectory-point embeddings.POIs and road segments provide contextual information, while temporal features include calendar and relative-time representations.
  • Region-Transferable Trajectory Encoder: TRIE models correlations between trajectory embeddings and relative spatial information to reduce bias toward region-specific spatial features.It applies a learnable spatiotemporal rotation to attention queries and keys, incorporating relative spatial relationships during attention computation.
  • Region-Transferable Trajectory Encoder: SC-MoE uses gated experts to model diverse movement patterns associated with different local spatial contexts.Each expert is implemented with two MLP layers, and the gating network selects among C expert networks.
  • Task-Transferable Input-Output Scheme: The task-transferable scheme masks either a modality at individual points or complete trajectory points and trains the model to recover the masked content.Pre-training mixes complete sub-trajectory masking with randomly selected spatial or temporal modality masking, using summed reconstruction losses.

5 Experiments

Experiments evaluate TransferTraj across three real-world datasets, three generative tasks, task transfer, zero-shot and few-shot region transfer, and component ablations. TransferTraj consistently outperforms baselines, while ablations identify relative spatial information, spatial-context experts, and multimodal context as important components.

  • Experimental setup: The evaluation uses Chengdu, Xi’an, and Porto trajectory datasets, with TP, TR, and OD TTE baselines and task-transfer settings involving 30 epochs of trajectory masking and recovery pretraining.Few-shot regional transfer fine-tunes models using 5,000 sampled trajectories.
  • Task transferability: TransferTraj consistently outperforms baselines across TP, TR, and OD TTE, with pretraining alone gaining 20.18%, 17.87%, and 7.94% over START, MM-STGED, and DOT, respectively.Fine-tuning further improves performance by 11.41%, 13.59%, and 8.62% on the three tasks.
  • Task transferability: TransferTraj can be pretrained once and perform different tasks without fine-tuning its learnable model or prediction modules, unlike comparison methods that require task-supervised encoder fine-tuning.The experiments include task-specific fine-tuning as an additional performance-improvement setting.
  • Region transferability: TransferTraj improves over SOTA models by 83.70% and 33.68% for TP, 18.08% for TR, and 10.88% and 13.07% for OD TTE under zero-shot or few-shot region transfer.The reported comparisons cover zero-shot TP and OD TTE, plus few-shot transfer across all three tasks.
  • Ablation study: Removing TRIE causes a significant decline, while removing SC-MoE produces a 14.52% drop on OD TTE, supporting relative spatial information and context-specialized movement modeling.Adding POI and road-network modalities further improves performance by providing richer spatial context.
  • Analysis of the SC-MoE: SC-MoE activates different expert distributions for high-, medium-, and low-density spatial contexts, indicating dynamic selection of specialized components.The density categories are defined by nearby POIs and road segments: above 15, 5–15, and below 5.

6 Conclusion

TransferTraj is designed to achieve both region and task transferability through RTTE and a unified masking-and-recovery scheme. Its mechanisms support cross-region spatial-context modeling and task transfer without retraining.

  • 6 Conclusion: TransferTraj combines RTTE for region transferability with a task-transferable input-output scheme for cross-task transfer.RTTE incorporates trajectory modalities, while the input-output scheme unifies tasks through masking and recovery.
  • 6 Conclusion: RTTE incorporates POI and road-network modalities to model spatial-context distributions across regions.The component also uses trajectory modalities to support region transferability.
  • 6 Conclusion: TRIE captures relative spatial correlations among trajectory points, while SC-MoE identifies shared movement patterns in similar spatial contexts.Together, these mechanisms are intended to reduce bias toward specific regions.
  • 6 Conclusion: Randomly masking and recovering trajectory modalities or entire trajectory points enables transfer to various tasks without retraining.The unified scheme is paired with an effective pre-training mechanism.

A.1 Limitations

The paper identifies cross-region classification as a limitation because regions contain different numbers of users and road networks. The evaluated datasets also vary in scale, sampling, spatial extent, origin, POIs, and road segments, while experiments use fixed implementation settings and repeated runs.

  • A.1 Limitations: Cross-region classification tasks remain challenging because regions differ in the numbers of users and road networks.Examples include trajectory-user linking and destination road-segment prediction.
  • A.1 Limitations: Chengdu, Xian, and Porto differ in data scale, sampling frequency, spatial size, geographic origin, POIs, and road segments.This heterogeneity is used to assess regional transferability.
  • A.1 Limitations: Experiments split trajectories chronologically into 8:1:1 training, validation, and testing sets and optimize TransferTraj with Adam at a 1e-3 learning rate.The batch size is 64, and baselines use hyperparameters reported in their original papers.
  • A.1 Limitations: The reported metrics are averaged over five repeated experiments with standard deviations.Experiments use Ubuntu 22.04 servers with Intel Xeon W-2155 CPUs and NVIDIA TITAN RTX GPUs.

B.3 Evaluation Metrics

The evaluation covers trajectory prediction, recovery, and origin-destination travel-time estimation using task-specific error metrics and broad baseline comparisons. It also tests TransferTraj variants that remove key architectural components or modalities.

  • B.3 Evaluation Metrics: TP and TR use MAE and RMSE, while OD TTE additionally uses MAPE; lower metric values indicate better performance.For TP and TR, the metrics evaluate distance error; for OD TTE, they assess travel-time discrepancy.
  • B.3 Evaluation Metrics: The evaluation compares TransferTraj with seven trajectory-representation baselines for TP, including methods based on denoising, RNNs, Transformers, and context-aware representations.Baseline outputs are followed by two MLP layers to predict destination location.
  • B.3 Evaluation Metrics: For TP, frozen-encoder variants of Trembr, START, and LightPath are compared after fine-tuning only their MLP layers.These variants are reported as Trembr (wo ft), START (wo ft), and LightPath (wo ft).
  • B.3 Evaluation Metrics: TR evaluation includes seven recovery baselines spanning three rule-based relu methods and four learning-based methods.The listed methods include Linear, MPR, TrImpute, DHTR, and MTrajRec.
  • B.3 Evaluation Metrics: OD TTE evaluation compares eight origin-destination travel-time estimation baselines, including embedding, historical-average, linear-regression, and gradient-boosting approaches.The listed examples include RNE, TEMP, LR, and GBM.
  • B.3 Evaluation Metrics: Ablations remove TRIE, SC-MoE, POI modality, or road-network modality to assess their contributions.The variants are named wo TRIE, wo SC-MoE, wo POI modality, and wo road network modality.

C Input and Output Structure for Different Tasks

TransferTraj represents different tasks through masked trajectory components and recovers task-specific outputs from the corresponding modalities. The task structures cover future-point prediction, dense trajectory recovery, and travel-time estimation from origin, destination, and departure-time information.

  • C Input and Output Structure for Different Tasks: Trajectory prediction masks the future segment after providing the historical points and evaluates predicted destinations using MAE and RMSE.The historical length is set to n′ = n − 5.
  • C Input and Output Structure for Different Tasks: Trajectory recovery reconstructs a dense trajectory from a sparse one by recovering the spatial modality of masked trajectory points.The recovery settings use sampling-gap ratios of 4·ϵ, 8·ϵ, and 16·ϵ, with MAE and RMSE evaluation.
  • C Input and Output Structure for Different Tasks: Origin-destination travel-time estimation uses starting location, ending location, and departure time as inputs and derives travel time from the recovered temporal modality of the last point.The task uses MAE, RMSE, and MAPE.

D The Proof of TRIE to Capture Relative Spatial Information

TRIE applies learnable spatiotemporal rotations to trajectory-point queries and keys so their attention interactions encode relative spatial information. The derivation shows that multiplying point-specific rotations yields a rotation based on the coordinate difference between points, helping reduce region-specific bias.

  • TRIE design: TRIE applies a learnable spatiotemporal rotation matrix RΦ(x,y) to trajectory-point queries and keys to encode relative spatial information.For points pi and pj, the spatial modalities are (xi, yi) and (xj, yj), respectively.
  • Relative-position derivation: The matrix product of rotations at two points produces a rotation based on their coordinate difference, RΦ(xi−xj,yi−yj).The derivation expands the resulting attention interaction into paired sine and cosine terms for each rotation block.
  • Transferability implication: This relative-information design helps mitigate region-specific biases and improves region transferability.The construction uses the attention mechanism rather than relying only on absolute spatial features.

E Performance Comparison in Task Transfer

TransferTraj transfers across trajectory-recovery tasks and geographic regions without retraining, with masking and recovery strategies supporting task transfer. In few-shot regional transfer, it remains strongest across prediction, recovery, and OD travel-time estimation after fine-tuning on limited target-region data.

  • Task transfer: 9.25% and 1.20% improvements over state-of-the-art trajectory-recovery baselines on Xi’an and Porto show task transferability without fine-tuning.These results use only the pretraining scheme, TransferTraj wo ft, and are attributed largely to modality and sub-trajectory masking and recovery.
  • Task transfer: 11.68% and 2.42% improvements over the baseline remain in the most challenging recovery setting, where µ = 16 ∗ϵ.Fine-tuning further increases performance by 9.54% and 5.56% on the two datasets.
  • Few-shot region transfer: 33.68%, 18.08%, and 13.07% improvements on TP, TR, and OD TTE show the best few-shot regional-transfer performance across the evaluated datasets.The setting fine-tunes source-region models with 5,000 trajectories from the target region.
  • Few-shot region transfer: The model’s few-shot transfer advantage is attributed to a region-agnostic feature representation that adapts across geographic areas.Baseline models gain substantially after target-region fine-tuning because they focus on region-specific representations.

G Hyperparameter Study

The hyperparameter study evaluates the hidden dimension, stacked-block count, and SC-MoE configuration for trajectory prediction. It identifies d = 256 and k = 4 with C = 8 as optimal settings, while excessive model capacity can reduce efficiency or complicate training.

  • Study scope: The study analyzes d, the number of stacked TRIE and SC-MoE blocks L, and the SC-MoE parameters k and C across three trajectory-prediction datasets.The results indicate that varying L significantly affects model performance, although the supplied passage does not state its optimal value.
  • Hidden dimension: d = 256 is optimal; smaller hidden dimensions limit learning capability, whereas larger values provide minimal gains and reduce computational efficiency.The study varies the hidden state dimension d across three datasets for trajectory prediction.
  • SC-MoE configuration: k = 4 and C = 8 yield optimal SC-MoE performance.Too few experts limit movement-pattern coverage, while too many increase complexity, complicate training, and compromise performance.
  • Efficiency: TransferTraj is lightweight and has training and testing efficiency competitive with other methods while supporting multiple tasks and regions after one training run.Its model size is comparable to RNN-based t2vec and Trembr and much smaller than START and LightPath.
Loading 2505.12672v1…