UniTraj: Learning a Universal Trajectory Foundation Model from Billion-Scale Worldwide Traces

Yuanshao Zhu; James Jianqiao Yu; Xiangyu Zhao; Xun Zhou; Liang Han; Xuetao Wei; Yuxuan Liang

Source-linked AI summary

UniTraj: Learning a Universal Trajectory Foundation Model from Billion-Scale Worldwide Traces

Yuanshao Zhu, James Jianqiao Yu, Xiangyu Zhao, Xun Zhou, Liang Han, Xuetao Wei, Yuxuan Liang

arXiv:2411.03859v3Nov 6, 2024cs.ETcs.AIcs.LGcs.SIphysics.soc-ph

TL;DR

Existing trajectory models struggle to generalize across tasks, regions, and heterogeneous data quality. UniTraj addresses these gaps with WorldTrace, adaptive resampling, self-supervised masking, and a flexible trajectory backbone, and experiments report improved performance across multiple tasks and settings. Its scope remains bounded by uneven regional coverage, motorized-movement focus, and substantial computational requirements.

Problem
Current trajectory models are limited by task specificity, regional dependency, and sensitivity to noisy, irregular, or incomplete data.
Method
UniTraj combines WorldTrace with adaptive resampling, self-supervised masking, and a flexible architecture for heterogeneous trajectory modeling.
Results
UniTraj demonstrates improved zero-shot and fine-tuning performance across multiple trajectory analysis tasks and real-world datasets.
Takeaways & Limitations
UniTraj and WorldTrace provide a versatile foundation for trajectory analysis across diverse tasks and geographic regions.
Takeaways & Limitations
WorldTrace unevenly represents some regions, focuses on motorized movement, and requires substantial computational resources for large-scale training and deployment.

Abstract

from arXiv · show

Building a universal trajectory foundation model is a promising solution to address the limitations of existing trajectory modeling approaches, such as task specificity, regional dependency, and data sensitivity. Despite its potential, data preparation, pre-training strategy development, and architectural design present significant challenges in constructing this model. Therefore, we introduce UniTraj, a Universal Trajectory foundation model that aims to address these limitations through three key innovations. First, we construct WorldTrace, an unprecedented dataset of 2.45 million trajectories with billions of GPS points spanning 70 countries, providing the diverse geographic coverage essential for region-independent modeling. Second, we develop novel pre-training strategies--Adaptive Trajectory Resampling and Self-supervised Trajectory Masking--that enable robust learning from heterogeneous trajectory data with varying sampling rates and quality. Finally, we tailor a flexible model architecture to accommodate a variety of trajectory tasks, effectively capturing complex movement patterns to support broad applicability. Extensive experiments across multiple tasks and real-world datasets demonstrate that UniTraj consistently outperforms existing methods, exhibiting superior scalability, adaptability, and generalization, with WorldTrace serving as an ideal yet non-exclusive training resource.

1 Introduction

Existing trajectory models are limited by task specificity, regional dependency, and sensitivity to noisy or irregular data. UniTraj addresses these gaps with WorldTrace, robust pre-training strategies, and a flexible architecture for broad trajectory analysis.

Motivation: Current approaches are often task-specific, region-dependent, and sensitive to noise, irregular sampling, and missing trajectory data.These limitations create a gap for a universal model spanning diverse tasks, geographic regions, and data-quality levels.
Approach: UniTraj combines WorldTrace, adaptive resampling, self-supervised masking, and a flexible architecture to support task-adaptive, region-independent, data-quality-resilient modeling.The architecture is designed to capture complex spatio-temporal dependencies while adapting across trajectory tasks.
WorldTrace: WorldTrace contains 2.45 million trajectories spanning 70 countries and provides globally diverse data for region-agnostic trajectory modeling.The dataset is presented as a large-scale, high-quality foundation for overcoming region-specific data limitations.
Results: UniTraj improves zero-shot and fine-tuning performance across multiple trajectory analysis tasks, especially when trained on diverse, high-quality data such as WorldTrace.The experiments support its use as a versatile backbone for diverse trajectory modeling tasks.

2 Related Work

Prior trajectory datasets differ substantially in geographic coverage, quality, and granularity, while foundation-model research has extended large-scale pretraining beyond language and vision into temporal and spatio-temporal domains.

Trajectory Datasets: Existing trajectory datasets vary in geographic coverage, data quality, and granularity, limiting their utility across mobility applications.GeoLife is cited as useful for travel-mode detection and traffic-flow analysis but limited by geographic coverage and participant diversity.

3 Preliminary

The paper formalizes trajectories as time-ordered GPS-point sequences and frames universal trajectory modeling as learning representations that adapt across tasks, regions, and heterogeneous data quality.

Definitions: A trajectory is a sequence of GPS points containing longitude, latitude, and timestamp values, with sampling intervals that may vary across data.The sequence length is n, and consecutive-point intervals are defined as Δt_i = t_i − t_{i−1}.
Definitions: A trajectory dataset is a collection of trajectories whose geographic coverage, sampling rates, and quality depend on the source and application.The dataset is represented as D = {τ_1, τ_2, . . . , τ_|D|}.
Problem Statement: Universal trajectory modeling seeks a representation function that supports diverse tasks, geographic contexts, and heterogeneous trajectory sources.The intended capabilities include task adaptability, region independence, and resilience to data-quality variation.

4 Methodology

UniTraj is developed through a global trajectory dataset, preprocessing pipeline, adaptive resampling, self-supervised masking, and an encoder-decoder representation architecture. These components target heterogeneous sampling, missing data, and cross-regional generalization.

4.1 WorldTrace Dataset Construction: WorldTrace is constructed from global motorized GPS traces and processed through normalization, filtering, and map matching to reduce redundancy, noise, and positioning errors.Normalization uses 1 Hz sampling; filtering removes very short, implausibly fast, or anomalous trajectories; map matching aligns points with road networks.
4.1 WorldTrace Dataset Construction: WorldTrace contains approximately 2.45 million trajectories and 8.8 billion GPS points across 70 countries and all inhabited continents.The data spans August 2021 to December 2023 and is normalized to a one-second sampling interval.
4.2 Pre-Training Strategies: Self-supervised Trajectory Masking hides trajectory points so the model learns to reconstruct local and global movement dependencies from incomplete inputs.The strategy includes random, block, and key-point masking to represent diverse missing-data patterns and emphasize structurally significant turns.
4.2 Pre-Training Strategies: Adaptive Trajectory Resampling dynamically adjusts sampling density by trajectory length and enforces consistent intervals to improve robustness across sampling rates and trajectory lengths.Its logarithmic schedule preserves motion patterns, reduces redundancy-related overfitting, and exposes the model to diverse temporal resolutions.

5 Experiments

Experiments evaluate UniTraj across six datasets and multiple trajectory tasks, finding strong transferability, robustness to heterogeneous data, and gains from scale, diversity, and adaptive pre-training.

Experimental Setup: UniTraj is evaluated on six real-world datasets spanning different regions, collection scenarios, quality levels, and motion patterns.The evaluation includes WorldTrace, Chengdu, Xi’an, GeoLife, Grab-Posisi, and Porto.
Trajectory Recovery: UniTraj achieves the lowest fine-tuned recovery errors across all datasets, including MAE 23.23 on GeoLife, a 32.73% reduction versus TrajBERT.Zero-shot results also show transferable spatio-temporal patterns, while fine-tuning further improves performance.
Effect of Dataset Scale and Quality: Increasing WorldTrace from 0.5M to 2.45M trajectories lowers in-domain MAE until roughly 1M trajectories, after which gains diminish and added noise can slightly worsen MAE.A curated high-quality 1M-trajectory subset provides more reliable learning, especially when data volume is limited.
Effect of Dataset Diversity: WorldTrace-trained models generalize better across diverse datasets such as GeoLife and Porto, whereas Chengdu-trained models perform best on similar Xi’an data.The comparison supports WorldTrace as an effective, though non-exclusive, resource for universal trajectory modeling.
Effect of Parameter Settings: Increasing encoder blocks from 2 to 8 reduces MAE before performance plateaus, while a 50% masking ratio produces the best result.These findings indicate diminishing returns from deeper architectures beyond eight blocks and sensitivity to masking choices.
Ablation Study: Dynamic Multi-scale Resampling improves performance on most datasets, especially irregular, lower-quality GeoLife and Grab-Posisi data.The strategy helps separate temporal sampling patterns from regional characteristics and improves transfer across sampling rates.

6 Conclusion

The paper presents UniTraj and WorldTrace as a globally distributed foundation for trajectory analysis that addresses task specificity, regional dependency, and data-quality limitations. It reports broad generalization while identifying geographic and modal coverage, contextual inputs, and efficiency as future priorities.

Conclusion: UniTraj is designed as a robust backbone that generalizes across diverse trajectory tasks and regions, addressing task specificity, regional dependency, and data-quality limitations.WorldTrace supports this goal with 2.45 million trajectories from 70 countries and broad sampling-rate diversity.
Conclusion: WorldTrace provides an open, high-quality global dataset intended to support adaptable and efficient trajectory research.The dataset and model are presented together as a foundation for trajectory analysis.
Future Work: Future work targets underrepresented regions and non-motorized travel, contextual inputs such as road networks and points of interest, and further architectural and pre-training optimization.These directions are stated as ways to improve coverage, predictive accuracy, real-world applicability, performance, and efficiency.

SUPPLEMENTARY MATERIAL

WorldTrace is constructed through global OSM retrieval, standardization, calibration, normalization, and multi-stage filtering to produce consistent trajectory data for modeling.

Data Acquisition and Processing: WorldTrace construction begins with global OSM trajectory retrieval and proceeds through collection, preprocessing, analysis, and dataset-quality assessment.The pipeline uses API-based retrieval and documents the resulting dataset characteristics.
Data Acquisition and Processing: The collection process avoids concurrent crawling to reduce server burden and takes about six months to produce approximately 4.5 million raw traces.Raw trajectories are parsed from GPX into a unified internal format, with descriptive metadata stored separately.
Filtering: Quality control removes anomalous or unsuitable traces using coordinate, duration, length, speed, distance-deviation, and loop-based filters.The filtering stage discards short or stationary traces, implausible speeds, large map-matching deviations, and apparent loops.
Normalization: The pipeline normalizes heterogeneous trajectories by resampling them to a uniform 1 Hz rate and standardizing coordinates to WGS84 with six-decimal precision.Higher-frequency segments retain the first point per second, while lower-frequency segments use linear interpolation.
Calibration: Calibration map-matches GPS points to road networks with a Hidden Markov Model-based approach using point-to-road distance and heading consistency.The process also enriches trajectory points with derived attributes.

A.3 Data Statistics and Analysis

WorldTrace combines broad global coverage with large scale and standardized trajectory statistics. Its distribution is geographically diverse but uneven, and the paper emphasizes privacy-compliant, open-data construction and adaptive modeling strategies.

Overall Statistics: Trajectories average about six minutes and 5.73 km, with average speed 48.0 km/h and 358 points per trajectory.Trajectory lengths range from 32 to over 600 points after processing.
Geographic Distribution: Geographic coverage includes urban, suburban, and rural environments, with dense concentrations in North America, East Asia, and parts of Europe.The United States, China, and Canada lead the top-10 countries by trajectory count, indicating uneven regional density.
Data Governance: WorldTrace is collected under privacy and ethical safeguards, with trajectories anonymized and personally identifiable information excluded.The derived dataset follows the ODbL terms associated with the OpenStreetMap source.
Adaptive Resampling: Adaptive Trajectory Resampling addresses cross-regional and cross-device heterogeneity through Dynamic Multi-Scale Resampling and Interval Consistent Resampling.The strategies are designed to accommodate differences in sampling frequencies, motion patterns, and dataset quality.

B.1.1 Dynamic Multi-Scale Resampling

Dynamic Multi-Scale Resampling uses a logarithmic sampling ratio that preserves all points in short trajectories, compresses longer ones, and keeps sampled lengths bounded. The broader resampling design also regularizes heterogeneous sampling intervals to improve cross-dataset generalization.

Dynamic Multi-Scale Resampling: The logarithmic resampling function preserves all points for n ≤ nmin, reaches Rmin for n ≥ nmax, and bounds the sampled trajectory length.The intermediate ratio decreases logarithmically with trajectory length, while the number of sampled points remains bounded by a finite maximum.
Dynamic Multi-Scale Resampling: The logarithmic strategy slows the decline of R(n) as n increases, producing a theoretically reasonable compromise between information preservation and computational efficiency.Its derivative decreases with trajectory length, matching the stated reduction in information return per additional point.
Dynamic Multi-Scale Resampling: Short trajectories retain fine-grained detail while longer trajectories are compressed to reduce redundancy and computational cost.The strategy is motivated by diminishing information density in longer trajectories and by the need to balance information preservation with processing efficiency.
Dynamic Multi-Scale Resampling: Figure 6 compares the proposed logarithmic dynamic resampling curve with a linear strategy in sampling ratio and sampled points.The comparison covers how each strategy changes sampling as trajectory length increases.
Interval Consistent Resampling: Interval-consistent resampling regularizes temporal sampling across datasets, reducing input-distribution differences and improving generalization across heterogeneous sampling rates.The stated mechanism is that more consistent input distributions reduce the domain gap encountered during learning.

C.5 Implementation Details

UniTraj uses a compact encoder-decoder configuration with rotary positional encoding and training procedures designed to balance trajectory-modeling capacity with computational efficiency.

C.5 Implementation Details: The model uses 8 encoders, 4 decoders, 4 attention heads, 128-dimensional embeddings, and approximately 2.38 million parameters.It supports arbitrary trajectory lengths and pads sequences to 200 points.
C.5 Implementation Details: Table 5 summarizes UniTraj’s general parameter settings.
C.5 Implementation Details: Training uses Adam with mean square error loss, an initial learning rate of 1 × 10^-3, 200 epochs, batch size 1024, scheduling, and validation-based early stopping.Experiments were conducted with PyTorch on NVIDIA A100/L40s GPUs for the foundation model and RTX 2080 Ti GPUs for baselines.

D.1 Datasets

The evaluation spans six diverse real-world trajectory datasets and tests UniTraj under recovery, prediction, and classification settings with zero-shot, fine-tuned, and task-specific comparisons.

D.1 Datasets: UniTraj is evaluated on six datasets covering different collection scenarios, data qualities, motion patterns, and geographic regions.
D.1 Datasets: Trajectory recovery masks 50% of points and compares zero-shot WorldTrace training with fine-tuning on each target dataset against diverse baselines.
D.1 Datasets: Future-trajectory prediction forecasts five points from historical observations on WorldTrace, Chengdu, and GeoLife using the recovery-task baseline and metric settings.
D.1 Datasets: Trajectory classification uses GeoLife and Grab-Posisi, retaining UniTraj’s encoder as a backbone with a classification head and evaluating both frozen-backbone and fine-tuned settings.GeoLife includes walking, bus, bike, and driving; Grab-Posisi includes car and motorcycle.

D.2.4 Trajectory Generation

UniTraj improves trajectory generation when integrated into ControlTraj, including a modest Chengdu gain and stronger transfer performance from Chengdu to Xi’an without retraining.

D.2.4 Trajectory Generation: The generation experiment replaces ControlTraj’s RoadMAE road-segment extraction module with UniTraj’s encoder to test its embedded representation.
D.2.4 Trajectory Generation: Figure 9 compares trajectory generation on Chengdu and transfer to Xi’an, with heatmaps encoding denser regions more brightly than sparser regions.
D.2.4 Trajectory Generation: 0.0152 density error versus ControlTraj’s 0.0171 is achieved when transferring generation from Chengdu to Xi’an without retraining.The result is reported as evidence of cross-region resilience.
D.2.4 Trajectory Generation: The study separately examines dataset scale and quality, geographic diversity, and encoder-decoder configurations as factors affecting UniTraj performance.The architecture search varies encoder, decoder, and attention-head counts, while the diversity study compares WorldTrace and Chengdu training.

E.1 Limitation

UniTraj’s limitations concern uneven geographic coverage, restricted movement types, deployment costs, and reliance on coordinate-temporal inputs without contextual features.

E.1 Limitation: WorldTrace underrepresents parts of Africa and Asia, potentially limiting model performance in those regions.
E.1 Limitation: The focus on motorized movement may restrict generalization to non-motorized mobility such as pedestrian trajectories.
E.1 Limitation: Training and deployment at scale require substantial computational resources, creating challenges for resource-constrained environments.The paper suggests more efficient architectures or distillation as possible directions.
E.1 Limitation: Using only coordinate and temporal information omits contextual features such as road networks, traffic conditions, and points of interest that could improve predictive accuracy.
E.1 Limitation: Broader adoption could raise surveillance concerns and widen disparities between resource-rich and resource-constrained organizations if misused or unevenly accessed.The paper calls for privacy-preserving techniques and equitable access policies.