Probabilistic Ship Trajectory Forecasting using Transformer Models on AIS Data

1 Introduction

• Autonomous-vessel decision support and autonomy require excellent situational awareness, for which anticipation of how situations develop is one of three main steps.The other steps are perception and understanding of the environment.Source: §1 · p. 1
• Future trajectory estimates are needed to plan short-term paths, complementing perception and understanding in situational awareness.AIS broadcasts position, speed, and heading for commercial vessels and some large leisure crafts.Source: §1 · p. 1
• Transformers have shown strong results in pedestrian forecasting, while their computational burden grows quadratically with input sequence length.The Performer uses FAVOR+ for linear complexity, though linear-complexity alternatives often do not match original Transformer performance.Source: §1 · p. 2
• Quantile loss estimates prediction quantiles during training, allowing the network to output a model-aware prediction interval in a single forward pass.This contrasts with approaches that approximate intervals by running a network multiple times.Source: §1 · p. 2

2 Related Work

• Schöller et al. combined an LSTM-GAN with CNN features from a local AIS heat map, finding that prior local vessel movement improved performance and enabled otherwise-unindicated movement predictions.The heat map supplied auxiliary information about how vessels had moved in the surrounding area.Source: §2 · p. 3
• Pedestrian-forecasting work encoded social interactions through pooling, attention, or graph attention, while Transformer forecasting was reported to outperform prior LSTM state of the art even without social modeling.Giuliari et al. used a quantized Transformer for probabilistic prediction intervals.Source: §2 · p. 3
• Existing AIS forecasting approaches may omit either environmental context or the target vessel’s own motion, while sampling-based probabilistic predictions increase inference time.Some methods can predict trajectories through land; clustering methods can produce averages based only on similar vessels’ movements.Source: §2 · p. 3
• The proposed hybrid Transformer approach uses the investigated vessel’s movement and local positional probability distributions, with quantile loss supplying prediction intervals without Monte Carlo sampling.The authors describe this as a marine-vessel trajectory-prediction application of Transformer models not previously investigated to their knowledge.Source: §2 · p. 3

3 Data

• The dataset uses AIS messages recorded in Danish waters, while forecasting inputs retain vessel positions alone to support a more general approach transferable to radar or visual tracking.Speed and heading are omitted, with the paper stating that the model can derive them from positions.Source: §3 · pp. 3–4
• AIS tracks are filtered to remove objects averaging below 0.5 knots and linearly resampled so positions are 20 seconds apart.Temporal gaps are filled from the nearest resampled points; deployment is intended to run only when a new AIS message arrives.Source: §3 · p. 4, Figure 1 · p. 4
• The forecast setup uses the 15 most recent positions and predicts 30 future time steps, corresponding to a 10-minute horizon.The stated horizon reflects a desired opportunity for navigators to react within a 10-minute time to CPA.Source: §3 · p. 4
• Data span one month across heavily trafficked Danish routes selected for sudden fairway turns, and KDE heat maps show AIS traffic tending to follow those fairways.Figure 2 depicts The Great Belt, Oresund, and Falsterborev; white lines mark chart fairways and black spaces mark route separation zones.Source: §3 · p. 5, Figure 2 · p. 5

4.1 Input & Output

• Past positional inputs are embedded in a D-dimensional space using a learnable linear transformation, while the model outputs probabilistic quantiles for the future trajectory.Source: §4.1 · p. 6

4.2 Model Architecture

• The encoder stacks multi-head attention and feed-forward layers, while the decoder uses merged spatial and temporal features to predict a 95% trajectory interval.Source: §4.2 · p. 6
• A local AIS heat map generated with 2D KDE is cropped around each vessel, encoded by a CNN, and concatenated with its temporal features.The combined representation is projected to the model dimension.Source: §4.2 · p. 6
• The Performer replaces scaled dot-product attention, which has quadratic complexity, with FAVOR+ attention, described as a linear-complexity method.Source: §4.2 · p. 7

5 Training Specifications

• 1,636,220 trajectories collected over one month were used for training, with AdamW and a warm-restart learning-rate scheduler.The reported base learning rate was 1e−4 and weight decay was 1e−2.Source: §5 · p. 8
• Both models used dmodel = 128 and eight attention heads, and the spatial CNN received 64 × 64 local heat maps.The latest vessel position was centered in a local heat map covering 8000 × 8000 meters.Source: §5 · pp. 8–9
• Quantile loss trained three outputs at q = [0.025, 0.5, 0.975], corresponding to the reported 95% prediction interval.Source: §5 · p. 9
• Ten percent of the training data was reserved for validation, and training stopped when validation loss converged.Source: §5 · p. 9

• Evaluation compared Gaussian-process, quantile linear-regression, LSTM-GAN, and heat-map/no-heat-map models on three test sets, using heat maps from a different time period than the test data.Source: §6 · pp. 9–10, Table 3 · p. 10
• Both Transformer and Performer averaged ADE around 0.18 km, with average FDE around 0.4 km (about half the baselines) and superior IAcc of ∼91%.Source: §6 · p. 10, Table 4 · p. 11
• Transformer-type models inferred significantly faster than the baselines; LSTM-GAN requires multiple forward passes to approximate its prediction interval.Source: §6 · p. 10
• Despite linear attention, Performer was slower than Transformer; the authors suggest that at 15-step inputs FAVOR+ kernel estimation costs exceed scaled dot-product attention.Source: §6 · p. 11
• Heat maps significantly improved results in Great Belt and Falsterborev, where both Transformer models outperformed all methods, but no-heat-map models did slightly better in unseen Oresund.Source: §6 · p. 11, Table 4 · p. 11
• With Oresund data from another month, heat-map models had lower displacement errors but lower interval accuracy; the authors prefer route-specific pretraining for deployment.Source: §6 · p. 12, Table 5 · p. 12
• In a Great Belt example, heat maps helped forecasts follow the fairway through a sudden turn, while models without spatial features predicted straight; ground truth lay within the proposed model’s interval.Source: §6 · p. 12

7 Conclusion

• KDE heat maps encoded by a CNN add local movement context to Transformer models; quantile loss yields intervals in one pass, at 6–7 ms versus 157 ms for sampling-based GANs.Source: §7 · p. 12
• ADE was 0.182 km for Transformer versus 0.185 km for Performer, while interval accuracy was 91.2% versus 90.7%; both beat LSTM-GAN (0.378 km) and linear and Gaussian process baselines.Despite linear-complexity attention, Performer was not faster than Transformer on the 15-step input sequences.Source: §7 · p. 12
• In the illustrated forecast, including the local heat map yields a far better prediction; orange ellipsoids show per-step 95% prediction intervals.Source: Figure 5 · p. 13
• Heat maps helped most in training-covered areas, while models without them performed slightly better in unseen Oresund, indicating spatial-feature overfitting.The authors recommend training on the deployment area or route.Source: §7 · p. 13
• Further work includes improving spatial-feature generalization to unseen areas, adding sea-chart information, and using positional data from smaller vessels without AIS.Source: §7 · p. 13