Source-linked AI summary
Shape and Time Distortion Loss for Training Deep Time Series Forecasting Models
Vincent Le Guen, Nicolas Thome
TL;DR
Multi-step forecasting for non-stationary signals must capture both trajectory shape and the timing of abrupt changes, which standard losses do not explicitly separate. The paper introduces differentiable DILATE losses and efficient optimization procedures, and reports stronger shape and timing performance than MSE-based training while remaining competitive on MSE.
Problem
Multi-step forecasting of non-stationary signals requires accurate prediction of trajectories containing abrupt changes in both shape and temporal location.
Method
DILATE trains deep forecasting models with separate differentiable shape and temporal terms, supported by custom back-propagation and a smooth temporally constrained DTW variant.
Results
DILATE significantly outperforms MSE-trained models on shape and temporal distortion metrics while maintaining very good MSE performance across datasets and architectures.
Takeaways & Limitations
DILATE provides a model-agnostic training objective that improves shape and timing performance, including over specialized state-of-the-art forecasting architectures.
Takeaways & Limitations
The paper leaves probabilistic forecasting and relaxed-supervision settings for future work.
Abstract
from arXiv · showhide
This paper addresses the problem of time series forecasting for non-stationary signals and multiple future steps prediction. To handle this challenging task, we introduce DILATE (DIstortion Loss including shApe and TimE), a new objective function for training deep neural networks. DILATE aims at accurately predicting sudden changes, and explicitly incorporates two terms supporting precise shape and temporal change detection. We introduce a differentiable loss function suitable for training deep neural nets, and provide a custom back-prop implementation for speeding up optimization. We also introduce a variant of DILATE, which provides a smooth generalization of temporally-constrained Dynamic Time Warping (DTW). Experiments carried out on various non-stationary datasets reveal the very good behaviour of DILATE compared to models trained with the standard Mean Squared Error (MSE) loss function, and also to DTW and variants. DILATE is also agnostic to the choice of the model, and we highlight its benefit for training fully connected networks as well as specialized recurrent architectures, showing its capacity to improve over state-of-the-art trajectory forecasting approaches.
1 Introduction
The paper targets multi-step forecasting of non-stationary signals with abrupt regime changes, where standard MSE can misvalue predictions. It introduces DILATE, which separates shape and temporal localization errors and improves evaluation-aligned training.
- Problem: Multi-step forecasting must describe complete future trajectories for non-stationary signals that may undergo abrupt regime changes.Applications include electricity consumption, renewable energy, traffic flow, ECG analysis, and stock-market prediction.
- Problem: Deep neural networks model complex nonlinear time dependencies, but most methods train with MSE or related losses.
- Motivation: Predictions with similar MSE can differ substantially: anticipating a sharp drop is preferable to missing the regime change, even with slight timing or amplitude error.
- DILATE: DILATE disentangles shape and temporal localization errors and assigns smaller losses to predictions that capture the change than to one that misses it.
- Results: Experiments show DILATE outperforms MSE on shape and temporal distortion metrics while retaining strong MSE performance across datasets and architectures.The method also improves over state-of-the-art forecasting approaches on shape and timing metrics.
2 Related work
Prior forecasting methods include linear models for stationary dynamics, deep architectures for nonlinear dependencies, and differentiable DTW surrogates. These approaches leave a need to address both shape and temporal localization, while many relevant evaluation metrics are non-differentiable.
- Traditional forecasting: ARIMA, exponential smoothing, and related state-space models handle linear dynamics and stationary series, but abrupt distribution changes violate stationarity.
- Deep forecasting: Deep forecasting research uses recurrent architectures, attention mechanisms, tensor factorizations, and state-space combinations for multi-step prediction and uncertainty modeling.This paper focuses on deterministic multi-step forecasting.
- Evaluation metrics: MAE, MSE, and variants dominate deep-model training and evaluation, while DTW, change-point metrics, ramp scores, and temporal distortion metrics assess shape or timing.
- Evaluation metrics: Relevant shape and temporal metrics are generally non-differentiable, preventing their direct use as losses for training deep neural networks.
- DTW-based losses: Smooth DTW losses focus on shape but ignore temporal localization because DTW is invariant to elastic distortions.
3 Training Deep Neural Networks with DILATE
DILATE trains forecasting models with separate shape and temporal-distortion terms, using differentiable dynamic-programming losses and efficient custom back-propagation. Its smooth variant connects shape-temporal penalization with temporally constrained DTW.
- DILATE objective: DILATE compares predicted and ground-truth trajectories through separate shape and temporal misalignment terms balanced by α.The objective is defined over predicted and actual future trajectories.
- Shape term: The shape term uses a smoothed DTW formulation to measure structural dissimilarity while remaining differentiable.A smooth-min operator replaces the non-differentiable DTW minimum.
- Temporal term: The temporal term penalizes misalignment by measuring deviation of the optimal DTW path from the first diagonal.This is based on a generalized Time Distortion Index.
- Temporal constraints: Temporal constraints can be encoded through the penalty matrix, including hard bands or distance-dependent weighting.The matrix can incorporate prior knowledge by penalizing some temporal offsets more heavily.
- Efficient implementation: Custom dynamic-programming implementations make the losses tractable, with forward and backward time complexity O(k^2).The implementations reuse intermediate results and speed optimization compared with vanilla auto-differentiation.
- Smooth variant: DILATE’s smooth variant combines shape and temporal criteria, whereas the main loss separates them and applies temporal penalization to the unconstrained optimal DTW path.Experiments compare the separated formulation with its tangled variant.
4 Experiments
Experiments evaluate DILATE on three non-stationary forecasting datasets using common architectures and compare it with MSE, smooth DTW, constrained variants, and state-of-the-art models. Across evaluations, DILATE improves shape and temporal metrics, while its custom backward pass accelerates longer-horizon training.
- Experimental setup: Experiments use Synthetic, ECG5000, and Traffic datasets with fully connected and Seq2Seq GRU architectures for multi-step forecasting.Forecast horizons are k = 20, 56, and 24, respectively; comparisons use the same architecture to isolate the training loss.
- DILATE forecasting performances: DILATE outperforms MSE on shape in all six experiments, with significant differences in 5/6, and performs better on time in all experiments.DILATE is equivalent to MSE on MSE in 3/6 experiments.
- DILATE forecasting performances: Compared with smooth DTW, DILATE has similar shape performance but is significantly better on time and MSE in all experiments.Shape results comprise 2 significant improvements, 1 significant drop, and 3 equivalent performances for DILATE.
- DILATE forecasting performances: Qualitative forecasts show MSE producing non-sharp predictions, DTW producing temporally misaligned sharp predictions, and DILATE combining sharp shape with precise timing.These examples cover Synthetic, ECG5000, and Traffic datasets.
- DILATE forecasting performances: On external metrics, DILATE is always better than MSE in shape and better or equivalent in time, while it is equivalent to DTWγ in shape in 2/3 experiments.For the Hausdorff time metric, DILATE is always better than DTWγ.
- DILATE Analysis: The custom backward pass reaches ×20 speedup for 20-step forecasts and up to ×35 for 100-step forecasts, while α controls the shape–time trade-off.When α approaches zero, temporal-only optimization causes MSE and shape errors to explode; α = 1 reduces LDILATE to DTWγ.
5 Conclusion and future work
The paper concludes that DILATE is a differentiable loss for deep multi-step forecasting that combines shape accuracy with temporal localization for non-stationary signals. It performs comparably to MSE on MSE while outperforming it and MSE-trained state-of-the-art methods on shape and timing measures; probabilistic and weakly supervised extensions remain future work.
- DILATE combines shape and temporal-localization terms to forecast non-stationary signals with sudden changes.
- DILATE is comparable to MSE on MSE evaluation but performs far better on several shape and timing metrics.
- Future work includes extending DILATE to probabilistic forecasting and relaxed supervision using semi-supervised or weakly supervised training.