Learning to Difference: Adaptive Reversible Differencing (AdaRDiff) for Time Series Forecasting

Morad Laglil; Younes Hlal; Marouane El Hadari; Emilie Devijver; Eric Gaussier

Source-linked AI summary

Learning to Difference: Adaptive Reversible Differencing (AdaRDiff) for Time Series Forecasting

Morad Laglil, Younes Hlal, Marouane El Hadari, Emilie Devijver, Eric Gaussier

arXiv:2608.28134v1Aug 28, 2026cs.LG

TL;DR

Long-horizon forecasting is challenged by trend and seasonality, while classical differencing requires manually selected schemes. AdaRDiff learns a reversible differencing operator around forecasting backbones and achieves state-of-the-art accuracy with broad plug-and-play gains across benchmarks and architectures.

Problem
Long-horizon forecasting is difficult because trend and seasonality challenge learning-based models, while classical differencing depends on hand-picked orders, periods, and coefficients.
Method
AdaRDiff learns channel-wise weighted differencing, forecasts the resulting residuals, and autoregressively reconstructs predictions in the original space.
Results
Across eight benchmarks, AdaRDiff attains state-of-the-art accuracy and improves MSE in 56 of 64 backbone–dataset pairs.
Takeaways & Limitations
AdaRDiff provides a model-agnostic way to adapt classical differencing to end-to-end deep forecasting while retaining interpretable temporal-structure weights.
Takeaways & Limitations
The evaluation excludes time series foundation models because their pretraining corpora may overlap with standard benchmarks, precluding fair comparison.

Abstract

from arXiv · show

Reliable long-horizon time series forecasting is an important yet difficult problem. Trends and seasonality introduce complex temporal structure that challenges learning-based forecasting models. Differencing, which subtracts nearby past values to remove such structure, is the classical remedy, but its reliance on hand-picked orders and periods has kept it largely absent from recent deep architectures. We propose \textbf{\underline{Ada}}ptive \textbf{\underline{R}}eversible \textbf{\underline{Diff}}erencing \textbf{(AdaRDiff)}, a generalized differencing approach that uses learnable weights to simplify the series through weighted differencing with previous time instants. This yields stabilized residuals on which forecasting is performed, after which the removed components are restored autoregressively to reconstruct the forecast, capturing trend and seasonality jointly through a single operator. This reconstruction admits a closed-form convolutional expression, which parallelizes on GPU and yields up to $33.7\times$ speedup over the naive recurrence. We furthermore rely on a two-phase training schedule that separates temporal structure discovery from reconstruction learning, as suggested by a theoretical analysis of the gradient when using a linear forecasting model. AdaRDiff attains state-of-the-art forecast accuracy across eight benchmarks spanning electricity, weather, traffic, and energy, at negligible parameter cost. Furthermore, it is designed as a plug-and-play module: integrating AdaRDiff improves eight diverse backbones, from linear models to Transformers, in the large majority of cases, by up to $25.9\%$ with a linear backbone and $18.3\%$ with iTransformer.

1 Introduction

Long-horizon forecasting is difficult because trend and seasonality complicate temporal structure, while classical differencing is effective but rigid. AdaRDiff learns a reversible differencing scheme that simplifies inputs, supports forecasting backbones, and restores the original-space forecast.

Trend and seasonality dominate structured variation in real-world series and make learning-based forecasting difficult.
Classical differencing removes low-order trends or periodic components, leaving simpler residuals for forecasting.
AdaRDiff replaces hand-set differencing coefficients with learnable, channel-wise weights over previous observations.
AdaRDiff wraps around forecasting backbones by forecasting simplified residuals and autoregressively reconstructing predictions in the original space.
Theoretical analysis motivates two-phase training by identifying tension between differencing and reconstruction pathways with a linear backbone.
AdaRDiff is evaluated across eight benchmark datasets and eight forecasting backbones.

2 Related Work

Prior work includes classical statistical forecasting, lightweight decomposition and periodicity methods, and Transformer architectures. AdaRDiff differs by learning one operator that jointly captures trend and multiple seasonalities, while foundation-model comparisons are excluded for fairness concerns.

Classical LTSF methods include exponential smoothing and the Box–Jenkins ARIMA/SARIMA family.
Linear and MLP models: Lightweight linear and MLP methods simplify series through moving-average decomposition, multi-scale mixing, recurrent cycles, or related structural assumptions.
AdaRDiff learns a single differencing operator that jointly captures trend and multiple seasonalities rather than one fixed structural form.
Transformer-based models: Transformer-based methods model long-range dependencies using efficient attention, decomposition, frequency-domain attention, tokens, patches, or periodic shifts.
Time series foundation models are not compared because pretraining corpora may overlap with standard benchmarks, creating a data-leakage concern.

3 AdaRDiff: Adaptive Reversible Differencing

AdaRDiff adaptively differences each channel, forecasts simplified residuals, and reversibly reconstructs forecasts in the original space. Its gradient analysis motivates stability controls and two-phase training to separate temporal structure discovery from reconstruction learning.

Adaptive Differencing and Reconstruction: AdaRDiff applies channel-wise learnable differencing, maps past residuals to future residuals with a backbone, and autoregressively reconstructs forecasts in the original space.The framework is model-agnostic and wraps around forecasting backbones.
Adaptive Differencing and Reconstruction: Learned differencing weights generalize first-order, second-order, and seasonal differencing, while combining lags can remove affine trends and periodic components jointly.Distinct weights are learned per channel to accommodate heterogeneous dynamics.
Efficient Reconstruction: 33.7× speedup is enabled by rewriting autoregressive reconstruction as a GPU-parallelizable convolution rather than applying the recurrence sequentially.Repeated squaring computes the recurrence sequence in O(log H) sequential steps versus O(H) for the naive recurrence.
Gradient Analysis with Linear Backbone: When the reconstruction spectral radius ρ < 1, recurrence coefficients and effective errors remain bounded independently of the forecast horizon; when ρ > 1, they can grow without bound.The stable regime localizes feedback to nearby errors and supports position-specific learning.
Gradient Analysis with Linear Backbone: The differencing gradient reflects data autocovariance and therefore carries learning signals shaped by trend and periodicity.This pathway differs from the reconstruction pathway, whose sensitivity depends on the recurrence dynamics.
Training: AdaRDiff balances temporal-structure fitting against stable reconstruction, motivating optional per-channel gain reparameterization and a two-phase training schedule.The schedule first fits data structure without reconstruction conditioning, then reintroduces reconstruction; it improves accuracy over single-phase training.

4 Experiments

Experiments evaluate AdaRDiff across eight benchmarks, compare it with diverse forecasting baselines and backbones, and analyze training, window-size, efficiency, and learned-weight behavior.

Experimental setup: AdaRDiff is evaluated on eight benchmarks spanning ETT, Weather, Traffic, Electricity, and Solar-Energy datasets.The comparison includes 11 long-term forecasting baselines across linear/MLP, Transformer, and CNN families.
Main results: AdaRDiff-Linear achieves the best results in 15 MSE cases and 11 MAE cases across the eight benchmarks.The AdaRDiff-MLP backbone is second on MSE with 14 top-ranked cases.
Training ablation: Two-phase training lowers test MSE on all eight datasets by 7.7% on average versus single-phase training.The largest gains occur on Electricity (+25.6%) and Traffic (+13.7%).
Plug-and-play evaluation: AdaRDiff improves MSE in 56 of 64 backbone–dataset pairs, with no regression exceeding 3.5%.Improvements reach 25.9% for a linear backbone, 18.3% on iTransformer, 11.8% on SparseTSF, and 10.5% on MixLinear.
Window-size analysis: The best differencing window broadly follows each dataset’s dominant period, while ETTm2 remains stable between P = 24 and P = 96.ETTh1 is optimal at P = 24, whereas Weather and Electricity favor larger windows.
Efficiency analysis: AdaRDiff adds C × P parameters and reaches up to 33.7× reconstruction speedup through parallel closed-form convolution.On Electricity, P = 48 versus P = 168 cuts reconstruction cost by approximately 43× for a 2.9% accuracy loss.

5 Conclusion

AdaRDiff revisits classical differencing as a learnable, reversible, model-agnostic operator. Across eight benchmarks and eight backbones, it attains state-of-the-art accuracy, plug-and-play gains, and interpretable temporal structures.

AdaRDiff is a model-agnostic module that revisits classical differencing through a learnable, reversible operator.
Across eight benchmarks and eight backbones, AdaRDiff attains state-of-the-art accuracy and consistent plug-and-play gains.
Its learned differencing weights provide interpretable evidence of the temporal structures recovered by the operator.
The reconstruction proof establishes a convolutional expression by induction over forecast steps.

A.2 Proof of Proposition 2

The proof characterizes the recurrence sequence h through its characteristic roots and generating function. It distinguishes bounded regimes from regimes where dominant roots force unbounded growth.

The recurrence sequence h is the fundamental solution of a Pth-order constant-coefficient recurrence governed by the characteristic polynomial χ.
When the largest root modulus satisfies ρ < 1, h is absolutely summable and therefore bounded.
When ρ = 1 and all unit-modulus roots are simple, the recurrence remains bounded.
When ρ > 1, or ρ = 1 with a repeated unit-modulus root, dominant modes make sup_n |h_n| infinite.
The generating function satisfies H(x) = 1/D(x), with the constant numerator fixed by h0 = 1 and zero negative-index terms.
The dominant modes cannot cancel permanently because a nonzero unit-modulus sum stays bounded away from zero on an infinite index set.

A.3 Proof of Proposition 3

The proof extends the recurrence analysis to convolution-derived sequences g^(j). Their generating functions inherit the poles of H, with doubled multiplicities in the unstable or repeated-unit-root regime.

The shifted convolution sequence g^(j) obeys the same recurrence as h, forced by the shifted sequence h_{n−j}.
Its generating function satisfies G^(j)(x)D(x) = x^jH(x), linking convolution, recurrence, and a shift by j.
For ρ < 1, the convolution-derived sequence decays geometrically and is summable.
For ρ = 1 with simple unit-modulus roots, the convolution-derived sequence remains bounded.
In the unstable or repeated-unit-root regime, poles double in multiplicity and g^(j) grows at least as c n^(2m−1)ρ^n on an infinite set.

A.4 Proof of Theorem 1

The theorem proves that if the differencing coefficients have ℓ1 norm below one, every characteristic root lies strictly inside the unit circle. Thus the maximal root modulus satisfies ρ < 1.

Assuming a characteristic root has modulus at least one, the triangle inequality yields |λ| ≤ ∥δ∥1 < 1, a contradiction.
Therefore every characteristic root satisfies |λ| < 1, and the largest root modulus is ρ < 1.

A.5 Proof of Proposition 4

The proof reduces the expected gradient-related quantity to a covariance-trace expression whose lag dependence appears only through the autocovariance argument γ(m + j).

Under the population setting, all lagged values needed by differencing and shifted inputs are included in X, with covariance γ(u − v).
With a linear backbone, the residual forecast error is affine in X, written as e = MX + b′ with M and b′ independent of X.
The shifted residual input can be represented as x^(j)_2:L = E_jX, allowing expectations to reduce to tr(NE_jΓ).
Entrywise expansion collapses the shift-index sum using the selector matrix E_j, extracting the relevant lag contribution.
The weights Φ(m) depend on δ, W_b, and b but not j, so the lag enters only through γ(m + j).

B.1 Implementation Details

AdaRDiff applies learnable depthwise-convolution differencing before a backbone and reversible reconstruction afterward, with efficient horizon computation and two-phase training.

Overview: AdaRDiff wraps any forecasting backbone with causal differencing and reconstruction that share channel-wise weights δ ∈ R^(C×P).The module adds C×P parameters and uses depthwise conv1d over channels.
Forward pass: The forward pass predicts residuals after differencing, then reconstructs original-space forecasts by applying the learned sequence h to the predicted residuals plus initial conditions.
Training: Training separates residual-space supervision in Phase 1 from full-pipeline forecast-loss optimization in Phase 2.Phase 1 bypasses reconstruction, while Phase 2 propagates the loss through reconstruction.
Reconstruction components: The reconstruction sequence h is generated from a companion-matrix recurrence, whose repeated-squaring implementation computes a horizon in ⌈log2 H⌉ rounds.Each round doubles the number of known states and produces new states in parallel through one matrix product.
Reconstruction components: Initial conditions are obtained by convolving the final P context values with δ, with taps beyond the available context zero-padded.
Options: The optional RevIN component normalizes each channel before differencing and inverts the transform on the forecast.

C.1 Experiment Settings

Experiments use eight established long-term forecasting benchmarks with standard dataset splits, normalization, and dataset-specific hyperparameter selection.

Datasets: The evaluation covers ETTh1, ETTh2, ETTm1, ETTm2, Weather, Traffic, Electricity, and Solar-Energy benchmarks.The datasets contain periodic structure, including daily cycles and weekly cycles for Electricity and Traffic.
Datasets: ETT datasets use a 6:2:2 train/validation/test split, while the remaining datasets use a 7:1:2 split.
Hyperparameter search: The window size P is tuned over {4, 24, 96} on ETT datasets and set to the dominant period elsewhere.
Hyperparameter search: Differencing initialization, RevIN, and ℓ1-reparameterization are selected through validation-set grid search.

C.2 More experimental results

Additional evaluations show that AdaRDiff remains highly competitive across standardized protocols and look-back lengths, while its forecasts recover dominant periodic and trending structure.

Unified TFB evaluation: AdaRDiff takes the top MSE on 29 of 35 dataset–horizon settings and the top MAE on 22 under the unified TFB protocol.
Unified TFB evaluation: On dataset-average MSE, AdaRDiff improves over the strongest baseline on six of seven datasets, with Traffic as the sole exception.The strongest-baseline comparison includes ETTh2 at 0.331 versus 0.347 and Traffic at 0.397 versus 0.399.
Look-back length: Longer look-back windows generally improve accuracy, with larger benefits at H = 720 than H = 96 on ETTh1 and ETTm2.Extending L from 48 to 720 reduces MSE by 0.089 versus 0.035 on ETTh1 and 0.081 versus 0.031 on ETTm2.
Qualitative forecasts: The forecasts recover Traffic spikes, daily cycles, and Solar-Energy day–night plateaus, while irregular peaks can be underestimated.
Look-back length: Figure 5 reports test MSE against look-back length L for horizons H ∈ {96, 192, 336, 720} across datasets.
Qualitative forecasts: Figure 6 compares observed history, ground-truth futures, and AdaRDiff forecasts across representative channels from six datasets.