Source-linked AI summary
Diffusion Models for Time Series Applications: A Survey
Lequan Lin, Zhengkun Li, Ruikun Li, Xuliang Li, Junbin Gao
TL;DR
Diffusion-based time series methods lack a methodical synthesis despite growing use across forecasting, imputation, and generation. This survey organizes and compares the literature, introduces diffusion fundamentals, and concludes with common limitations and future directions. It presents six forecasting models, four imputation models, and one generation model, while noting application-specific scope boundaries and limitations.
Problem
Existing diffusion-based time series methods lacked a methodical summary and discourse spanning forecasting, imputation, and generation.
Method
The paper surveys diffusion-based time series models, introduces their fundamental formulations, categorizes applications, compares approaches, and discusses connections and limitations.
Results
The survey presents six forecasting models, four imputation models, and one generation model, alongside empirical evidence of good performance.
Takeaways & Limitations
The survey provides a self-contained resource for understanding diffusion-based time series approaches and identifying potential future research directions.
Takeaways & Limitations
PriSTI is limited to single-feature spatiotemporal-graph imputation, leaving extension to more generalized spatiotemporal graphs as a future topic.
Abstract
from arXiv · showhide
Diffusion models, a family of generative models based on deep learning, have become increasingly prominent in cutting-edge machine learning research. With a distinguished performance in generating samples that resemble the observed data, diffusion models are widely used in image, video, and text synthesis nowadays. In recent years, the concept of diffusion has been extended to time series applications, and many powerful models have been developed. Considering the deficiency of a methodical summary and discourse on these models, we provide this survey as an elementary resource for new researchers in this area and also an inspiration to motivate future research. For better understanding, we include an introduction about the basics of diffusion models. Except for this, we primarily focus on diffusion-based methods for time series forecasting, imputation, and generation, and present them respectively in three individual sections. We also compare different methods for the same application and highlight their connections if applicable. Lastly, we conclude the common limitation of diffusion-based methods and highlight potential future research directions.
1 Introduction
Diffusion models generate samples by learning to reverse progressive noise corruption, and have recently been extended to time series forecasting, imputation, and generation. This survey reviews these applications, introduces foundational formulations, compares approaches, and identifies limitations.
- Diffusion models learn to generate samples resembling observed data through deep learning-based generative processes.
- Their core procedure progressively injects Gaussian noise into observed data and learns a reverse transition process that recovers the data.
- Time series diffusion methods address forecasting, imputation of missing values, and generation of additional samples with similar characteristics.
- The survey introduces three predominant diffusion formulations before categorizing models by their major time series functions.
- It reviews existing literature, compares approaches, highlights models serving multiple purposes, and identifies potential limitations.
2 Basics of Diffusion Models
Diffusion models learn generative processes by perturbing data through forward noise injection and reversing that process with learned denoising dynamics. The survey presents DDPMs, SGMs, and SDEs as related formulations that differ in how diffusion and sampling are represented.
- Diffusion models progressively perturb observed data with noise, then recover it through a learned backward denoising process.
- Denoising Diffusion Probabilistic Models: DDPMs implement forward and backward diffusion as discrete Markov chains, with noise levels changing across diffusion steps.
- Denoising Diffusion Probabilistic Models: DDPM sampling starts from standard Gaussian noise and repeatedly applies a parameterized reverse transition to produce the final sample.
- Score-based Generative Models: SGMs estimate Stein scores for noisy data distributions and use annealed Langevin dynamics to iteratively generate samples.
- Stochastic Differential Equations: SDEs generalize discrete DDPM and SGM formulations by modeling diffusion continuously and sampling through a time-reverse SDE or probability flow ODE.
3 Time Series Forecasting
This section frames diffusion-based forecasting as learning predictive distributions for future multivariate time series conditioned on historical context, with covariates and spatial structure sometimes included. It reviews DDPM- and SDE-based approaches, including TimeGrad, ScoreGrad, DSPD, and D3VAE, alongside methods for limited and noisy observations.
- 3.1 Problem Formulation: Multivariate forecasting predicts future observations from multiple interacting features that evolve over time, often using a context window and prediction interval.Forecasting may also incorporate covariates known across the full period.
- 3.1 Problem Formulation: Diffusion-based forecasting learns the joint distribution of the prediction interval conditioned on historical information rather than imposing a strictly constrained target-distribution structure.This formulation addresses a stated shortcoming of earlier generative forecasting models such as WaveNet and conditional normalizing flows.
- 3.2 TimeGrad: TimeGrad injects noise at each predictive time point and progressively denoises through a backward kernel conditioned on historical information encoded by an RNN hidden state.Training minimizes negative log-likelihood, while generation proceeds step by step and feeds each prediction back into the RNN.
- 3.3 ScoreGrad: ScoreGrad extends the diffusion process from discrete steps to continuous SDE integration while retaining a TimeGrad-like conditional target distribution and feature-extraction module.It uses conditional score matching and a predictor-corrector sampler, with VE, VP, and sub-VP SDEs as possible choices.
- 3.4 D3VAE: D3VAE addresses short, noisy historical series with coupled diffusion for data augmentation, a bidirectional auto-encoder, denoising score matching, and latent-variable disentanglement.Its objective also includes prediction-window mean squared error; the coupled process augments context and prediction windows and can mitigate generative and observed-data uncertainty.
- 3.5 DSPD: DSPD replaces independent Gaussian noise in DDPM with discretized Gaussian-process noise, enabling predictions at any future point in a continuous interval and joint generation for multiple time points.Its implementation remains only slightly different from DDPM because the principal change is the noise process.
4 Time Series Imputation
Time series imputation addresses incomplete observations caused by changing collection conditions, sensor failures, and human errors. The survey reviews diffusion-based approaches that condition on observed data, use different generative targets, and extend to spatial and forecasting settings.
- Problem formulation: Missing observations can impair analysis and forecasting, motivating methods that infer unavailable values from incomplete time series.The survey describes collection changes, sensor failures, and human errors as sources of missingness.
- Diffusion-based approaches: Diffusion-based imputation offers flexible assumptions about the true data distribution and includes methods for multivariate and spatiotemporal data.The survey covers three multivariate imputation methods and one STG imputation method.
- Problem formulation: A typical formulation zero-pads incomplete data and uses a binary mask to distinguish observed entries from missing values.The mask contains 1 for known entries and 0 for missing entries.
- CSDI: CSDI applies diffusion and reverse processes to the missing-data matrix while conditioning on observed values.Its reverse transition uses a learnable variance, unlike DDPM's pre-specified variance formulation.
- SSSD: SSSD instead diffuses the whole time series matrix and conditions on a concatenation of zero-padded observations and an observation mask.Its second objective is reported as typically better in practice.
- SSSD: Imputation models can address forecasting by treating future values as a right-side block of missing values, although SSSD underperformed Autoformer in forecasting experiments.This reframes forecasting as a special missing-value pattern but does not guarantee competitive forecasting performance.
- PriSTI: PriSTI’s practical value is limited because it supports STG imputation only when the graph has a single feature.The survey identifies extension to more general STGs as a future research topic.
5 Time Series Generation
Time series generation synthesizes data while modeling temporal dependencies and historical patterns. The survey contrasts diffusion-based generation with GAN and VAE approaches and presents TSGM, which generates latent states before decoding them into time series.
- Problem formulation: Time series generation creates synthetic sequences resembling real-world data while learning temporal dependencies, patterns, and trends.The generation process accounts for historical information when producing later observations.
- Related approaches: GAN-based generators can be unstable and suffer model collapse, while TimeVAE requires a user-defined distribution for its probabilistic process.The survey motivates diffusion as a more flexible alternative in the form of the target distribution.
- TSGM: TSGM conditionally generates each observation from previously generated observations using an encoder, decoder, and conditional score-matching network.The encoder embeds time series into latent space, the score network samples hidden states, and the decoder converts them into time series samples.
- TSGM: TSGM diffuses and samples hidden states rather than directly applying the diffusion process to the observed time series.The sampled hidden states are mapped back to time series through the decoder.
- TSGM: Pre-training the encoder and decoder generally produced better performance than training them simultaneously with the score-matching network.The reported training procedure pre-trains the representation components before score matching.
- Results and limitation: TSGM achieved state-of-the-art sampling quality and diversity against a range of established time series generation methods.The survey also notes that diffusion models are generally more computationally expensive than GANs.
6 Conclusion
The survey organizes diffusion-based methods for time series forecasting, imputation, and generation, while identifying high computational cost and limited theoretical discussion as continuing concerns. It positions the review as a starting point for researchers and future work.
- Scope: The survey presents six forecasting models, four imputation models, and one time series generation model.It also discusses interactions and differences among the reviewed approaches.
- Limitations: Although reviewed models show good empirical performance, diffusion-based methods are usually associated with very high computational costs.This is identified as a common limitation across the surveyed approaches.
- Future directions: The survey reports that deeper discussion and exploration of the rationale behind these theoretically grounded models remains limited.This motivates further research beyond empirical model development.
- Purpose: The paper aims to provide a self-contained starting point for new researchers and inspiration for future directions.Its contribution is a methodical summary and discussion of diffusion approaches for time series.