PaperlayerView exact revision on arXiv

Source-linked AI summary

Diffusion-TS: Interpretable Diffusion for General Time Series Generation

Xinyu Yuan, Yan Qiao

arXiv:2403.01742v3cs.LGcs.AI

TL;DR

Synthesizing realistic, diverse multivariate time series is important where data access is constrained, while controllable generation must support tasks such as forecasting and imputation. Diffusion-TS combines seasonal-trend decomposition, a transformer architecture, and Fourier-based training to generate interpretable samples and adapt to conditional tasks, achieving competitive performance across a wide range of time-series generation settings.

  • Problem

    The paper addresses the need to synthesize realistic time series, including controllable samples, when access to dynamical data is constrained or privacy-sensitive.

  • Method

    Diffusion-TS combines denoising diffusion with transformer-based seasonal-trend decomposition, Fourier-based objectives, and instance-aware guidance for conditional generation.

  • Results

    Experiments show realistic, diverse, and novel generation with competitive performance across a wide range of time-series tasks, alongside interpretable reconstructions and stable design benefits.

  • Takeaways & Limitations

    The framework provides an interpretable general time-series generator that can be extended to conditional generation without changing the underlying model.

  • Takeaways & Limitations

    At high missing ratios without additional constraints, conditional generation may cause the entire sequence to deviate from the ground truth.

Abstract

from arXiv · show

Denoising diffusion probabilistic models (DDPMs) are becoming the leading paradigm for generative models. It has recently shown breakthroughs in audio synthesis, time series imputation and forecasting. In this paper, we propose Diffusion-TS, a novel diffusion-based framework that generates multivariate time series samples of high quality by using an encoder-decoder transformer with disentangled temporal representations, in which the decomposition technique guides Diffusion-TS to capture the semantic meaning of time series while transformers mine detailed sequential information from the noisy model input. Different from existing diffusion-based approaches, we train the model to directly reconstruct the sample instead of the noise in each diffusion step, combining a Fourier-based loss term. Diffusion-TS is expected to generate time series satisfying both interpretablity and realness. In addition, it is shown that the proposed Diffusion-TS can be easily extended to conditional generation tasks, such as forecasting and imputation, without any model changes. This also motivates us to further explore the performance of Diffusion-TS under irregular settings. Finally, through qualitative and quantitative experiments, results show that Diffusion-TS achieves the state-of-the-art results on various realistic analyses of time series.

1 INTRODUCTION

Diffusion-TS addresses limited access to private dynamical data and shortcomings in existing time-series diffusion methods by combining seasonal-trend decomposition, transformer modeling, and diffusion generation. The framework targets high-quality, interpretable generation and supports controllable conditional tasks through instance-aware guidance.

  • Motivation: Synthesizing realistic time series is motivated by the ubiquity of such data and privacy risks that restrict access to dynamical datasets.The paper identifies applications across finance, medicine, biology, retail, and climate modeling.
  • Problem: Existing time-series diffusion approaches are largely task-agnostic and may require additional information for applications such as imputation and forecasting.The introduction frames high-quality time-series generation as a challenging extension of diffusion models.
  • Problem: Existing solutions poorly preserve temporal dynamics, lack inductive bias for seasonal periodicity, and often provide limited interpretability for conditional and unconditional generation.These limitations are exacerbated for time series with apparent seasonal oscillations.
  • Contribution: Diffusion-TS combines seasonal-trend decomposition with denoising diffusion and a transformer-based architecture to model complicated multivariate and long-term temporal dynamics.The framework learns a disentangled seasonal-trend constitution through an embedding of a deep decomposition architecture and a Fourier-based training objective.
  • Contribution: Instance-aware guidance based on target metrics enables Diffusion-TS to support different controllable conditional generation tasks in a plug-and-play manner.The cited example target metric is reconstruction.
  • Results: Experiments show realistic, diverse, and novel time series under challenging settings, competitiveness with existing diffusion-based downstream methods, and model explainability through case studies.The introduction reports these findings without specifying numerical results.

2 PROBLEM STATEMENT

The problem defines unconditional time-series generation as learning a diffusion-based mapping from Gaussian vectors to samples resembling the dataset. It also models signals through trend, multiple seasonal components, and residual noise or outliers, while requiring controllable generation from p(.|y).

  • Unconditional generation: A time series is represented as X_1:τ ∈ R^τ×d, where τ is the number of time steps and d is the observed-signal dimension.The dataset comprises N samples of time-series signals.
  • Unconditional generation: The unconditional goal is to use a diffusion-based generator to map Gaussian vectors Z_i ∈ R^τ×d×T to signals resembling samples in dataset D_A.T denotes the total diffusion step.
  • Signal decomposition: The time-series model decomposes each observed signal into a trend component, multiple seasonal components, and a remainder containing noise and possible outliers.The notation identifies x_j as observed values, ζ_j as trend, s_i,j as the i-th seasonal component, and e_j as the remainder.
  • Controllable generation: Controllable generation aims to generate samples from the conditional distribution p(.|y), where y is a real-world signal that dictates the synthesis.The control variable may be any real-world signal.

3 DIFFUSION-TS: INTERPRETABLE DIFFUSION FOR TIME SERIES

Diffusion-TS combines diffusion modeling with an interpretable transformer decomposition that separates trend, seasonality, and error to capture complex temporal dynamics. It directly reconstructs clean series with time- and frequency-domain objectives and supports conditional imputation and forecasting through guided sampling.

  • Diffusion process: The diffusion process gradually noises data toward Gaussian noise, while the reverse process denoises samples using a learned neural transition.The forward transition uses β_t ∈ (0, 1), and the reverse transition is parameterized by μ_θ(x_t, t) and Σ_θ(x_t, t).
  • Architecture: Diffusion-TS uses an encoder-decoder transformer with a deep decomposition decoder to capture global correlations and disentangled temporal representations.The decoder blocks combine transformer, feed-forward, and interpretable layers.
  • Interpretable decomposition: Trend, seasonality, and error representations receive distinct constraints: trend models smooth changes, seasonality models periodic patterns, and error captures remaining components.The Fourier layer selects the most significant frequency amplitudes and transforms them back to the time domain for seasonal modeling.
  • Training objective: The model predicts clean series x̂_0(x_t, t, θ) directly and adds a Fourier-transform loss to improve reconstruction in the frequency domain.The loss combines time-domain and frequency-domain terms with weights λ_1 and λ_2; λ = 0.01 in the reweighting strategy.
  • Conditional generation: Conditional Diffusion-TS guides reverse sampling with classifier and reconstruction-based gradients to enforce conditional consistency for imputation and forecasting.The hyperparameters γ and η trade off fluency, conditional consistency, and guidance strength, while repeated gradient updates improve control quality.

4 EMPIRICAL EVALUATION

Diffusion-TS is evaluated on real and simulated datasets through interpretable reconstruction, unconditional synthesis, conditional imputation and forecasting, and cold-start imputation. Across these settings, it generally outperforms baselines, while ablations show that attention, residuals, FFT loss, signal prediction, and interpretability contribute to performance.

  • Datasets: The evaluation uses four real-world and two simulated datasets spanning Stocks, ETTh, Energy, fMRI, and other benchmark settings.Stocks contains six daily Google stock-price features; ETTh records electricity-transformer variables every 15 minutes; Energy contains 28 values.
  • Interpretable reconstruction: With 50 noise steps, Diffusion-TS reconstructs corrupted multivariate series using separate temporal-trend and season-and-error components.The reconstruction outputs aim to restore ground truth from corrupted inputs across Sines, Stocks, and ETTh examples.
  • Unconditional generation: On 24-length generation, Diffusion-TS consistently achieves higher-quality synthetic samples than baselines across almost all metrics and improves discriminative score by an average of 50% across six datasets.The evaluation compares against TimeVAE, Diffwave, TimeGAN, Cot-GAN, and DiffTime.
  • Conditional generation: Diffusion-TS conditional generation supports both imputation and forecasting, using geometrically distributed masks to control missing-sequence lengths and missing ratio.Conditional experiments compare Diffusion-TS-G with CSDI, Diffwave, and Diffusion-TS-R; seasonal-trend decomposition can facilitate infilling but may deviate from ground truth at high missing ratios without constraints.
  • Cold-start imputation: Using 10/25/50/75% of each dataset for training and leaving 10% ∼20% values missing, the model is tested for performance under regular data deficits.This experiment evaluates whether Diffusion-TS avoids degradation during cold starts with little or no available training data.
  • Ablation study: Ablations show that removing attention or residuals often causes large performance drops, while FFT loss, signal prediction, and interpretable design are generally important.On high-frequency, high-dimensional fMRI, the interpretable-only network provides the most significant improvement.

5 CONCLUSIONS … C ADDTIONAL EXPERIMENTAL RESULTS

The paper presents Diffusion-TS as a DDPM-based framework for general time-series generation, combining time-series-specific loss design with transformer-based decomposition and conditional sampling. The surrounding sections situate this approach among generative modeling, interpretable decomposition, DDPM foundations, and additional experiments.

  • 5 CONCLUSIONS: Diffusion-TS combines a DDPM-based generator with time-series-specific loss design and a transformer-based deep decomposition architecture.The method is proposed for general time-series generation.
  • Time Series Generation.: Time-series synthesis has predominantly used GANs, with many architectures employing recurrent networks to model temporal dynamics.The passage identifies time-series generation as a challenging generative modeling task.
  • Time Series Generation.: Alternative approaches address adversarial instability through energy models, normalizing flows with spectral filters, and variational autoencoders.The cited examples include stepwise-decomposable energy modeling, exact likelihood optimization, and TimeVAE.
  • Denoising Diffusion Probabilistic Models.: GuidedDiffTime removes embedding-based side information and handles constraints such as trends or fixed values without retraining.This passage describes the method in the context of related DDPM-based time-series work.
  • B DENOISING DIFFUSION PROBABILISTIC MODELS (DDPMS): DDPM generation reverses a gradual noising process, beginning from noise and using a learned reverse process to recover the original data.The forward process progressively corrupts x0 through a fixed Markov chain, while xT is isotropic Gaussian noise.
  • C ADDTIONAL EXPERIMENTAL RESULTS: The paper reports additional experiments omitted from the main body because of limited space.These experiments are presented in the additional-results section.

C.1 DETAILED EXPERIMENTS FOR LONG-TERM TIME SERIES GENERATION · C.2 ADDITIONAL 2-DIMENSIONAL PLOTS ON ETTH DATASET

The detailed experiments assess Diffusion-TS for long multivariate time-series generation using discriminative and predictive scores, while PCA and t-SNE visualizations compare generated and real distributions on ETTh. Diffusion-TS samples show significantly greater overlap with the original data than TimeGAN.

  • C.1 DETAILED EXPERIMENTS FOR LONG-TERM TIME SERIES GENERATION: The long-term generation experiments add discriminative and predictive scores to further assess performance stability on long multivariate time-series data.The supplied passage does not provide the corresponding numerical results.
  • C.1 DETAILED EXPERIMENTS FOR LONG-TERM TIME SERIES GENERATION: Table 3 presents detailed results for long-term time-series generation, with bold formatting indicating the best performance.No table values are included in the supplied passage.
  • C.1 DETAILED EXPERIMENTS FOR LONG-TERM TIME SERIES GENERATION: Figure 7 uses t-SNE plots to compare real and generated series of lengths 24, 64, 128, and 256 synthesized by Diffusion-TS and TimeGAN.Red dots denote real data instances, while blue dots denote generated samples.
  • C.1 DETAILED EXPERIMENTS FOR LONG-TERM TIME SERIES GENERATION: Figure 8 uses PCA plots to compare real and generated series of lengths 24, 64, 128, and 256 synthesized by Diffusion-TS and TimeGAN.Red dots denote real data instances, while blue dots denote generated samples.
  • C.2 ADDITIONAL 2-DIMENSIONAL PLOTS ON ETTH DATASET: On the ETTh dataset, PCA and t-SNE analyses visualize how generated time-series distributions cover real data distributions and compare their diversity.The analyses specifically compare Diffusion-TS and TimeGAN.
  • C.2 ADDITIONAL 2-DIMENSIONAL PLOTS ON ETTH DATASET: Diffusion-TS synthetic samples have significantly more overlap with the original data than those produced by the SOTA method.The comparison is based on the supplied PCA and t-SNE figures and identifies the SOTA method as TimeGAN in the accompanying figure descriptions.

C.3 ADDITIONAL EXPERIMENTS FOR IMPUTATION AND FORECASTING PERFORMANCE COMPARISON · C.4 IRREGULAR TRAINING

The additional experiments evaluate Diffusion-TS on imputation and forecasting against further baselines, while reporting qualitative prediction examples and irregular-training results. The study uses MuJoCo imputation, solar and stock forecasting, and four real-world datasets for irregular training.

  • C.3 ADDITIONAL EXPERIMENTS FOR IMPUTATION AND FORECASTING PERFORMANCE COMPARISON: Imputation experiments repeat the Alcaraz & Strodthoff (2022) setting on MuJoCo sequences of length 100 against additional baselines.Baseline results were collected from the original publications.
  • C.3 ADDITIONAL EXPERIMENTS FOR IMPUTATION AND FORECASTING PERFORMANCE COMPARISON: Imputation performance is evaluated using averaged MSE for a single imputation per test sample over 5 trials.The results are presented in Table 4 for the MuJoCo dataset.
  • C.3 ADDITIONAL EXPERIMENTS FOR IMPUTATION AND FORECASTING PERFORMANCE COMPARISON: Table 4 reports MuJoCo imputation MSE values using bracket notation for least-significant-digit uncertainty and a 1e-3 scale.For example, 0.572(12) denotes 0.572 ± 0.012.
  • C.3 ADDITIONAL EXPERIMENTS FOR IMPUTATION AND FORECASTING PERFORMANCE COMPARISON: Forecasting experiments include solar data, with results summarized in Table 5.Table 5 is titled “Time series forecasting results for the solar data set.”
  • C.3 ADDITIONAL EXPERIMENTS FOR IMPUTATION AND FORECASTING PERFORMANCE COMPARISON: Figures 9 and 10 show solar and stock prediction examples in which Diffusion-TS produces high-quality forecasts close to the ground truth.Figure 9 visualizes forecasting on the Solar dataset.
  • C.4 IRREGULAR TRAINING: Irregular-training results are provided for four real-world datasets from the main paper in Figure 11.These results are presented specifically for the irregular training setting.

C.5 DISENTANGLEMENT VALIDATION ON SYNTHETIC DATASET · C.6 HYPERPARAMETER TUNING AND SENSITIVITY

On synthetic time series, Diffusion-TS recovers disentangled trend and seasonal components that closely resemble the ground truth. The study also performs limited hyperparameter tuning across datasets and evaluates guidance strength, model size, and computational details.

  • C.5 DISENTANGLEMENT VALIDATION ON SYNTHETIC DATASET: Diffusion-TS is evaluated on 2,000 synthetic series of 160 training time steps, each composed of seasonal and trend-cyclical parts.The experiment visualizes ground-truth and learned trend and seasonality patterns.
  • C.5 DISENTANGLEMENT VALIDATION ON SYNTHETIC DATASET: The learned disentangled components are reported to be very similar to the synthetic ground-truth trend and seasonality patterns.This visual comparison validates the interpretability associated with Equation 2.
  • C.6 HYPERPARAMETER TUNING AND SENSITIVITY: The study conducts limited hyperparameter tuning to identify default settings that perform well across datasets.The considered ranges include batch size [32; 64; 128], attention heads [4; 8], basic dimension [32, 64, 96, 128], diffusion steps [50, 200, 500, 1000], and guidance strength [1., 1e-1, 5e-2, 1e-2, 1e-3].
  • C.6 HYPERPARAMETER TUNING AND SENSITIVITY: Guidance-strength sensitivity is evaluated using mean squared errors across γ values from 1. to 1e-3.Table 6 compares γ ∈[1., 1e-1, 5e-2, 1e-2, 1e-3].
  • C.6 HYPERPARAMETER TUNING AND SENSITIVITY: The hyperparameter analysis includes a separate comparison of model size.The model-size results are presented in Table 7.
  • C.6 HYPERPARAMETER TUNING AND SENSITIVITY: The study reports hyperparameters, training details, and compute resources used for each model.These details are compiled in Table 8.

C.7 ABLATION STUDY

The ablation studies compare Diffusion-TS with variants removing its Fourier loss, seasonal-trend design, Transformer components, or direct reconstruction parameterization. Removing any disentangled representation component reduces performance, indicating that the proposed representations jointly enhance generative performance.

  • Model architecture and options: The study compares full Diffusion-TS against variants without Fourier loss, seasonal-trend design, encoder/self-attention, or direct reconstruction parameterization.The variants are evaluated on 24-length time series in Table 9.
  • Decomposition: A fine-grained decomposition ablation is conducted on the MuJoCo dataset to assess the disentangled framework.The results are reported in Table 10.
  • Decomposition: Removing any component causes performance to drop, validating that all disentangled representations contribute to generative tasks and jointly improve the final model.This conclusion is based on the ablation results in Table 10.

D REPLACE-BASED IMPUTATION WITH DIFFUSION MODEL … G EXPERIMENT DETAILS

The paper presents replace-based diffusion imputation, specifies Diffusion-TS’s encoder–decoder architecture, and introduces reconstruction-guided sampling with multiple gradient updates and replacement of observed values. The supplied passages do not include experiment details.

  • D REPLACE-BASED IMPUTATION WITH DIFFUSION MODEL: Replace-based imputation conditions diffusion sampling on observed values while generating missing values to produce an intact sample with the correct conditional and marginal distributions.Known dimensions are replaced with exact forward-process samples at each iteration, while unknown dimensions are updated through denoising.
  • D REPLACE-BASED IMPUTATION WITH DIFFUSION MODEL: The method defines observed and missing components of an irregular sample as xob and xta, respectively, and samples conditionally from the jointly trained diffusion model pθ(x).Known and unknown dimensions are represented by Ω(xt) and ¯Ω(xt).
  • E MODEL DETAILS: Diffusion-TS uses a sequence encoder and an interpretable decoder within a transformer-inspired architecture.The overview describes the framework as containing two parts: a sequence encoder and an interpretable decoder.
  • E MODEL DETAILS: Each encoder block combines full attention with a feed-forward network, while each decoder transformer block combines full attention with cross attention.Decoder cross attention incorporates encoding information.
  • F ALGORITHMS: Reconstruction-guided sampling takes multiple gradient steps per diffusion step to improve generative quality and uses more updates at large diffusion step t.The reverse process is described as creating in early stages and smoothing in later stages, motivating more guidance during early creation.
  • F ALGORITHMS: At each reverse step, the algorithm predicts x̂0, computes reconstruction and transition losses, updates x̂0 by η∇xt(L1 + γL2), and samples xt−1.The supplied algorithm then applies Replace(xa, xt−1, t) before continuing to the next step.

G.1 DATASETS … H ADDITIONAL VISUALIZATIONS

The supplementary sections document dataset statistics, baseline implementations, evaluation metrics, optimized conditional sampling, and additional visualizations. They define discriminative, predictive, Context-FID, and correlational measures for assessing synthetic time series.

  • G.1 DATASETS: Algorithm 2 optimizes conditional sampling by updating x_t with gradients of reconstruction and trade-off losses before replacing observed values.The procedure samples x_T, iterates over diffusion steps and K[t], computes L1 and L2, and performs replacement at each step.
  • G.1 DATASETS: Table 11 reports the statistics of the datasets, which are available online.The paper provides dataset details in Table 11 and an online-access statement.
  • G.2 BASELINES: The generative experiments apply and optimize accessible source codes, including CSDI and DiffTime.The supplementary material lists CSDI / DiffTime with a repository link.
  • G.2 BASELINES: Baseline configurations match model capacity by using a 3-layer GRU for GAN methods and roughly equal trainable parameter sizes for TimeVAE and Diffwave.CSDI uses 4 residual layers, 64 residual channels, and 8 attention heads, with convolution kernel sizes changed to follow DiffTime.
  • Discriminative & Predictive score.: The discriminative score is |accuracy − 0.5|, while the predictive score is test-set mean absolute error between predictions and ground truth.Both scores reuse TimeGAN’s experimental settings, with 2-layer GRU classifier and sequence-prediction models.
  • Context-FID score.: Context-FID replaces the original FID Inception model with TS2Vec time-series representations, and lower scores indicate distributions closer to real data.The cited work reports that the lowest-scoring models correspond to the best-performing downstream models.
  • Correlational score.: The correlational score measures correlation between real and synthetic data using covariance estimates for the ith and jth features.The supplementary text introduces the covariance estimation before presenting the correlation metric.
  • H ADDITIONAL VISUALIZATIONS: Additional visualization and distribution outcomes are presented in Figures 14 and 15.These figures extend the reported qualitative analyses with supplementary visual and distribution results.

I ADDITIONAL EXAMPLES

The section presents additional Energy-dataset examples illustrating time-series imputation and forecasting. It includes visual comparisons of Diffusion-TS and Diffwave under varying missingness and forecasting windows, with median imputations and 5%–95% quantile ranges.

  • Additional examples: Additional examples illustrate time-series imputation and forecasting on the Energy dataset in Figures 16–19.The examples cover different missingness levels and forecasting windows.
  • Multivariate comparisons: Figures 14 and 15 visualize multivariate-dataset results through t-SNE embeddings and distributions for original versus synthetic data.Figure 14 uses red for original data and blue for synthetic data; Figure 15 uses a blue solid line and yellow dotted line.
  • Imputation: For Energy imputation with 90% missingness, Figure 16 compares Diffusion-TS and Diffwave across all 28 features.Observed values are red crosses, ground-truth targets are blue circles, and method outputs use green and gray.
  • Imputation: For Energy imputation with 50% missingness, Figure 17 compares Diffusion-TS and Diffwave across all 35 features.Each method is shown with a median line and shaded 5% and 95% quantiles.
  • Forecasting: Figures 18 and 19 compare Energy forecasting for 36- and 24-step windows, respectively, using all 28 features.The figures show observed values, ground-truth targets, and median predictions with 5% and 95% quantile shading for Diffusion-TS and Diffwave.
Loading 2403.01742v3…