PaperlayerView exact revision on arXiv

Source-linked AI summary

Utilizing Image Transforms and Diffusion Models for Generative Modeling of Short and Long Time Series

Ilan Naiman, Nimrod Berman, Itai Pemper, Idan Arbiv, Gal Fadlon, Omri Azencot

arXiv:2410.19538v1cs.LGcs.CV

TL;DR

Time-series generators are commonly specialized for short or long sequences, creating a need for a unified varying-length framework. The paper transforms sequences with invertible maps into images and applies vision diffusion models, achieving state-of-the-art results across short-to-ultra-long and conditional benchmarks.

  • Problem

    Existing time-series generative models typically target either short or long sequences because of recurrent gradient issues, transformer costs, and state-space-model expressiveness limits.

  • Method

    The framework uses invertible time-series-to-image transforms, including delay embedding and STFT, followed by a vision diffusion model and inverse mapping.

  • Results

    The approach achieves state-of-the-art results against strong baselines for sequence lengths [24, 17.5k], including a new very-long-sequence benchmark.

  • Takeaways & Limitations

    A single image-based diffusion framework supports short-, long-, and ultra-long-range time-series generation as well as interpolation and extrapolation.

Abstract

from arXiv · show

Lately, there has been a surge in interest surrounding generative modeling of time series data. Most existing approaches are designed either to process short sequences or to handle long-range sequences. This dichotomy can be attributed to gradient issues with recurrent networks, computational costs associated with transformers, and limited expressiveness of state space models. Towards a unified generative model for varying-length time series, we propose in this work to transform sequences into images. By employing invertible transforms such as the delay embedding and the short-time Fourier transform, we unlock three main advantages: i) We can exploit advanced diffusion vision models; ii) We can remarkably process short- and long-range inputs within the same framework; and iii) We can harness recent and established tools proposed in the time series to image literature. We validate the effectiveness of our method through a comprehensive evaluation across multiple tasks, including unconditional generation, interpolation, and extrapolation. We show that our approach achieves consistently state-of-the-art results against strong baselines. In the unconditional generation tasks, we show remarkable mean improvements of 58.17% over previous diffusion models in the short discriminative score and 132.61% in the (ultra-)long classification scores. Code is at https://github.com/azencot-group/ImagenTime.

1 Introduction

Existing time-series generators are usually specialized for short or long sequences, motivating a unified diffusion framework that represents sequences as images. The proposed approach targets varying lengths and reports state-of-the-art results across unconditional and conditional benchmarks.

  • Existing generators commonly target either multivariate short-term or univariate long-range sequences, leaving a unified framework as an open question.
  • RNNs face vanishing and exploding gradients, transformers incur O(L^2) costs, and state space models remain underexplored for long-range generation.
  • Representing sequences as images addresses these challenges while enabling vision diffusion models and scaling a 256 × 256 image to sequences up to 65k.
  • The framework uses invertible time-series-to-image transforms to process short-, long-, and ultra-long-range sequences without significant architectural or training modifications.
  • The approach achieves state-of-the-art results against strong baselines across sequence lengths [24, 17.5k], including a new very-long-sequence benchmark.

2 Related work

Prior work has transformed time series into images and developed diffusion, adversarial, variational, and state-space generators. However, diffusion-based image representations for general time-series generation across multiple tasks remain underexplored.

  • Time-series-to-image transforms such as Gramian Angular Fields, Recurrence Plots, and Line Graphs enable computer-vision methods for classification and imputation.
  • STFT-based diffusion research and time-series images combined with Wasserstein GANs provide related foundations, but general diffusion modeling remains limited.
  • Unconditional generation, interpolation, and extrapolation of general time series represented as images within diffusion models remain largely underexplored.
  • GANs, VAEs, and state-space models have been applied to time-series generation, while GANs remain vulnerable to unstable training and mode collapse.

3 Background

The paper formulates unconditional time-series generation and introduces invertible delay-embedding and STFT representations alongside the diffusion-process background. These transforms preserve recoverability while adapting sequences to image-based modeling.

  • The task is to learn a distribution ˜p(x) resembling an unknown p(x) from observed samples x ∈ R^L×K, where L is length and K is feature count.
  • Time series to image transforms: The delay embedding arranges a univariate sequence into image columns using skip and column-dimension parameters, padding as needed and enabling inverse extraction.
  • Time series to image transforms: A 256 × 256 delay-embedding image can encode sequences up to 65k, supporting long-range inputs.
  • Time series to image transforms: STFT maps signals to frequency-domain images using rolling windows, stores real and imaginary channels, and supports reverse transformation with negligible information loss.
  • Diffusion models: Diffusion models add noise through a forward SDE and learn a score estimator to support reverse-time sampling or a deterministic probability-flow ODE.

4 Method

ImagenTime transforms time series into images, denoises them with a vision diffusion model, and maps generated clean images back to sequences. It uses efficient invertible transforms and EDM to support sampling across sequence lengths.

  • ImagenTime consists of a time-series-to-image module, a diffusion model, and an image-to-time-series component, with the diffusion model as the learnable part.
  • The pipeline transforms x ∈ R^L×K into x_img ∈ R^C×H×W, denoises the image, and during inference starts from N(0, I) before applying the inverse transform.
  • The framework selects transforms that are efficient, informative, scalable across sequence lengths, and equipped with closed-form inverses.
  • The model uses EDM, whose second-order reverse ODE provides a balance between rapid sampling and generation quality, with low-FID images reported in 35 NFEs.
  • Training applies delay embedding or STFT to batches before learning the EDM score, while inference samples through the reverse ODE.

5 Experiments

The framework is evaluated on unconditional and conditional generation benchmarks spanning short, long, and ultra-long time series, plus image-transform ablations. It achieves strong results across sequence lengths and tasks, while remaining robust to transform choice.

  • Short-Term Unconditional Generation: On the short unconditional benchmark, the framework reaches state-of-the-art results on all datasets and metrics, improving MuJoCo over the second-best method by 88% discriminatively and 21% predictively.The qualitative t-SNE comparison also reports generated point clouds closer to real data than GT-GAN.
  • Long-Term Unconditional Generation: The method outperforms competing techniques in almost all long-term unconditional cases, with weaker results only on NN5 Daily prediction and Temp Rain marginal scores.The authors note especially strong classification and prediction scores for Temp Rain, FRED-MD, and NN5 Daily.
  • Ultra-Long Unconditional Generation: The method consistently achieves superior ultra-long unconditional results, including a KDD-Cup prediction score of .001 versus .049 for LS4.The ultra-long datasets contain sequences of 17,544 and 10,920 steps, supporting the claimed scalability to very long sequences.
  • Conditional Generation of Time Series: Conditional generation is strong across varying lengths: short interpolation is ≈4 times better than CRU, short extrapolation improves SOTA by ≈30%, and ultra-long interpolation is ≈4 times better than ODE-RNN.Long interpolation improves by two orders of magnitude on Physionet and one order on USHCN.
  • Ablation Studies: Image-transform ablations show robustness: delay embedding and STFT are slightly better on short and long sequences, while other transforms remain reasonably effective; GAF does not scale to long sequences.The study also reports robustness to image resolution and hyperparameter choices, with training and inference time comparable to LS4.

6 Conclusion

The framework represents time series as images and applies diffusion modeling to support short-, long-, and ultra-long-range generation. It uses invertible transformations with corresponding reconstruction procedures, while sharing an EDM backbone across experiments.

  • 6 Conclusion: The framework uses image representations of time series to apply diffusion modeling across varying sequence lengths.Its design targets short-, long-, and ultra-long-range sequences within one framework.
  • Folding: Folding converts a sequence into an image row by row, pads when necessary, and reconstructs the sequence by unfolding the unpadded region.The transformation scales to very long sequences and can be viewed as a delay-embedding instance.
  • Delay Embedding: Delay embedding organizes univariate sequence values into image columns using skip value m and column dimension n, with zero padding for network inputs.For multiple channels, the matrices are concatenated along channels; reconstruction follows marginal progressions through the columns.
  • STFT: STFT converts time-domain signals into frequency-domain images while retaining real and imaginary components rather than only magnitude spectra.The output uses doubled channels, with image dimensions determined by n_fft and hop_length.
  • Diffusion Backbone: All experiments use the EDM diffusion architecture with AdamW and 1000 training epochs, although convergence typically occurs after 300–500 epochs.Task-specific settings and evaluation protocols vary by experiment.

B.1 Short-term unconditional generation.

The appendix describes datasets, hyperparameters, and evaluation procedures spanning short, long, ultra-long, and conditional time-series generation. Evaluation uses predictive, discriminative, classification, prediction, and MSE-based measures across these settings.

  • Short-term unconditional generation: Short-term unconditional generation uses four length-24 synthetic and real-world datasets, including Stocks, Energy, MuJoCo, and Sine.The datasets cover financial random walks, correlated appliance energy, simulated physics, and synthetic sine functions.
  • Evaluation: Short-term evaluation uses predictive and discriminative scores, where higher scores indicate greater utility or distributional similarity.The predictive score trains an independent predictor on generated data, while the discriminative score uses a proxy discriminator.
  • Long-term unconditional generation: Long-term unconditional experiments use FRED-MD, NN5 Daily, and Temperature Rain, selected using average 1-lag autocorrelation.The benchmark evaluates long-range generation with classification and prediction models using linear encoders and an S4 layer.
  • Ultra-long-term unconditional generation: The ultra-long benchmark introduces Traffic and KDD-Cup datasets with sequence lengths of 17,544 and 10,920, evaluated using the same classification and prediction procedure.These datasets represent freeway occupancy and air-quality measurements.
  • Conditional generation: Interpolation masks 50% of inputs, whereas extrapolation masks the second half; performance is measured by MSE against ground truth.Long-term sparse and irregular data use the masking procedures specified by the established benchmark.

C.2 Wasserstein distance analysis

Wasserstein-distance analysis compares generated and original clusters across datasets and sequence lengths. The proposed approach achieves the best scores against GT-GAN on short sequences and LS4 on long and ultra-long sequences.

  • Wasserstein distance analysis: The approach yields the best Wasserstein distances across all datasets against GT-GAN for short sequences and LS4 for long and ultra-long sequences.Lower Wasserstein distance indicates greater similarity between generated and original clusters.

C.3 Short-term unconditional generation qualitative analysis

Qualitative t-SNE, density, and quantitative Wasserstein analyses assess whether generated samples reproduce real-data distributions. The reported visual and quantitative results demonstrate distribution learning across multiple datasets, while long-sequence results are also reported with standard deviations.

  • Short-term unconditional generation qualitative analysis: Visual t-SNE and density analyses, together with Wasserstein distances, demonstrate the framework’s ability to learn true distributions across multiple datasets.The qualitative analyses cover Stocks, Energy, and MuJoCo, while Table 11 reports quantitative cluster distances.
  • Long-term unconditional generation: Table 12 reports long-sequence unconditional-generation results with standard deviations and compares the method statistically with LS4.The supplied passages identify the comparison and uncertainty reporting but not individual table values.

C.5 Long-term unconditional generation qualitative analysis

The qualitative long-term analysis compares synthetic and real data through t-SNE embeddings and probability-density functions, with quantitative cluster evaluation using Wasserstein distance. The authors report that their method outperforms competing techniques and handles very long sequences.

  • The method outperforms competing techniques in qualitative t-SNE and density analyses of long-term datasets.The analysis covers Temp Rain, NN5 Daily, Traffic, and KDD-Cup, with quantitative t-SNE-cluster evaluation using Wasserstein distance.
  • The t-SNE visualizations compare synthetic data from the proposed method and SOTA tools against real data, while lower panels compare probability densities.
  • Ultra-long unconditional-generation results are reported with standard deviations to assess statistical significance against competitive state-of-the-art methods.
  • The Traffic and KDD-Cup analyses highlight the method’s ability to handle very long sequences.

C.8 Image size ablation

The image-size ablation evaluates whether transform-specific image resolutions affect generation quality and computational cost. Results suggest that image size has no significant quality effect, favoring the minimum adequate resolution.

  • The ablation uses STFT for long-term datasets and delay embedding for short-term datasets across different image sizes.
  • The analysis is motivated by image size’s substantial impact on the computational resources required by the framework.
  • Image size does not significantly affect generation quality, so the minimum resolution supported by each transformation is computationally preferable.The study tests sizes 8 and 16 for 24-step sequences and 32, 64, and 128 for approximately 750-step sequences; size 32 is the minimum adequate long-sequence resolution.

C.9 Hyeprparameters Ablation

The hyperparameter ablations examine diffusion sampling steps, batch size, and learning rate, revealing limited sensitivity to batch size but collapse at sufficiently high learning rates. Sampling-step effects remain unclear across datasets and metrics.

  • Diffusion sampling-step ablations show no consistent trend across datasets and metrics.Unlike findings in [45], increasing the number of sampling steps does not consistently improve FID in this framework.
  • Different batch sizes leave the framework’s results largely unaffected, suggesting adaptability across low- and high-memory environments.
  • Learning rates of 10^-3 or higher can collapse the diffusion backbone, producing irrelevant or random signals.At 10^-3, discriminative scores are .256 for MuJoCo and 0.499 for Energy; lowering the learning rate avoids observed collapse across datasets and tasks.
  • The resource analysis compares training and inference runtime, model size, and FLOPs with LS4 and DiffTime.Although the proposed method has more parameters, its training and inference time remains comparable to LS4.

C.11 Scaling Laws Analysis

The scaling-law analysis studies how image-diffusion model size affects time-series generation and compares the proposed framework with other generative models. It also reports resource and reproducibility information relevant to interpreting these experiments.

  • C.11 Scaling Laws Analysis: Scaling experiments evaluate the proposed image-diffusion model from thousands to hundreds of millions of parameters and compare size trends with state-of-the-art time-series generators.
  • C.11 Scaling Laws Analysis: The analysis includes Stocks scaling results, with lower discriminative scores indicating better performance.
  • C.11 Scaling Laws Analysis: Increasing model size can cause memory collapse, making batch size one infeasible under the reported training resources.
  • C.11 Scaling Laws Analysis: The framework’s scaling results are reported alongside LS4, DiffTime, and KDD-Cup scaling tables.
  • Experimental Reproducibility: The paper reports model architecture and hyperparameters in the appendix and states that code will be publicly available after the double-blind process.
  • Open access to data and code: The checklist states that datasets are public and that code is intended for later public release.
Loading 2410.19538v1…