Source-linked AI summary
TimeVAE: A Variational Auto-Encoder for Multivariate Time Series Generation
Abhyuday Desai, Cynthia Freeman, Zuhui Wang, Ian Beaver
TL;DR
Synthetic time-series generation needs methods that can handle limited or restricted real data while preserving temporal structure and supporting domain knowledge. TimeVAE uses a VAE architecture with a user-defined decoder, and it consistently matches or exceeds other generators on similarity and next-step prediction tests across datasets and training sizes.
Problem
Synthetic data generation is needed when real data are unavailable, insufficient, privacy-restricted, or inadequate for simulating specific situations, while existing GAN-based methods can be difficult to train and data-hungry.
Method
TimeVAE is a VAE-based multivariate time-series generator with a decoder supporting user-defined distributions and custom temporal constructs such as level, trend, and seasonality.
Results
TimeVAE consistently meets or exceeds other generators on next-step prediction, produces high-quality similarity results, and remains competitive at low training thresholds.
Takeaways & Limitations
The architecture provides interpretable generated signals, denoising through its VAE bottleneck, and strong prediction performance even with limited training data.
Takeaways & Limitations
C-RNN-GAN was excluded because the authors could not reproduce reported performance and observed instability during the tasks.
Abstract
from arXiv · showhide
Recent work in synthetic data generation in the time-series domain has focused on the use of Generative Adversarial Networks. We propose a novel architecture for synthetically generating time-series data with the use of Variational Auto-Encoders (VAEs). The proposed architecture has several distinct properties: interpretability, ability to encode domain knowledge, and reduced training times. We evaluate data generation quality by similarity and predictability against four multivariate datasets. We experiment with varying sizes of training data to measure the impact of data availability on generation quality for our VAE method as well as several state-of-the-art data generation methods. Our results on similarity tests show that the VAE approach is able to accurately represent the temporal attributes of the original data. On next-step prediction tasks using generated data, the proposed VAE architecture consistently meets or exceeds performance of state-of-the-art data generation methods. While noise reduction may cause the generated data to deviate from original data, we demonstrate the resulting de-noised data can significantly improve performance for next-step prediction using generated data. Finally, the proposed architecture can incorporate domain-specific time-patterns such as polynomial trends and seasonalities to provide interpretable outputs. Such interpretability can be highly advantageous in applications requiring transparency of model outputs or where users desire to inject prior knowledge of time-series patterns into the generative model.
1 INTRODUCTION
Synthetic time-series generation must capture feature distributions and temporal relationships despite limited data, while supporting user-specified temporal knowledge. TimeVAE addresses these needs with a VAE-based model designed for controllability and temporal modeling.
- Generative models can support privacy-constrained data use, simulation of unseen or exceptional situations, and testing scenarios involving outliers or changepoints.
- Users may need to inject subject-matter expertise and control generated data, although synthetic outputs can differ substantially from real-world data.
- Time-series generators must capture both feature distributions and temporal relationships, making synthetic generation challenging.
- Real-world applications may have limited samples or short histories, increasing the need for methods that work with low-volume data.
- Prior time-series generation work has largely used GANs with recurrent neural networks, requiring special mechanisms to capture temporal dependencies.
- TimeVAE combines a VAE with a decoder supporting user-defined distributions and custom temporal constructs such as level, trend, and seasonality.
2 RELATED WORKS
Related time-series generators include GAN-based and recurrent VAE approaches, but prior methods face training, runtime, diversity, and data-availability challenges. TimeVAE is presented as an attempt to address these limitations.
- TimeGAN combines an unsupervised GAN with autoregressive models and is described as the current state-of-the-art in synthetic time-series generation.
- RCGAN uses recurrent generator and discriminator networks conditioned on auxiliary information to generate labeled real-valued sequential data.
- C-RNN-GAN generates sequential music data with noisy inputs, deep LSTM modules, fully connected layers, and adversarial discrimination.
- A recurrent VAE encodes input sequences with an RNN, samples from the encoded representation, and decodes the sample, but prior testing reported inconsistent results on sine-wave data.
- Existing generative models can suffer mode collapse, require more than a day to train in some settings, and need sufficient data for GAN-based training.The reported runtime involved TimeGAN and RCGAN trained for 5,000 epochs on larger datasets using a V100 GPU.
3 METHODS
TimeVAE uses a VAE framework with convolutional encoder–decoder components and interpretable decoder branches that inject level, trend, and seasonal structures.
- The method targets realistic time-series generation and domain-specific temporal structure injection, especially when real-world data are insufficient.
- The encoder models a probabilistic posterior over latent representations, while the decoder models the conditional likelihood of the input.
- The ELBO loss combines negative log-likelihood with KL divergence between the encoded latent distribution and the prior.The reparameterization trick keeps sampling differentiable for end-to-end training.
- Base TimeVAE: The base architecture uses convolutional layers, flattening, and dense layers to encode N × T × D inputs into an m-dimensional latent Gaussian, then reconstructs the original shape.The decoder samples z, reshapes it, applies transposed convolutions, and uses a time-distributed fully connected layer.
- Interpretable TimeVAE: The interpretable decoder adds parallel trend and seasonality blocks whose outputs are combined to produce the final signal.The original base decoder can also serve as a residual branch, and these branches may be enabled or disabled.
- Trend Block: The trend block reconstructs polynomial trends from latent coefficients and a time-power matrix, with coefficient values identifying trend orders for each sample and feature.A zeroth-order polynomial gives a flat trend equivalent to the level component.
- Seasonality Block: Each seasonality pattern is specified by the number of seasons and each season’s duration, allowing structures such as daily or day-of-week periodicity.Latent coefficients are indexed by season and summed to form seasonal estimates, with coefficients exposing the contribution of individual seasons.
4 EXPERIMENTS
The experiments compare TimeVAE with RCGAN, T-Forcing, and TimeGAN across four multivariate datasets, training-data fractions, and discriminator, similarity, and prediction evaluations.
- Comparison methodologies: The comparison includes TimeVAE, RCGAN, T-Forcing, and TimeGAN.T-Forcing uses ground truth as the next-step input during autoregressive training, while RCGAN conditions sequential generation on additional input.
- Comparison methodologies: C-RNN-GAN is excluded because the authors could not reproduce reported performance and observed instability on the evaluated tasks.The exclusion is documented as a methodological limitation of the comparison study.
- Comparison metrics: Similarity is assessed with t-SNE plots and a discriminator score defined as held-out accuracy −0.5, where values near 0 are better.Predictive scores are also computed on next-step prediction tasks, and lower scores are better for the comparison metrics.
- Experimental setup: Four multivariate datasets are evaluated with 100%, 20%, 10%, 5%, and 2% training-data thresholds.Synthetic data from each trained generator is used to train post-hoc sequence models for discrimination and prediction tests.
5 RESULTS/DISCUSSION
TimeVAE shows strong similarity to original data across datasets and training sizes, while its next-step prediction performance consistently meets or exceeds the other generators. TimeGAN is competitive on several datasets, but some methods fail or deteriorate when training data are scarce.
- Similarity results: TimeVAE synthetic data heavily overlaps original data across all datasets and training thresholds, including 2% training.The plots also show denoising, particularly for the noisy stockv and energy datasets.
- Similarity results: TimeVAE is superior on t-SNE similarity overall, while all methods except TimeVAE produce inferior-quality data on the sine dataset.TimeGAN performs well on air at 100% training but degrades at 2%, 5%, and 10% thresholds.
- Discriminator results: At 100% training, TimeVAE has the best discriminator results on air, sine, and stockv, while lower-data results favor TimeVAE and TimeGAN over T-Forcing and RCGAN.The discriminator results do not conclusively distinguish TimeVAE from TimeGAN except on sine.
- Discriminator results: TimeGAN and RCGAN cannot train successfully on 2% of stockv.This is reported as an N/A outcome associated with insufficient training data for those models.
- Predictor results: TimeVAE consistently meets or exceeds the other generators on next-step prediction, nearly matching original-data performance on stockv and sine even at 2%.TimeVAE is best on sine; on stockv it is indistinguishable from TimeGAN within the confidence interval, and on energy it ties T-Forcing as best.
- Training times: Training times are reported for all models using 100% of the data on the same AWS instance.The supplied passage specifies the hardware setting but not the comparative time values.
6 CONCLUSIONS
TimeVAE provides a VAE-based approach for multivariate time-series generation that performs competitively with existing methods while requiring less training time. Its domain-specific temporal constructs support interpretable outputs and future evaluation on more complex forecasting tasks.
- TimeVAE consistently produced high-quality generated data and met or exceeded state-of-the-art performance across similarity, discriminator, and next-step prediction tests.
- TimeVAE requires significantly less computing time, and therefore cost, to train than existing GAN-based methods.
- Domain-specific temporal constructs enable TimeVAE to produce interpretable results.
- Future work will further assess temporal constructs on multi-step time-series forecasting with limited data.