Source-linked AI summary
TimeDP: Learning to Generate Multi-Domain Time Series with Domain Prompts
Yu-Hao Huang, Chang Xu, Yueying Wu, Wu-Jun Li, Jiang Bian
TL;DR
TimeDP addresses the limited ability of existing time series generators to learn across domains, especially when domain labels or reliable text descriptions are unavailable. It learns semantic time series prototypes and prototype-based domain prompts for a diffusion model, extracting prompts from few-shot target-domain samples. The paper reports state-of-the-art in-domain generation quality and strong unseen-domain generation capability.
Problem
Existing time series generation models are mostly single-domain, while cross-domain generation is difficult because domains have divergent patterns and may lack usable labels or text descriptions.
Method
TimeDP is a label-free, text-free diffusion model that learns time series prototypes and assigns prototype weights to construct domain prompts for generation.
Results
TimeDP outperforms baselines with state-of-the-art in-domain generation quality and strong unseen-domain generation capability.
Takeaways & Limitations
Few-shot target-domain samples can provide domain prompts for generating time series without explicit labels or text descriptions.
Takeaways & Limitations
Natural-language conditioning remains difficult because domain-specific descriptions can be noisy, incomplete, ambiguous, or impractical for new domains.
Abstract
from arXiv · showhide
Time series generation models are crucial for applications like data augmentation and privacy preservation. Most existing time series generation models are typically designed to generate data from one specified domain. While leveraging data from other domain for better generalization is proved to work in other application areas, this approach remains challenging for time series modeling due to the large divergence in patterns among different real world time series categories. In this paper, we propose a multi-domain time series diffusion model with domain prompts, named TimeDP. In TimeDP, we utilize a time series semantic prototype module which defines time series prototypes to represent time series basis, each prototype vector serving as "word" representing some elementary time series feature. A prototype assignment module is applied to extract the extract domain specific prototype weights, for learning domain prompts as generation condition. During sampling, we extract "domain prompt" with few-shot samples from the target domain and use the domain prompts as condition to generate time series samples. Experiments demonstrate that our method outperforms baselines to provide the state-of-the-art in-domain generation quality and strong unseen domain generation capability.
Introduction
TimeDP addresses the gap in multi-domain time series generation with a label-free, text-free diffusion model that learns domain prompts from time series prototypes. The method targets both in-domain quality and generation for unseen domains.
- Time series generation remains important for medical synthesis, financial simulation, and dataset augmentation, but most existing models target single domains.
- Cross-domain generation is difficult because domains exhibit divergent patterns and may lack explicit labels or precise natural-language descriptions.
- TimeDP learns time series prototypes as semantic basis elements and combines prototype assignments into domain prompts for diffusion-based generation.
- During sampling, TimeDP extracts domain prompts from few-shot target-domain samples rather than requiring labels or text descriptions.
- Experiments report state-of-the-art in-domain generation quality and strong unseen-domain generation capability relative to baselines.
Related Work and Backgrounds
Prior work includes GAN-, VAE-, and diffusion-based time series generators, while multi-domain foundation models commonly use broad pretraining or mixed-domain training. TimeDP formulates a unified diffusion-generation setting in which domain semantics condition synthesis without relying on history.
- Related Work: Existing time series generators use GANs, VAEs, and diffusion models, with diffusion methods supporting unconditional and conditional generation.
- Related Work: Multi-domain time series foundation models commonly pretrain representations or masked-reconstruction models across many datasets before task-specific adaptation.
- Denoising Diffusion Probabilistic Models (DDPMs): Diffusion models progressively add Gaussian noise to data and learn a reverse process that reconstructs samples by predicting the added noise.
- Denoising Diffusion Probabilistic Models (DDPMs): Unlike forecasting, time series generation does not use historical context; TimeDP instead uses c to provide domain semantics during generation.
- Problem Formulation: The problem setting mixes M domains into one dataset and seeks a shared model for their joint time-series distributions, using a channel-independent univariate formulation.
Methodology
TimeDP trains a single diffusion model across multiple time-series domains, using shared prototypes and prototype assignments as domain-specific conditioning. Few-shot samples from a target domain provide prompts for in-domain or unseen-domain generation.
- Prototype Representation: Shared time-series prototypes represent elementary features, while prototype assignments encode each sequence as importance weights over those prototypes.The model uses these assignments to distinguish domain-specific combinations of shared temporal features.
- Prototype Assignment Module: A neural weight extractor maps each input sequence to prototype weights, discards negative values for sparsity, and uses the assignments to condition cross-attention noise prediction.The resulting weighted combinations allow sequences from different domains to use different subsets of the shared prototypes.
- Domain-Unified Training: TimeDP trains one diffusion model on multiple datasets while reweighting samples so each domain has equal training probability.The conditional denoising objective uses prototype assignments, and the sampling weight for each sample is set to 1/(N_i·|D|).
- Domain-Unified Training: Conditional denoising incorporates prototype-derived context into intermediate noise-prediction layers through spatial attention.The conditional objective predicts noise from the corrupted sequence, timestep, and prototype assignments.
- Generation with Domain Prompt: For a selected domain, few-shot samples are converted into a distribution of domain prompts and repeatedly used as conditioning to generate the requested number of samples.This procedure preserves domain adherence without constraining generation to the temporal patterns of the selected examples.
- Unseen-Domain Generation: The same few-shot prompting procedure supports unseen domains by extracting prototype assignments from samples in a dataset absent from training.The prototype representation is intended to extend beyond the training domains.
Experiments
Experiments evaluate TimeDP across 12 datasets and four domains for in-domain and unseen-domain synthesis. TimeDP generally outperforms baselines, and its prototype prompts are important for both distributional fidelity and unseen-domain generalization.
- Experiment Settings: The evaluation covers 12 datasets from electricity, solar, wind, transport, nature, and economic domains, using MMD, K-L, and MDD to assess distribution similarity and temporal dependence.Baselines are retrained on the combined multi-domain dataset, with TimeVQVAE-C additionally using data-domain labels.
- Evaluation of Generation Quality: TimeDP achieves the best results on most of twelve in-domain datasets, generating samples with distributions closest to real data jointly and marginally.The comparison uses MMD and K-L for sequence length 168, with results reported in Table 1.
- Evaluation of Generation Quality: Mixed-domain training generally fails to match real-data distributions, whereas TimeDP outperforms class-conditional TimeVQVAE without explicit class labels.TimeDP also maintains the best performance across long and short sequence lengths.
- Evaluation of Unseen Domain Time Series Synthesis: TimeDP obtains the best general MMD and K-L scores in unseen-domain synthesis, using few-shot samples to infer target distributions without additional tuning.Performance improves as the number of few-shot samples increases, while baseline fine-tuning is not consistently beneficial in the low-data regime.
- Ablation Study: Removing the prototype assignment module or prompt conditioning substantially worsens MMD, while K-L remains stable, indicating that prompts target sequence-wise distributional structure.Performance remains consistent once the number of prototypes is sufficiently large.
Conclusion
TimeDP is a multi-domain time series diffusion model that uses learned semantic prototypes and few-shot domain prompts for conditional generation. Experiments report state-of-the-art in-domain quality and strong unseen-domain generation capability.
- Conclusion: TimeDP learns time series prototypes as semantic basis elements and assigns prototype weights to construct domain prompts for diffusion-based generation.Few-shot samples from a target domain provide prompts during sampling.
- Conclusion: Experiments show state-of-the-art in-domain generation quality and strong unseen-domain generation capability compared with baselines.
Architecture
The model uses a U-Net denoising network and a convolutional prototype assignment module. These components implement diffusion denoising and prototype-weight extraction for domain prompting.
- Architecture: The denoising network is a U-Net with four up/down-sampling blocks, residual blocks, cross-attention, and a middle attention block.Residual blocks use 1-D convolutions, cross-attention uses eight heads, and SiLU is the activation.
- Architecture: The prototype assignment module uses convolutional feature extraction, residual convolutions, and a linear projection to produce prototype assignments.Its nonlinear activations are ReLU.
Description on Datasets
The experiments use diverse datasets spanning energy, transport, nature, and economic domains, plus additional Stock and Web datasets for unseen-domain generation. The data are preprocessed into univariate sequence slices or described individually in the dataset materials.
- Description on Datasets: The energy and environmental datasets include electricity consumption, solar production, wind production, air quality, temperature, and rain series.The supplied descriptions include hourly, daily, and high-frequency observations from multiple geographic and measurement settings.
- Description on Datasets: The economic datasets include ATM cash withdrawals, macroeconomic indicators, and exchange rates across currencies.
- Description on Datasets: Additional dataset descriptions include Stock data for GOOG daily prices and Web data containing 145063 daily Wikipedia traffic series.These datasets are identified as part of the unseen-data generation materials.
Additional Results for Generation Quality Experiment
Additional results show that TimeDP remains strongest across marginal-distribution evaluation and multiple sequence lengths.
- 10 out of 12 datasets receive the best Marginal Distribution Distance score from TimeDP.
- TimeDP consistently outperforms baselines for sequence lengths 24, 96, and 336, supporting robustness across temporal scales.
- The reported MDD results cover in-domain generation at sequence length 168, with best and second-best methods explicitly marked.
Marginal Distribution Distance Score for Ablation Study
The ablation and visualization analyses indicate stable marginal-distance performance and domain-specific structure in learned prototypes and prompts.
- Marginal Distribution Distance Score for Ablation Study: TimeDP’s marginal distribution distance performance remains generally stable in the ablation study.
- Prompt Visualization: Samples from the same dataset receive similar prototype weights, while different datasets show diverse weight distributions.
- Prompt Visualization: t-SNE visualizations show that learned domain prompts differentiate among domains.
- Prototype Visualization: One-hot prototype prompting produces distinct time-series semantics, including different seasonality and trend patterns.
- Few-Shot Generation: Few-shot prompting generates samples most similar to real Web and Stock data without finetuning, whereas finetuned baselines fail to generate realistic sequences.