PaperlayerView exact revision on arXiv

Source-linked AI summary

Wavelet Flow Matching for Time Series

Lucas Poinsignon, Jorge da Silva Goncalves, Samuel Ruiperez-Campillo, Julia E. Vogt

arXiv:2609.39374v2cs.LGcs.AI

TL;DR

Multivariate time-series generation must capture both multi-scale temporal structure and cross-channel dependencies. This paper applies flow matching to multilevel wavelet coefficients, pairing their natural variance differences with channel-token attention. Across seven datasets and four sequence lengths, the model matches or improves on the strongest baseline for a majority of dataset-metric pairs, with especially consistent gains on Context-FID.

  • Problem

    Multivariate time-series generation must represent both multi-scale temporal structure and strong, heterogeneous dependencies across channels.

    Source: §1 · p. 1

  • Method

    The method transports noise to multilevel DWT coefficients with flow matching and uses channel-token self-attention to model dependencies across channels.

    Source: §1 · p. 1, §3 · p. 3

  • Results

    Across seven datasets and four sequence lengths, the model matches or improves on the strongest baseline for a majority of dataset-metric pairs, with the largest and most consistent gains on Context-FID.

    Source: §6 · p. 10

  • Takeaways & Limitations

    Natural level-variance differences induce coarse-to-fine generation under a shared linear probability path, and ablations support preserving those scale-dependent statistics.

    Source: §1 · p. 2, §6 · p. 10

    The decomposition depth is fixed by window and filter length rather than adapted to each dataset’s scale structure.

    Source: §6 · p. 10

Abstract

from arXiv · show

Synthetic time series are increasingly used for data augmentation, privacy-preserving data sharing, and downstream model development, yet faithfully reproducing both multi-scale temporal structure and cross-channel dependencies remains challenging. We study multivariate time-series generation through flow matching in the wavelet domain. By operating on multilevel discrete wavelet coefficients rather than directly in the time domain, the model represents coarse structure and progressively finer details at separate scales. Their naturally different variances further induce an implicit coarse-to-fine generative process without requiring an explicit multi-scale schedule. Since the transform acts independently on each channel, we pair it with a channel-token transformer whose attention directly models cross-channel dependencies. Across seven benchmark datasets and four sequence lengths, our method is best or tied on a majority of dataset-metric combinations, with the largest and most consistent improvements in Context-FID and discriminative score.

1 INTRODUCTION

Multivariate time-series generation must capture both structure across temporal scales and dependencies among channels. This work combines wavelet-domain flow matching with channel-wise attention, using natural coefficient-variance differences to induce coarse-to-fine generation and reporting strong benchmark performance.

1 INTRODUCTION

  • Multivariate time-series generation must capture multi-scale temporal structure and strong, heterogeneous dependencies among channels.The paper motivates synthetic data for settings where real data can be scarce, expensive to collect, privacy-sensitive, or imbalanced.Source: §1 · p. 1
  • Wavelet flow matching maps each channel to multilevel DWT coefficients and transports isotropic Gaussian noise to their joint distribution with a single flow.Figure 1 depicts the inverse DWT mapping final coefficients back to the time domain.Source: §1 · p. 1, Figure 1 · p. 2
  • Natural wavelet-level variance differences give coarse levels earlier data-dominated SNR trajectories, inducing coarse-to-fine generation under a shared linear path.Equalizing level-wise variances collapses the SNR crossing-time differences and substantially degrades generation quality.Source: §1 · p. 2
  • Across seven datasets and four sequence lengths, the model achieves the strongest overall performance among five recent baselines.The reported gains are particularly consistent in Context-FID and discriminative score.Source: §1 · p. 2

2 RELATED WORK

Prior time-series diffusion and flow-matching models largely generate in the time domain, while other work explores transformed representations. This paper situates wavelet-domain generation among these approaches and distinguishes its shared flow and channel-focused attention from level-specific or image-based designs.

2 RELATED WORK

  • Time-series generation methods include diffusion models such as Diffusion-TS and time-domain flow models such as FlowTS; TSFlow instead applies flow matching with Gaussian-process priors for forecasting.Flow matching supports deterministic ODE sampling, whereas Diffusion-TS requires iterative reverse-time sampling.Source: §2 · p. 2
  • Wavelet representations are localized in time and scale; WaveletDiff uses dedicated transformers by level, while DSFM combines wavelets with blockwise DCT and a vision backbone for fMRI.The proposed framework instead uses one shared flow over multilevel DWT coefficients and self-attention across original channels.Source: §2 · pp. 2–3
  • Prior work imposes coarse-to-fine generation hierarchies explicitly, while this setting derives distinct SNR crossing times across wavelet levels from natural variance differences under one shared linear path.The paper probes this mechanism by equalizing level-wise variances.Source: §2 · p. 3

3 METHOD

The method applies flow matching to multilevel wavelet coefficients, using their variance differences to induce coarse-to-fine generation and channel-token attention to model cross-channel dependencies. Training samples intermediate flow times, and inference integrates the learned ODE with Euler steps before reconstructing the time series.

3 METHOD

  • A channel-wise discrete wavelet transform exposes temporal structure at multiple scales while leaving cross-channel dependencies for the learned velocity field.The transform is invertible and preserves input dimensionality, producing exactly T coefficients per channel.Source: §3 · p. 3, §3.1 · p. 3

3.1 FLOW MATCHING IN THE WAVELET DOMAIN

  • The method models the wavelet-transformed coefficient distribution by transporting Gaussian noise to coefficients and mapping generated coefficients back through the inverse DWT.A shared linear path connects each noise–coefficient pair, with constant conditional velocity equal to their difference.Source: §3.1 · pp. 3–4

3.2 COARSE-TO-FINE FLOW

  • Coarse wavelet levels have larger variance and become data-dominated earlier than fine levels, inducing coarse-to-fine progression under a shared linear path.This ordering is reported across all seven datasets; equalizing level variances collapses all SNR crossing times to t = 1/2.Source: §3.2 · p. 4, Figure 2 · p. 4

3.3 VELOCITY NETWORK: CHANNEL-TOKEN TRANSFORMER

  • The channel-token transformer embeds each channel’s multilevel coefficients into one token and uses self-attention across channels to model their dependencies.Level-specific heads map channel representations back to velocity coefficients at each scale; AdaLN-Zero injects flow-time conditioning into transformer blocks.Source: §3.3 · p. 5, Figure 3 · p. 5

3.4 TRAINING AND INFERENCE

  • Training samples flow times from a symmetric distribution concentrated at intermediate times, while inference uses explicit Euler integration with 100 uniformly spaced steps.Generated coefficients are mapped back to the time domain after integration.Source: §3.4 · pp. 5–6

4 EXPERIMENTAL SETUP

The experiments evaluate unconditional generation on seven multivariate time-series benchmarks, comparing the proposed model with five recent baselines across four sequence lengths and four metrics.

4 EXPERIMENTAL SETUP

  • Seven datasets span 6–28 channels and four window lengths, T ∈{24, 32, 64, 128}; MuJoCo uses independent rollouts rather than a continuous recording.The benchmarks cover energy, finance, EEG, household energy, and simulated robotics data.Source: §4 · p. 6
  • Five recent baselines cover time, signature, frequency, and wavelet representations, plus FlowTS; all use official implementations and the same windows and evaluation pipeline.The compared models are Diffusion-TS, SigDiffusion, FourierDiffusion, WaveletDiff, and FlowTS.Source: §4 · p. 6
  • Four metrics assess overall fidelity, cross-channel correlation fidelity, and downstream temporal utility, with lower values better throughout.At T=24, results average three training runs and five evaluations per run; longer lengths use one training run and five evaluations.Source: §4 · p. 6
  • The velocity network uses a three-level periodized DWT and an eight-block transformer with width 256 and eight attention heads.Wavelet order varies with sequence length: db2 for T ≤32, db4 for T = 64, and db6 for T = 128.Source: §4 · p. 6
  • The short-sequence benchmark reports unconditional generation at T = 24 as mean ± standard deviation over three end-to-end runs with five evaluations each.Bold and underlined values mark best and second-best results, respectively.Source: Table 1 · p. 7

5 RESULTS

At T=24, the model performs strongly across benchmark metrics, and its mean rank remains best as sequence length increases. Ablations support the roles of natural wavelet scaling and channel-aware multiresolution modeling.

5.1 TIME SERIES GENERATION

  • At T=24, our model is best or tied in 18/28 dataset–metric combinations; Context-FID and discriminative score are 2.9× and 1.7× lower than the strongest competitors on average.The predictive score is largely saturated across methods at this length, while correlational-score performance is more mixed.Source: §5.1 · p. 7
  • The model maintains the best mean rank across all four metrics as sequence length increases, while method ordering changes little.Mean ranks are computed across seven datasets separately for each metric and sequence length.Source: §5.1 · p. 7, Figure 4 · p. 8
  • On Energy, generated samples substantially overlap real data in the t-SNE embedding and closely match its empirical density.The visualization marks real data in red and generated data in blue.Source: §5.1 · p. 8, Figure 5 · p. 8

5.2 ANALYSIS OF THE PROPOSED METHOD

  • Natural scaling is better or tied in 25/28 dataset–metric combinations; Context-FID is 2.5× lower on average and discriminative score 1.6× lower on average than with per-level standardization.EEG is the main exception because its wavelet levels already have similar variances.Source: Figure 6 · p. 8, §5.2 · p. 9
  • Removing channel-identity embeddings or the wavelet transform generally degrades Context-FID, while uniform time sampling also worsens Context-FID.The channel-identity ablation has little effect on EEG; the effect of uniform sampling on other metrics is comparatively small.Source: §5.2 · p. 9, Table 2 · p. 9
  • No wavelet transform consistently dominates across datasets and metrics, and learning the transform provides no systematic improvement over fixed bases.Learned parameters remain close to their db2 initialization, suggesting performance depends primarily on the multiresolution representation.Source: §5.2 · p. 9
  • N = 100 uniformly spaced Euler steps provides a robust default in the sampler sensitivity analyses.Source: §5.2 · p. 9

6 CONCLUSION

The conclusion reports competitive benchmark results for wavelet-domain flow matching and identifies fixed decomposition depth and incomplete privacy/memorization evaluation as scope boundaries.

6 CONCLUSION

  • On a majority of dataset-metric pairs, the model matches or improves on the strongest baseline across seven datasets and four sequence lengths, with largest gains on Context-FID.Ablations support the importance of preserving natural scale-dependent coefficient statistics.Source: §6 · p. 10
  • The decomposition depth is fixed by window and filter length rather than adapted to each dataset’s scale structure; standard metrics also do not directly measure memorization.The authors identify dataset-adaptive decomposition and conditional forecasting or imputation as future directions.Source: §6 · p. 10

A EXTENDED RELATED WORK

  • Channel-token attention suits multiresolution representations because the transform separates within-channel temporal structure while leaving cross-channel dependence untouched.Each channel token carries its full coefficient stack.Source: A EXTENDED RELATED WORK · p. 19

B WAVELET TRANSFORM BACKGROUND

  • The DWT decomposes each channel into a coarse approximation and detail levels, with periodic extension used for filtering near window boundaries.The periodized transform is linear, invertible, critically sampled, and acts independently on channels.Source: §B.1 · p. 20

B.1 PERIODIZED DWT AND COEFFICIENT LAYOUT

  • The periodized DWT concatenates the approximation and detail sequences into as many coefficients as the original series has time steps.Exact reconstruction depends on the filter-bank admissibility conditions.Source: §B.1 · pp. 20–21

B.2 WAVELET FILTER FAMILIES

  • Wavelet families trade filter properties: orthonormal Daubechies filters have minimal length for their vanishing moments, while spline biorthogonal filters give up orthonormality for symmetry.The appendix compares four wavelet families and states conditions for exact reconstruction.Source: §B.2 · pp. 21–22

B.3.1 REQUIREMENTS

  • Restricting learned filters to orthonormal banks guarantees perfect reconstruction and energy preservation, preventing the transform from reducing the training objective merely by shrinking its output.The learnable bank therefore controls how energy is distributed across levels.Source: §B.3.1 · p. 22

B.3.2 THE POLYPHASE MATRIX; B.3.3 LATTICE PARAMETRIZATION

  • The polyphase matrix collects even- and odd-indexed filter components, and its paraunitary lattice factorization parameterizes orthonormal banks with angles.Constraining the angles preserves one vanishing moment at every training step, keeping approximation and detail coefficients separated by scale.Source: §B.3.2 · p. 22, §B.3.3 · p. 23

C SUPPORTING DERIVATIONS AND PROOFS

The appendix establishes the lattice parameterization’s orthonormality and normalization properties, then relates these filter properties to the periodized transform.

C.1 ORTHONORMALITY IS PARAUNITARITY

  • Orthonormality of the filters is equivalent to paraunitarity of their polyphase matrix.The proof expresses the matrix condition through even-lag correlations of the filters.Source: §C.1 · p. 24

C.2 THE LATTICE FACTORIZATION

  • The lattice factorization represents admissible banks through angle parameters, and the training direction constructs paraunitary banks from those angles.The appendix also gives recursions for mapping between angles and filters.Source: §C.2 · pp. 24–25

C.2.1 THE NORMALIZATION IS AN AFFINE CONDITION ON THE ANGLES

  • The normalization condition on the learned filter bank is an affine constraint on the angles.The appendix derives this by evaluating the lattice product at z = 1.Source: §C.2.1 · p. 26

C.3 ORTHOGONALITY OF THE PERIODIZED TRANSFORM

  • Under periodization, the multilevel transform is orthogonal, so it preserves energy and has its transpose as inverse; these properties fail under symmetric extension.The multiresolution stages also require the filter length not to exceed the coarsest stage length.Source: §C.3 · p. 26

D VELOCITY NETWORK ARCHITECTURE

The velocity network conditions channel-token transformer blocks on flow time and maps their outputs back to wavelet levels.

D.1 TIME CONDITIONING

  • A sinusoidal flow-time embedding passes through a two-layer MLP and is shared across channel tokens as the conditioning signal.The resulting vector conditions each transformer block and the final adaptive normalization.Source: §D.1 · p. 27

D.2 ADALN-ZERO TRANSFORMER BLOCKS

  • Each AdaLN-Zero block uses time-derived modulation for attention and feed-forward branches, with both residual branches initialized inactive.The identity-map initialization provides the stable starting point used in DiT.Source: §D.2 · p. 27
  • Self-attention runs over the F channel tokens rather than the T wavelet coefficients, giving attention cost O(F^2dm) per block.The passage states this is small next to the O(Fd^2m) cost of token-wise projections and MLPs.Source: §D.2 · p. 27

D.3 ADAPTIVE READ-OUT

  • A final time-conditioned adaptive normalization is applied before channel representations are projected back to individual wavelet levels.The modulation parameters are shared across channel tokens.Source: §D.3 · p. 27

D.4 PARAMETER COUNT AND COMPUTATIONAL COST

The default configuration’s parameter count is concentrated in transformer blocks, while the wavelet-specific maps contribute little and channel additions are small.

D.4 PARAMETER COUNT AND COMPUTATIONAL COST

  • Time conditioning accounts for a third of the model, and AdaLN-Zero projections cost 6d^2m per block against 4d^2m for attention.These figures describe the default configuration.Source: §D.4 · p. 28
  • Each additional channel adds exactly dm = 256 parameters because F appears in only one term of the parameter-count expression.The default configuration uses dm = 256.Source: §D.4 · p. 28

E.1 DATASETS

The experiments use seven time-series benchmarks and compare six generative models, with sampling cost reported as network evaluations per sample.

E.1 DATASETS

  • Table 4 lists dataset properties for transformer measurements, Google stock data, and daily currency exchange rates.ETTh1 and ETTh2 contain hourly measurements from two Chinese electricity transformers; Stocks covers Google prices and volume, while Exchange contains eight currencies against the US dollar.Source: §E.1 · p. 29, Table 4 · p. 29
  • The remaining benchmarks comprise a single-person EEG recording, household energy and environmental measurements, and simulated Hopper trajectories.The Energy data include two identical random-control channels; MuJoCo uses 10,000 rollouts from random initial configurations.Source: §E.1 · p. 29

E.2 BASELINES

  • Diffusion-TS models time-domain windows with trend and seasonal components, while WaveletDiff uses level-specific denoisers and cross-level attention on DWT coefficients.Both are diffusion models; WaveletDiff uses DDPM or DDIM sampling.Source: §E.2 · pp. 29–30
  • The other described baselines are SigDiffusion, FourierDiffusion, and FlowTS; sampling cost is measured by network function evaluations per generated sample.The comparison includes six models in total, and each method’s default sampler is used for the NFE report at T = 24.Source: §E.2 · p. 30

E.3 EVALUATION METRICS

Four metrics compare equally sized sets of real and generated windows, and lower scores are better.

E.3 EVALUATION METRICS

  • The discriminative score measures how well a post-hoc classifier separates real from generated windows; zero indicates indistinguishable sets.It is |acc − 0.5| on a held-out split, with 0.5 indicating perfect separability.Source: §E.3 · p. 31
  • The predictive score is a one-step-ahead forecaster’s mean absolute error on real windows after training on generated windows.It measures the usefulness of synthetic data as a substitute for real training data.Source: §E.3 · p. 31
  • Context-FID compares Gaussian distributions fitted to TS2Vec embeddings of real and generated windows.The TS2Vec encoder is trained on real windows, and the implementation uses output dimension 320.Source: §E.3 · p. 31
  • The lag-zero correlation score compares real and generated cross-channel correlations, with a factor of 1/10 retained for comparability to prior baselines.This score is not normalized by the number of channels and therefore is not comparable across datasets.Source: §E.3 · p. 31

E.4 TRAINING AND IMPLEMENTATION DETAILS

The experiments use fixed wavelet and transformer settings, then report coefficient statistics, longer-sequence evaluations, qualitative diagnostics, and component ablations.

E.4 TRAINING AND IMPLEMENTATION DETAILS

  • The main configuration uses periodic-boundary DWTs and a 256-wide, eight-layer velocity network, trained for 2,400 epochs per dataset.Wavelet choice varies with sequence length; sampling uses 100 uniform Euler steps, and final EMA weights are evaluated without validation-based model selection.Source: §E.4 · pp. 31–32

F.1 COEFFICIENT STATISTICS

  • Coarser db2 levels have larger coefficient standard deviations and cross SNR = 1 earlier; the crossing order a3 → d3 → d2 → d1 holds on six datasets.On EEG, the three detail-level crossing times nearly coincide.Source: Table 6 · p. 33, §F.1 · p. 33, Figure 7 · p. 33

F.2 FULL RESULTS ON LONGER SEQUENCES

  • Tables 7–9 report all four metrics for seven datasets and six models at T = 32, 64, and 128, with Figure 8 plotting results across sequence lengths.The tables report mean ± standard deviation over five metric recomputations on the same generated set.Source: §F.2 · p. 34, Table 7 · p. 34, Table 8 · p. 35

F.3 QUALITATIVE COMPARISON ON ALL DATASETS

  • At T = 24, Figures 9 and 10 provide t-SNE embeddings and real-versus-generated probability distributions for every method and dataset.The figures are qualitative diagnostics; the supplied descriptions do not state which method performs best.Source: §F.3 · p. 38, Figure 9 · p. 38, Figure 10 · p. 39

F.4 METHOD COMPONENT ABLATION

  • The T = 24 component ablations change one factor at a time: channel-identity embeddings, level standardization, the wavelet transform, or training-time sampling.Without channel-identity embeddings, the generated law is exchangeable across channels; the time-domain variant uses the identity transform.Source: §F.4 · p. 40, Table 10 · p. 41

F.5 SAMPLER SETTINGS

Sampler experiments vary the inference-time Euler grid and step count, while wavelet comparisons assess the sensitivity of results to the transform family.

F.5 SAMPLER SETTINGS

  • The sampler time grid is set by t_k = f_α(k/N): α > 1 concentrates steps near t = 1, α < 1 near t = 0, and α = 1 gives a uniform grid.Sampler ablations use the same trained model, isolating inference-time choices.Source: §F.5 · p. 42
  • Reducing Euler steps from 100 to 50 keeps discriminative and predictive scores within one standard deviation on six datasets, but triples EEG’s discriminative score.Average Context-FID increases from 0.007 to 0.008.Source: §F.5 · pp. 42, 44, Table 12 · p. 44

F.6 CHOICE OF WAVELET TRANSFORM.

  • No wavelet family wins across all dataset-metric combinations, and learned filters remain close to their db2 initialization.The learned angles stay within 2° of initialization, and final filters differ from db2 by at most 5% in ℓ2.Source: §F.6 · p. 45, Table 13 · p. 45, Figure 11 · p. 45

Related papers

Loading 2609.39374v2…