PaperlayerView exact revision on arXiv

Source-linked AI summary

Neuralized Multi-Wavelet Decomposition for Time Series Classification and Forecasting

Xiaohan Jiang, Jingyuan Wang, Jiahao Ji, Yongyao Wang, Chen Yang, Junjie Wu

arXiv:2609.29317v1cs.LGcs.AI

TL;DR

Real-world time series contain temporal patterns and frequency structures, while existing approaches often model these aspects separately. The paper proposes neuralized multi-wavelet decomposition and task-specific models, which achieve state-of-the-art performance across diverse settings.

  • Problem

    Existing time-series approaches often focus on frequency decomposition or temporal-pattern extraction in isolation, despite the joint structure of real-world series.

    Source: §1 · pp. 1–2

  • Method

    m-WCN neuralizes multi-wavelet decomposition for joint temporal-pattern and frequency extraction, supporting TFBC classification and FTB forecasting models.

    Source: §1 · p. 2

  • Results

    The m-WCN-based models achieved state-of-the-art performance over many real-world datasets and across diverse settings.

    Source: §1 · p. 2, §7 · p. 15

  • Takeaways & Limitations

    m-WCN adds only linear time and memory overhead with input length, supporting integration into downstream classification and forecasting models without excessive complexity.

    Source: §5.3 · p. 10

    The framework is naturally suited to univariate series; handling multivariate series requires treating variables independently, which can be cumbersome.

    Source: §7 · p. 15

Abstract

from arXiv · show

Time series analysis is fundamental in domains such as finance, healthcare, and meteorology. Real-world time series often exhibit multiscale characteristics shaped by diverse latent factors, resulting in intricate temporal patterns and rich frequency structures. However, existing approaches typically focus on either frequency-domain decomposition or time-domain pattern extraction in isolation, neglecting their joint structure. This decoupled modeling limits representation expressiveness and undermines performance in tasks requiring simultaneous temporal and spectral reasoning. To address this gap, we propose m-WCN, a novel end-to-end deep learning framework that neuralizes multi-wavelet decomposition for joint extraction of temporal patterns and frequency components. By approximating the classical GHM multi-wavelet transform with trainable convolutional operators and enforcing orthogonality constraints, m-WCN produces interpretable multi-resolution representations. Built on this foundation, we introduce two task-specific architectures: TFBC for time series classification, which boosts discriminative features across frequency scales, and FTB for forecasting, which ensembles frequency-aware predictors. Extensive experiments on 64 UCR datasets and seven public forecasting benchmarks demonstrate the effectiveness of our approach. Built on the neuralized m-WCN, our TFBC and FTB outperform various baseline models across diverse datasets, achieving average improvements of 19.97% in classification and 19.92% in forecasting tasks.

1 INTRODUCTION

Real-world time series contain both complex temporal patterns and diverse frequency components, but existing decomposition approaches often model these structures separately. The paper proposes a trainable joint decomposition framework and task-specific models for classification and forecasting.

1 INTRODUCTION

  • Real-world time series can combine latent-factor-driven temporal patterns with diverse frequency components, making both views relevant to their analysis.The passage illustrates this with traffic trends, sudden fluctuations, and distinct daily intervals.Source: §1 · p. 1
  • Existing end-to-end decomposition models often focus on frequency decomposition while overlooking temporal patterns, motivating a unified learnable framework for both representations.The paper argues that time- and frequency-domain representations provide complementary insights.Source: §1 · p. 2
  • m-WCN jointly decomposes temporal patterns and frequency components by neuralizing GHM multi-wavelet steps, with orthogonal regularization to preserve interpretability.Its pre-filter and scaling and wavelet functions use trainable convolutional kernels.Source: §1 · p. 2
  • TFBC boosts classification features from coarse to fine frequency scales, while FTB assigns a dedicated predictor to each frequency component and aggregates their forecasts.The models use frequency contrastive learning for TFBC and frequency representation pre-training for FTB.Source: §1 · p. 2
  • 19.97% and 19.92% average performance improvements over various baselines were reported for classification and forecasting, respectively; the models achieved state-of-the-art performance across many real-world datasets.The evaluation covered 64 UCR datasets for classification and seven public datasets for forecasting.Source: §1 · p. 2

2 RELATED WORK

Related approaches extract temporal patterns or frequency information, but often do not combine both effectively. The section also surveys classification and forecasting methods that may overlook mixed pattern-frequency features.

Pattern Analysis

  • Pattern-analysis methods such as empirical mode decomposition, tensor decomposition, and shapelet methods can overlook frequency information.Pattern analysis supports downstream models by extracting informative features.Source: §2 · p. 3

Frequency Analysis

  • Frequency-analysis approaches include DFT, DWT, and neural methods, but often lack effective integration with pattern-based methods.DMWT jointly extracts frequency and pattern information, but its pattern modes may not adapt to varying data characteristics.Source: §2 · p. 3

Time Series Classification and Forecasting

  • Deep-learning classifiers and forecasters use varied feature-learning approaches, yet integrating both pattern and frequency features remains a challenge.The cited classification methods may overlook mixed Pattern-Frequency features, and few forecasting studies address both.Source: §2 · p. 3

3 PRELIMINARIES

The preliminaries establish wavelet decomposition as a way to separate high- and low-frequency information, then extend it to multi-wavelet decomposition, which separates temporal patterns before frequency analysis.

3.1 Notations

  • The notation section specifies scalar, vector, matrix, and tensor conventions and uses superscripts to identify frequency components and their decomposition layers.For example, ∗ln denotes the low-frequency component at layer n, while ∗hn denotes the high-frequency component.Source: §3.1 · p. 3

3.2 Wavelet Decomposition

  • Wavelet decomposition extracts high-frequency components reflecting short-term variation and low-frequency components capturing long-term trends.The notation distinguishes frequency components by layer and by high- or low-frequency subscripts.Source: §3.2 · p. 3, §3.1 · p. 3

3.3 Multi-wavelet Decomposition

  • Multi-wavelet decomposition first transforms a scalar series into multiple temporal-pattern subseries, then applies multiple wavelet and scaling functions to decompose their frequencies.The pre-filter matrix P maps each local segment to a vector time slice, with each resulting subseries representing a projection onto a distinct subspace.Source: §3.3 · p. 4
  • For N layers, multi-wavelet decomposition retains each layer’s high-frequency component and the final low-frequency component, using intermediate low-frequency components only for recursion.Separating patterns before frequency analysis can produce more homogeneous subseries that are easier to model and predict downstream.Source: §3.3 · pp. 4–5

4 MULTI-WAVELET CONVOLUTION NETWORK

m-WCN neuralizes the GHM multi-wavelet process with convolutional networks, learning temporal-pattern and frequency decompositions while regularizing kernels toward orthogonality.

4 MULTI-WAVELET CONVOLUTION NETWORK

  • m-WCN approximates GHM multi-wavelet decomposition in a neural network, enabling task-specific learning and end-to-end use of its outputs in downstream networks.GHM supplies the backbone; its fixed parameters motivate learning transformation coefficients from the input series.Source: §4 · p. 5, §4.1 · p. 5, §4.3 · p. 6

4.1 GHM Multi-wavelet and m-WCN Framework

  • The framework pairs a pre-filter network for temporal-pattern decomposition with a wavelet-decomposition network for frequency decomposition.The GHM projection transforms the scalar input into a two-dimensional vector series before frequency decomposition.Source: §4.1 · p. 5, Figure 2 · p. 5

4.2 Pre-filter Network of m-WCN

  • The pre-filter network neuralizes GHM prefiltering with convolution and learnable parameters, producing multidimensional features whose dimensions represent temporal patterns.A sigmoid activation is used to capture complex dependencies while retaining the GHM formulation’s theoretical structure.Source: §4.2 · pp. 5–6

4.3 Wavelet Decomposition Network of m-WCN

  • The wavelet-decomposition network adds learnable parameters to GHM scaling and wavelet tensors, allowing adaptive frequency extraction and task-specific tuning.Its output includes high-frequency components at successive layers and the lowest-frequency component at the final layer.Source: §4.3 · p. 6

4.4 Orthogonality Regularization

  • Orthogonality regularization constrains convolution kernels toward orthonormality, and the block-Toeplitz formulation makes the constraint a single matrix multiplication.The authors associate learned orthogonal filters with less redundant responses and more stable gradient propagation; the formulation also supports parallel computation.Source: §4.4 · pp. 6–7

5 M-WCN-BASED TIME SERIES ANALYSIS

The section introduces m-WCN-based architectures for time series classification and forecasting.

5 M-WCN-BASED TIME SERIES ANALYSIS

  • The section proposes two m-WCN-based network architectures for time series classification and time series forecasting.The passage identifies the downstream tasks but does not specify architecture details here.Source: §5 · p. 7

5.1 m-WCN-based Time Series Classification

TFBC builds on m-WCN’s frequency components to classify time series by progressively refining predictions across frequency scales. It combines frequency contrastive pre-training with layerwise supervised optimization.

5.1.1 Time-Frequency Boosting Classification

  • TFBC uses m-WCN’s frequency components in a boosting-style classifier, starting with low frequencies and refining predictions with higher-frequency residuals.The final class prediction ensembles outputs across decomposition levels, which represent different frequency resolutions.Source: §5.1.1 · pp. 7–8

5.1.2 Frequency Contrastive Learning

  • Frequency Contrastive Learning treats representations of the same series at different frequency levels as positives and those from different series as negatives.This pre-training encourages frequency-related embeddings that provide initialization for downstream classification fine-tuning.Source: §5.1.2 · p. 8

5.1.3 Optimization for TFBC

  • TFBC’s overall classification objective combines layerwise cross-entropy, frequency contrastive loss, and orthogonal regularization.The final prediction receives weight 1, while the lowest-frequency-only prediction receives weight 1/N.Source: §5.1.3 · p. 9

5.2 m-WCN-based Time Series Forecasting

FTB applies m-WCN to forecasting, using frequency-specific predictors and a pre-training strategy that aligns representations with future frequency components.

5.2.1 Bagging of TSMixer for Forecasting

  • FTB assigns separate TSMixer predictors to frequency components, then uses attention to integrate their representations and an MLP to produce the forecast.Its bagging design combines contributions from components capturing different temporal scales.Source: §5.2.1 · p. 9

5.2.2 Frequency Representation Pre-training

  • Frequency Representation Pre-training trains each past frequency representation to predict its corresponding component in the future series.Future components are extracted with GHM multi-wavelet decomposition and used as pre-training targets.Source: §5.2.2 · p. 10

5.2.3 Optimization for FTB

  • After pre-training, FTB is fine-tuned end-to-end with mean square error between predicted and ground-truth future sequences.The model is initialized with parameters learned during pre-training.Source: §5.2.3 · p. 10

5.3 Complexity Discussion

With fixed convolutional kernel sizes and channel counts, m-WCN adds linear time and memory overhead in input length.

Complexity Discussion

  • m-WCN’s additional time complexity is O(T), with fixed-size convolutions and downsampling across decomposition layers.The pre-filter scans the sequence once, and the layerwise costs sum to O(T).Source: §5.3 · p. 10
  • m-WCN’s space complexity is O(T), including retained frequency components and intermediate activations stored during training.The total retained sequence length is bounded by T up to fixed channel factors.Source: §5.3 · p. 10
  • The decomposition module therefore adds linear time and memory overhead with input length.The paper describes this overhead as allowing integration into downstream models without excessive complexity.Source: §5.3 · p. 10

6.1 Task I: Time Series Classification

TFBC is evaluated on 64 representative UCR datasets against seven baselines, with results favoring TFBC and ablations supporting the proposed components.

6.1.1 Datasets

  • The evaluation uses 64 representative UCR datasets, selected to avoid nearly saturated cases while covering seven major data categories.The categories are Device, ECG, Image, Motion, Sensor, Simulated, and Spectro.Source: §6.1.1 · p. 10

6.1.2 Evaluation Metric and Baselines

  • Classification is measured by error rate against seven baselines spanning feature-based, ensemble, deep-learning, and wavelet-based approaches.The baselines include miniRocket, Hivecote2, TS-Chief, OS-CNN, TimeURL, MILLET, and mWDN.Source: §6.1.2 · pp. 10–11

6.1.3 Implementation Details

  • Training uses Adam with batch size 16 and up to 200 epochs; decomposition levels and hidden dimensions are selected using validation performance.The learning rate decreases from 0.001 to 0.0001 and then 0.00001 across the three training intervals.Source: §6.1.3 · p. 11

6.1.4 Results and Analysis

  • TFBC is best on 34 of 64 datasets, with arithmetic and geometric average ranks of 1.88 and 1.59, and reduces average error rate by 19.97% over all baselines on average.The results are summarized in Table 3; superiority is also verified by a Nemenyi test at the 5% level.Source: Table 3 · p. 11, §6.1.4 · pp. 11–12
  • TFBC beats competing models across dataset categories; comparison with mWDN highlights pattern decomposition, and results on all 128 UCR datasets are consistent.The authors report larger improvements over the second-best model on IMAGE, MOTION, and SENSOR datasets.Source: §6.1.4 · p. 12

6.1.5 Ablation of Important Modules

  • Removing m-WCN raises average error rate from 0.146 to 0.294 and worsens arithmetic average rank from 2.281 to 5.125.TFBC has the lowest average error rate, 0.146, and the best arithmetic and geometric average ranks, 2.281 and 1.921.Source: §6.1.5 · p. 12, Table 4 · p. 12
  • Replacing learnable pre-filtering with fixed parameters raises average error rate to 0.162, while replacing multi-wavelet decomposition with standard wavelets raises it to 0.170.These comparisons support the contributions of learnable pattern decomposition and multi-wavelet frequency decomposition.Source: §6.1.5 · p. 12
  • Removing orthogonality regularization raises average error rate to 0.156, while removing frequency contrastive learning raises it to 0.154.The authors associate orthogonality regularization with reducing component redundancy and frequency contrastive learning with improving frequency-representation discrimination.Source: §6.1.5 · p. 12

6.2 Task II: Time Series Forecasting

The forecasting evaluation tests FTB on seven public datasets and reports stronger aggregate performance than baselines. Ablations, efficiency measurements, and sample visualizations further examine its behavior.

6.2.1 Datasets

  • On seven public forecasting datasets, the evaluation includes four ETT subsets, Electricity, Traffic, and Weather.The ETT subsets use hourly or 15-minute measurements; Weather uses hourly observations.Source: §6.2.1 · p. 12

6.2.2 Evaluation Metric and Baselines

  • FTB is evaluated with MSE and MAE against seven competitive forecasting baselines.The baselines include decomposition-based models, multi-scale temporal models, and TimeLLM.Source: §6.2.2 · p. 13

6.2.3 Implementation Details

  • FTB is trained for 70 epochs with Adam at a learning rate of 0.001, using validation data to select decomposition levels and hidden dimension.The search covers 1–5 decomposition levels and hidden dimensions {16, 24, 32, 48}.Source: §6.2.3 · p. 13

6.2.4 Results and Analysis

  • FTB outperforms competing baselines across most tasks on seven datasets, reducing average MSE by 24.45% and average MAE by 15.38%, with the best overall average on both metrics.The second-best model varies across tasks.Source: §6.2.4 · pp. 13–14, Table 6 · p. 14

6.2.5 Ablation of Important Modules

  • On Electricity, ablations indicate every component contributes; removing m-WCN or r/wd sharply decreases performance, while pre-training helps maintain performance and reduce overfitting risk.The pre-training task recovers MWD decomposition outcomes and constrains learning.Source: Table 7 · p. 14, §6.2.5 · p. 14

6.2.4 Results and Analysis

  • FTB generally shows smoothly increasing prediction error as the forecast horizon lengthens, while several baselines show more dataset-dependent trends.The authors interpret this pattern as consistent with increasing difficulty in long-term forecasting.Source: §6.2.4 · p. 14

6.2.6 Efficiency Comparison

  • FTB takes 63.76 seconds per training epoch and 6.24 seconds for inference, with 464.07 MB training memory and 407.62 MB inference memory.Measurements average across seven datasets on one NVIDIA GeForce RTX 3090 GPU, with input and prediction lengths of 336.Source: Table 8 · p. 14, §6.2.6 · p. 14

6.2.7 Prediction Visualization

  • On 96-step Electricity samples, FTB captures rising and falling periodic patterns and high- and low-frequency detail in non-periodic series, where PatchTST misses some patterns or details.The visualizations compare the two models on the same periodic and non-periodic samples.Source: §6.2.7 · pp. 14–15, Figure 5 · p. 15

7 CONCLUSION AND FUTURE WORK

The conclusion presents m-WCN as an end-to-end framework combining frequency and pattern decomposition, with TFBC and FTB built for classification and forecasting. The authors report strong benchmark performance while identifying multivariate handling as a current limitation.

7 CONCLUSION AND FUTURE WORK

  • m-WCN integrates frequency and pattern decomposition in a trainable end-to-end architecture, using neuralized MWD and orthogonality regularization to support adaptability and interpretability.The framework is described as integrable into deep-learning pipelines.Source: §7 · p. 15
  • TFBC and FTB are designed for classification and forecasting, respectively, and experiments report superiority to state-of-the-art baselines across diverse real-world settings.The authors characterize the models as robust and versatile across those settings.Source: §7 · p. 15
  • The current framework is naturally suited to univariate time series; multivariate use requires handling variables independently, which the authors describe as cumbersome.Extending m-WCN to support multivariate series more naturally is identified as future work.Source: §7 · p. 15

Related papers

Loading 2609.29317v1…