PaperlayerView exact revision on arXiv

Source-linked AI summary

An Empirical Survey of Data Augmentation for Time Series Classification with Neural Networks

Brian Kenji Iwana, Seiichi Uchida

arXiv:2007.15951v4cs.LGstat.ML

TL;DR

Small time series datasets motivate data augmentation, but selecting suitable methods remains difficult because effects vary across datasets and neural architectures. This paper surveys and taxonomizes four augmentation families, evaluates 12 methods on 128 classification datasets with six neural-network types, and reports mixed model-dependent results with Window Warping and slicing among the strongest practical choices.

  • Problem

    Many time series recognition datasets are small, while data augmentation is not yet a standard procedure and method suitability varies across datasets.

  • Method

    The paper surveys and taxonomizes transformations, pattern mixing, generative models, and decompositions, then evaluates 12 methods across 128 datasets and six neural-network types.

  • Results

    Results were mixed and architecture-dependent: Slicing, Window Warping, and DGW tended to help, while Rotation, Permutation, and Time Warping often degraded accuracy.

  • Takeaways & Limitations

    Window Warping is recommended as a general, easy-to-use method, with slicing also effective in most cases, while other models require careful method selection.

  • Takeaways & Limitations

    Pattern-mixing methods can be slow for longer patterns, so the paper recommends them mainly for shorter time series.

Abstract

from arXiv · show

In recent times, deep artificial neural networks have achieved many successes in pattern recognition. Part of this success can be attributed to the reliance on big data to increase generalization. However, in the field of time series recognition, many datasets are often very small. One method of addressing this problem is through the use of data augmentation. In this paper, we survey data augmentation techniques for time series and their application to time series classification with neural networks. We propose a taxonomy and outline the four families in time series data augmentation, including transformation-based methods, pattern mixing, generative models, and decomposition methods. Furthermore, we empirically evaluate 12 time series data augmentation methods on 128 time series classification datasets with six different types of neural networks. Through the results, we are able to analyze the characteristics, advantages and disadvantages, and recommendations of each data augmentation method. This survey aims to help in the selection of time series data augmentation for neural network applications.

1 Introduction

Time series classification uses order-dependent structural patterns, but many recognition datasets are small, making data augmentation relevant for improving neural-network generalization. The paper organizes augmentation methods into four families and evaluates them across varied data and models.

  • Time series classification categorizes order-dependent sequences used in applications including signals, biometrics, sound, and trajectories.
  • Only 12 of 128 datasets in the 2018 UCR Time Series Archive have more than 1,000 training patterns.This illustrates the limited data available for many time series recognition tasks.
  • Data augmentation generates synthetic patterns to reduce overfitting, expand model decision boundaries, and support generalization on small or imbalanced datasets.
  • Unlike image recognition, data augmentation is not yet a standard procedure for time series recognition, where random noise, slicing, scaling, and temporal or frequency warping are common.
  • The taxonomy organizes augmentation into transformations, pattern mixing, generative models, and decompositions, with family, domain, and method as hierarchical levels.
  • The study surveys methods, evaluates them across diverse data types and neural models, and discusses their advantages, disadvantages, characteristics, and usage recommendations.

2 Random transformation-based data augmentation

Random transformation-based augmentation modifies existing time series in magnitude, time, or frequency domains. Its methods alter values, time steps, or frequencies, but their suitability depends on dataset properties and transformation assumptions.

  • Random transformation-based augmentation generates a new pattern x′ from a reference sequence x using transformation functions.
  • Transformations operate in magnitude, time, or frequency domains, with hybrid methods and serial or parallel combinations also possible.
  • Magnitude domain: Jittering adds Gaussian noise to each time step, with σ controlling noise magnitude and the assumption that unseen patterns differ by noise.
  • Magnitude domain: Rotation may change a time series’ class and has shown no effect or detrimental effects in some studies, although improvement was reported when combined with other methods.
  • Magnitude domain: Scaling multiplies every value by a random scalar, whereas magnitude warping multiplies values by a smoothly varying interpolated curve.
  • Magnitude domain: Magnitude warping requires two hyperparameters and still assumes the random transformation is realistic for the dataset.
  • Time domain: Time-domain methods displace elements across time steps; slicing removes end segments, while permutation rearranges segments without preserving time dependencies.
  • Time and frequency domains: Time warping perturbs temporal structure through smooth paths or fixed windows, while frequency-domain transformations target periodic signals and acoustic data.

3 Pattern mixing

Pattern mixing creates new time series by combining existing patterns rather than applying transformations assumed to be typical. It relies on the premise that similar patterns can be combined reasonably.

  • Pattern mixing combines one or more existing patterns to generate new ones without assuming every transformation is appropriate for the dataset.
  • Magnitude-domain mixing: Magnitude-domain mixing linearly combines values at corresponding time steps, including averaging and interpolation between reference patterns.
  • Interpolation: SMOTE interpolates an under-represented sample with one of its k-nearest neighbors using a random λ in {0, 1}.
  • Guided warping: Guided warping uses DTW alignment to warp a reference pattern according to a teacher pattern’s time steps.
  • Other domains: Pattern mixing can also combine patterns in frequency or multiple domains, including sound mixing with frequency-band perturbations.
  • Warping and averaging: SPAWNER uses constrained suboptimal time warping and aligned-pattern averaging to create an almost unlimited number of new time series.
  • Warped averaging: DBA averages time-aligned series while preserving underlying patterns, and wDBA extends it with weighting schemes such as Average All, Average Selected, and Average Selected with Distance.

4 Generative models

Generative augmentation samples new time series from learned feature distributions rather than directly transforming or mixing existing patterns. The survey distinguishes statistical models from neural-network-based generators.

  • Generative models sample time series from feature distributions and are categorized as statistical models or neural-network-based models.
  • Statistical models: Statistical methods build models of the data and include forecasting models, Gaussian trees for imbalanced classes, and mixture autoregressive models for simulation.
  • Neural-network models: Direct sequence-to-sequence networks, including LSTMs and temporal CNNs, can generate sequences for data augmentation.
  • Neural-network models: Encoder-decoder methods generate new patterns by decoding vectors sampled from a lower-dimensional latent space.
  • GANs: GANs jointly optimize a generator and discriminator, then use sampled z-vectors with the generator to create new time series.
  • GANs: Time series GAN architectures include fully connected, recurrent, temporal or 1D convolutional, and spectrum-based 2D convolutional networks.
  • Conditional generation: Conditional GANs control generated patterns through conditions supplied to the generator and discriminator, with reported results varying across comparisons.

5 Time series decomposition

Decomposition methods generate augmented time series by extracting, recombining, or perturbing underlying features and patterns. Examples include EMD, ICA-based reconstruction, and STL-based resampling.

  • Decomposition methods extract underlying features or patterns, which can be independently used, recombined, or perturbed to generate augmented data.
  • Empirical Mode Decomposition targets nonlinear and non-stationary signals and improved classification when augmenting noisy automobile sensor data in a CNN-LSTM.
  • ICA combined with D-FANN models independent signal components, reconstructs the transformed signal, and increased MLP performance for filling time-series gaps.
  • STL decomposes signals into seasonal, trend, and remainder components, with one method bootstrapping the remainder using a moving block bootstrap before reassembly.

6 Comparative evaluations

The comparative evaluation tests general-purpose augmentation methods across 128 UCR datasets and six neural-network architectures. Standardized training settings and architecture-specific designs support consistent within-model comparisons.

  • Datasets: The evaluation uses 128 datasets from the 2018 UCR Time Series Archive, covering device, biomedical, object-contour, motion, sensor, simulated, and other time-series categories.
  • Network models: Six evaluated models are 1D VGG, 1D ResNet, MLP, LSTM, BLSTM, and LSTM-FCN, spanning convolutional, recurrent, feedforward, and hybrid architectures.
  • Training protocol: All evaluations use 10,000 iterations, plateau-based learning-rate reduction, and training sets augmented to four times their original pattern count.
  • Augmentation methods: Selected augmentation methods are general methods applicable to any time series, excluding frequency-domain methods and methods requiring external training, such as generative models.
  • Augmentation methods: The evaluated methods include no augmentation, jittering, rotation, scaling, magnitude warping, permutation, and slicing, with specified random or geometric transformations.

7 Results and discussion

Results vary substantially with neural-network architecture: some augmentations improve accuracy while others reduce it. Slicing and related time-domain methods are generally favorable, whereas several methods are detrimental, especially for MLP and LSTM-FCN.

  • Tables 1–3 report average results by augmentation and model, alongside paired t-tests comparing each augmentation with no augmentation.
  • Augmentation methods show dramatic architecture-dependent differences in results across the comparative evaluations.
  • Slicing, Window Warping, and DGW tend to help across models, while Rotation, Permutation, and Time Warping significantly degrade accuracy.
  • Every augmentation tends to improve VGG most, with Slicing producing its largest gain; augmentation is mostly detrimental for MLP and LSTM-FCN.

7.1 Differences between augmentation methods

PCA visualizations illustrate how augmentation methods alter the generated-pattern distribution relative to original training data. Similarity to the originals can correspond to smaller accuracy changes, while inappropriate transformations can reduce performance.

  • PCA projects original and generated GunPoint training patterns onto the first two principal axes to compare augmentation-induced distributions.
  • wDBA and Jittering produce patterns similar to the originals, and both generally have minor accuracy effects on GunPoint.

7.2 Relationship between dataset properties and accuracy

The analysis relates augmentation gains to dataset properties, finding that dataset size, variance, class structure, and time-series length influence accuracy changes differently across methods and neural-network architectures.

  • Analysis procedure: The study correlates augmentation-induced accuracy change with training-set size, patterns per class, series length, dataset variance, intra-class variance, and class imbalance.Spearman’s rank correlation is used because many dataset properties are skewed and the coefficient is robust to outliers.
  • Training set size: For most model–augmentation pairs, accuracy gains are larger on smaller training sets, although several methods show weak, absent, or positive correlations.ResNet and LSTM-FCN show especially strong negative correlations, while Scaling with MLP and LSTM shows positive correlation.
  • Patterns per class: Patterns per class behaves similarly to total training-set size, but correlations are generally weaker, so other dataset properties should also guide augmentation selection.MLP, VGG, ResNet, and LSTM show similar correlations for both properties; LSTM-FCN is a notable exception for several methods.
  • Time series length: Slicing gains increase with series length for CNN-based models, whereas correlations are generally negative for LSTM and BLSTM.The paper suggests longer series may contain less information at endpoints, which Slicing removes.
  • Variance: Dataset variance and intra-class variance generally correlate positively with augmentation gains, especially for CNN-based models and ResNet, respectively.LSTM is an outlier with negative correlations for magnitude-domain augmentation methods.
  • Class imbalance: As class imbalance rises, magnitude-domain methods tend to produce larger accuracy changes for selected models, while time-domain methods show negative or very small correlations.Positive correlations are reported for Jittering, Rotation, Scaling, and Magnitude Warping in specific model combinations.

7.3 Computation time

Theoretical complexity is similar for many augmentation methods, but observed execution time differs substantially, particularly between simple transformations and DTW-based pattern mixing.

  • Theoretical complexity: Simple transformations have theoretical complexity O(T), while complex transformations and pattern mixing methods have complexity O(T^2).T denotes the number of time steps.
  • Observed execution time: Transformation-based methods take a negligible fraction of a second on average to double a dataset.Execution times were measured by augmenting each dataset once using the Python implementation described in the study.
  • Observed execution time: DTW-based pattern mixing is much slower: SPAWNER and RGW take about a minute, while wDBA and DGW take 2,300 and 4,290 seconds on average.The overhead is driven primarily by repeated DTW alignment, with DGW additionally using slower shapeDTW.
  • Observed execution time: Long series can make DTW-based augmentation extraordinarily slow compared with transformation methods.HandOutlines, with 2,709 time steps, is given as an example; interpolation may avoid this issue because it does not use DTW.

7.4 Number of tunable parameters

Augmentation methods differ in the number of manually selected parameters: random transformations use fewer parameters but depend more heavily on hyperparameter choices, whereas pattern mixing offers fewer tuning choices despite greater complexity.

  • Parameter selection: Methods require manually selected hyperparameters, design choices, and variations, so parameter-rich methods may need repeated adjustment and evaluation.The paper lists manually defined parameters in Table 4.
  • Parameter selection: Random transformations generally have fewer parameters but are more dependent on hyperparameters because of their random element.Their effectiveness can therefore be sensitive to parameter settings.
  • Parameter selection: Pattern mixing methods are more complex but rely on dataset patterns for randomness, leaving fewer choices to tune.This contrasts their tuning burden with the parameter sensitivity of random transformations.

7.5 Recommendations on data augmentation usage

Recommendations depend on dataset type, model, and transformation behavior: fast time-domain methods are broadly useful, while pattern-mixing methods trade potential gains for computation time.

  • Dataset type and data augmentation method: Table 5 ranks the five highest-average-rank augmentation methods for each dataset category and model combination, breaking ties by execution time.It is intended as a general guide alongside Figure 5.
  • Magnitude domain transformations: Rotation decreased accuracy for all models and ranked highly only for power data with BLSTM.Flipping is often inappropriate for the shape-based datasets in the UCR archive.
  • Time domain transformations: Permutation severely harmed accuracy because it breaks time dependency, making it mainly plausible for periodic or very sparse series.Other time-domain transformations far outperformed it as general-purpose methods.
  • Time domain transformations: Slicing and Window Warping performed well across most datasets and models, significantly improving CNN-based VGG and ResNet models.Their positive effects and very fast computation time make them strong first choices for general-purpose augmentation.
  • Time domain transformations: Time Warping performed poorly because over-transforming series introduced noise and required careful parameter selection to avoid confusing classes.The user must balance sufficient transformation for generalization against excessive transformation.
  • Pattern matching methods: Pattern-mixing methods can improve generalization but have slow computation on longer patterns, so they are recommended mainly for shorter series.Their execution time is negligible for short time series, and accuracy changes negatively correlated with series length.
  • Pattern matching methods: wDBA usually produced disappointing results because its ASD weighting generated patterns insufficiently diverse from existing samples.The limitation may depend on the ASD weighting scheme.
  • Pattern matching methods: SPAWNER, RGW, and DGW had mixed results; DGW achieved the largest performance increase among pattern-mixing methods but was also the slowest.DGW had the highest average rank among all augmentation methods for BLSTM, while its computational cost limits use on long series.

8 Conclusion

The paper surveys and categorizes time-series augmentation methods, then compares them across neural networks and UCR datasets. It recommends Window Warping and Slicing broadly, while emphasizing model- and dataset-dependent selection and noting unresolved opportunities for combining methods.

  • The survey proposes a taxonomy covering transformation-based methods, pattern mixing, generative models, and decomposition methods.
  • The evaluation compares 12 augmentation methods across six neural network models and all 128 datasets in the 2018 UCR Time Series Archive.
  • Window Warping is the most recommended general-purpose method, while Slicing is also effective in most cases.Window Warping had the highest average rank for VGG, ResNet, and LSTM; DGW performed well but is recommended only for shorter series because of computational cost.
  • LSTM-FCN generally responded poorly to augmentation, VGG often improved, and ResNet, LSTM, and BLSTM required careful method selection.
  • Augmentation effects generally decreased with training-set size and increased with dataset and intra-class variance, with model-dependent correlations for series length and class imbalance.The study also ranked methods by model and dataset type.
  • The survey provides guidance for selecting augmentation by dataset type, dataset properties, and model, alongside an implementation of the evaluated algorithms.
  • The study evaluates one augmentation method per model, leaving possible synergies from serially combining multiple methods largely unexplored.
Loading 2007.15951v4…