PaperlayerView exact revision on arXiv

Source-linked AI summary

GRATIS: GeneRAting TIme Series with diverse and controllable characteristics

Yanfei Kang, Rob J Hyndman, Feng Li

arXiv:1903.02787v2stat.MLcs.LGstat.CO

TL;DR

Time-series analysis methods need diverse benchmarks for reliable evaluation, but collected datasets can be homogeneous and existing generation approaches have coverage limits. GRATIS uses mixture autoregressive models to generate varied series and tune them to target features. The authors report broader diversity than existing benchmark datasets and improved forecasting accuracy for their feature-based scheme on M3 data.

  • Problem

    Collected time-series datasets can be homogeneous, and the field lacks diverse, controllable benchmarks for evaluating methods across varied data.

    Source: §1 · pp. 1–2

  • Method

    GRATIS simulates time series with mixture autoregressive models and tunes their parameters to generate series with controllable target features.

    Source: §1 · p. 3, §6 · p. 27

  • Results

    MAR-generated data show greater diversity than existing benchmark datasets, while generated-data-trained feature-based selection has the lowest MASE across M3 groups and overall.

    Source: §3.2 · p. 17, §5.2 · p. 24

  • Takeaways & Limitations

    GRATIS provides a benchmarking resource for forecasting comparison, model averaging, and training time-series models with self-generated data.

    Source: §6 · p. 27

    Default settings leave some feature-space areas denser than others, so coverage may require parameter adjustment, more simulations with accept-reject sampling, or targeted-feature generation.

    Source: §3.2 · p. 17

Abstract

from arXiv · show

The explosion of time series data in recent years has brought a flourish of new time series analysis methods, for forecasting, clustering, classification and other tasks. The evaluation of these new methods requires either collecting or simulating a diverse set of time series benchmarking data to enable reliable comparisons against alternative approaches. We propose GeneRAting TIme Series with diverse and controllable characteristics, named GRATIS, with the use of mixture autoregressive (MAR) models. We simulate sets of time series using MAR models and investigate the diversity and coverage of the generated time series in a time series feature space. By tuning the parameters of the MAR models, GRATIS is also able to efficiently generate new time series with controllable features. In general, as a costless surrogate to the traditional data collection approach, GRATIS can be used as an evaluation tool for tasks such as time series forecasting and classification. We illustrate the usefulness of our time series generation process through a time series forecasting application.

1 Introduction

Time-series datasets are often homogeneous and the field lacks diverse, controllable benchmarks for evaluating analysis methods. The paper introduces GRATIS, which uses mixture autoregressive models to generate varied series and supports feature-targeted generation.

1 Introduction

  • Time-series mining performance depends on training-data diversity for generalization, yet datasets are often homogeneous and diverse, controllable benchmarks remain scarce.The paper notes that existing benchmarks such as UCR and M4 address particular tasks, but the field still lacks broadly diverse, controllable evaluation data.Source: §1 · pp. 1–2
  • Earlier generators trade breadth for speed or scope: Kang et al. (2017) use a slow genetic algorithm, while Kegel et al. (2017) generate only additive trend and seasonality.Talagala et al. (2018) augment data by simulating from exponential-smoothing and ARIMA processes.Source: §1 · p. 3
  • GRATIS uses Gaussian mixture autoregressive models to generate a wide range of non-Gaussian and nonlinear series as heterogeneous benchmarking data.The authors present it as requiring minimal human effort and computational resources.Source: §1 · pp. 3–4
  • Unlike Gaussian-mixture methods that generate classification instances from desired features, GRATIS reverses the procedure to generate time series with specified features.This approach is presented as consistent with prior finite-mixture work, including Muñoz and Smith-Miles (2017).Source: §1 · p. 4
  • The workflow simulates MAR series, extracts features, embeds them in a two-dimensional instance space for coverage comparisons, and tunes MAR parameters to target features.The paper also applies the generation process to a forecasting application.Source: §1 · p. 4

2 Time series generation from MAR models

The section uses mixture autoregressive models to generate diverse time series by varying model parameters, and extends the approach to series with multiple seasonal patterns. The generated series can be formed from weighted combinations of simulated series at different frequencies.

2.1 Mixture autoregressive models

  • MAR models combine autoregressive components and can represent nonlinear, non-Gaussian, heteroskedastic, seasonal, and non-stationary time series.Mixtures with distinct component means and unbalanced weights can simulate recurrent extremes.Source: §2.1 · pp. 4–6

2.2 Diverse time series generation

  • The simulation draws parameter values from distributions rather than fixing them, generating 20,000 yearly, 20,000 quarterly, 40,000 monthly, and 10,000 weekly series.Series lengths are drawn from M4 data; the examples are compared with closest M4 series in feature space.Source: §2.2 · p. 7, Figure 1 · p. 8

2.3 Multi-seasonal time series generation

  • For multiple seasonal patterns, the method aggregates simulated series at the corresponding frequencies using weights that sum to one.The illustrated half-hourly Victoria demand series has daily and weekly patterns at frequencies 48 and 336, respectively.Source: §2.3 · pp. 9, 11, Figure 2 · p. 10

3 Diversity and coverage of generated time series

The study characterizes time series with a diverse, scale-independent feature set and compares simulated series with forecasting benchmarks in feature space. MAR-generated series cover benchmark feature ranges and show greater diversity than existing benchmarks, though their feature-space density is uneven.

3.1 Time series features

  • The analysis uses 26 diverse features restricted to be ergodic, stationary, and independent of scale.Some new features are intended to measure multiple seasonality, non-stationarity, and heterogeneity.Source: §3.1 · pp. 11–12, Table 3 · p. 12
  • For multiple-seasonal series, F2 records the number of seasonal periods, F3 lists those periods, and F18 measures seasonal strength separately for each period.For example, monthly data has F3 = 12, while hourly data can have F3 = (24, 168, 8760)′.Source: §3.1 · pp. 12–13

3.2 Diversity and coverage analysis

  • Simulated feature-plot shapes match theoretical ranges, and their feature quantiles cover the shapes of features in the benchmark datasets.Figure 3 compares simulated yearly, quarterly, monthly, and weekly series with M1, M3, M4, NNGC1, and Tourism.Source: §3.2 · p. 13, Figure 3 · p. 14
  • t-SNE captures nonlinear structure in the M3 feature data better than PCA, whose linear projection loses substantial information, especially with more features.Figure 4 displays period and entropy distributions in two dimensions, with t-SNE on the left and PCA on the right.Source: §3.2 · pp. 13, 16, Figure 4 · p. 15
  • The complement of pairwise miscoverage can indicate dataset A’s relative diversity and, against real data, the simulated set’s realism.The comparison uses feature-space grids to quantify miscoverage between datasets.Source: §3.2 · p. 17
  • Default settings leave some feature-space areas more populated than others; the authors suggest retuning, more simulations with accept–reject, or target-feature generation to address this.The coverage analysis indicates that generation in a specified feature space is possible.Source: §3.2 · p. 17
  • 0.017, 0.013, 0.030, and 0.001 are DGP-over-M4 miscoverage values for yearly, quarterly, monthly, and weekly series; M4-over-DGP values are substantially higher, indicating greater diversity in the simulations.The table caption gives the yearly pair: simulated data over M4 is 0.017, while M4 over simulated data is 0.053.Source: §3.2 · p. 17, Table 5 · p. 18

4 Efficient time series generation with target features

GRATIS tunes a MAR model’s parameters to generate series near a chosen target in feature space, rather than evolving each time-series value. This lower-dimensional search makes target-feature generation more efficient than the compared approach.

4.1 Tuning a MAR model with target features

  • The genetic algorithm tunes MAR parameters in a lower-dimensional space than evolving all n time-series values.It seeks MAR parameters whose simulated series’ feature vector is close to a selected target feature vector.Source: §4.1 · p. 19
  • The search starts from a selected target feature point and a randomly initialized population of MAR parameter vectors.The target may come from an existing series, and the initial parameter values are sampled uniformly from the ranges in Table 1.Source: §4.1 · p. 19
  • Each iteration simulates a series for each parameter vector and scores it by scaled distance from the target features.The distance measure can be Euclidean; the scaling constant is usually the norm of the target feature vector.Source: §4.1 · p. 20
  • Crossover, mutation, and survival of the fittest produce successive generations, and the closest series is retained for each target.The selected series has the largest fitness value.Source: §4.1 · p. 20
  • For very short target series, feature calculations involving lags require series length to exceed the lag length.Expert information can also be supplied through the settings in Table 1, especially when historical data are limited or particular features are desired.Source: §4.1 · p. 20

4.2 Efficiency analysis

  • 40 times faster on average for a comparable task, GRATIS generated series with twice as many features as Kang et al. (2017).The comparison concerns generating 100 series of length 100; the authors say the speedup makes controllable generation feasible in reasonable time.Source: §4.2 · p. 20, Table 6 · p. 21

5 Application to time series forecasting

The paper trains a feature-based forecasting selector on generated time series, using time-series features to predict forecasting methods’ MASE and choose or average methods. In M3, the authors report lower median MASE for selection than for the individual methods across each group and overall.

5.1 Forecasting based on feature spaces in generated time series

  • For training, GRATIS simulates 10,000 yearly, quarterly, and monthly series, then features and out-of-sample MASE are computed for nine forecasting methods.The horizons are 6, 8, and 18 for yearly, quarterly, and monthly series, respectively.Source: §5.1 · p. 22
  • Nonlinear regression relates training features to method-specific MASE; test-series features then guide method selection or forecast averaging.MASE is the accuracy measure used, though the authors say the method can in principle use other forecasting measures.Source: §5.1 · p. 22
  • The approach trains solely on generated series and can select a method without testing every candidate on the test series.The authors also state it needs no real data for training and takes transformed, nonsensitive features rather than raw series as forecasting input, which they identify as useful when privacy is a concern.Source: §5.1 · p. 23

5.2 Application to M3 data

  • Feature-based selection trained with GRATIS had the lowest median MASE in every M3 group and overall; averaging further improved MASE.Table 7 compares the selection and averaging procedures with nine individual forecasting methods.Source: §5.2 · p. 24, Table 7 · p. 24
  • M3 boxplots show the proposed method is best on average and avoids very large MASE deviations relative to the nine other methods.The figure covers yearly, quarterly, monthly, and overall M3 data.Source: §5.2 · p. 24, Figure 7 · p. 26
  • The M3 feature-space projections show distinct patterns in which forecasting methods are predicted to perform best.The projections cover yearly, quarterly, and monthly series.Source: §5.2 · p. 24, Figure 8 · p. 27
  • Fitting MAR models directly to yearly M3 series yielded median MASE 2.311, which the authors say is much greater than selection and averaging MASE.They interpret this as evidence that MAR need not fit each candidate series adequately to generate a sufficiently rich collection for predicting method performance from series characteristics.Source: §5.2 · p. 25

6 Conclusions and discussion

The conclusions present GRATIS as an efficient MAR-based generator for realistic, diverse series with broad feature-space coverage and controllable target features. The authors identify forecasting and classification as applications and outline extensions to other data settings and deep-learning pretraining.

6 Conclusions and discussion

  • GRATIS uses MAR models with statistically distributed parameters to produce realistic series with higher feature-space coverage than existing time-series benchmarks.The authors describe the method as requiring minimal human effort and computational resources and propose its generated data for benchmarking and model training.Source: §6 · p. 27
  • Tuning MAR parameters supports efficient generation for chosen features, including settings where only particular features matter.The authors report a 40-times speedup over Kang et al. (2017) for generating series from given features.Source: §6 · p. 27
  • The paper demonstrates a forecasting application based on features of generated series and points to training forecasting models with generated data as a potential direction for private data.The authors also list extensions to exogenous, multivariate, cross-sectional, and discrete series, and to deep-learning pretraining.Source: §6 · pp. 27–28

A.1 Description of time series features

The appendix represents series with feature vectors covering length and seasonal structure, dependence, forecastability, nonlinearity, memory, stability, shifts, decompositions, and heterogeneity. It describes how the feature groups are computed and notes properties that make the features suitable for diverse series.

A.1 Description of time series features

  • The authors state that the features are ergodic for stationary and difference-stationary processes and do not depend on series scale.They therefore describe the features as well-suited to application across a large, diverse set of series.Source: §A.1 · p. 28
  • Each series is summarized by a feature vector, with length and seasonal-period features among its components.The passage reports 42 elements for non-seasonal and single-seasonal series, and a few more for multiple-seasonal series.Source: §A.1 · pp. 28–29
  • Dependence and predictability features include autocorrelation, partial autocorrelation, and spectral entropy, where larger entropy indicates a harder-to-forecast series.The appendix also defines a nonlinearity coefficient as a modified test statistic divided by series length, converging to an indication of nonlinearity as length tends to infinity.Source: §A.1 · p. 29
  • Other features capture long-term memory, stability and lumpiness, unit-root test statistics, and shifts between adjacent windows.Stability is the variance of window means and lumpiness the variance of window variances; shift features measure changes in mean, variance, and KL divergence.Source: §A.1 · pp. 29–30
  • The decomposition-based features extend STL to multiple seasonal components and quantify trend, seasonal strength, spikiness, linearity, curvature, and remainder autocorrelation.For non-seasonal series, the trend is estimated with Friedman’s super smoother; the appendix says the seasonal procedure uses two iterations and a fixed window span of 21.Source: §A.1 · p. 30
  • Heterogeneity features pre-whiten the series, fit GARCH(1,1), and measure ARCH effects in the series and residuals.The feature set includes sums of squared autocorrelations and autoregressive-model R2 values for squared series and residuals.Source: §A.1 · p. 31

A.2 Web application for time series generation

The web application lets users choose time-series features and settings, then displays the generated series. Users can also download the simulated data.

A.2 Web application for time series generation

  • Users specify the desired seasonal pattern, series length, number of series, and required feature values.The interface first lets users choose the features of interest.Source: §A.2 · p. 31
  • The application displays generated series after users set their desired features and series settings.Source: §A.2 · p. 31
  • Simulated data can be downloaded to a local computer.Source: §A.2 · p. 31

A.3 Penalized linear quantile regression

The section defines conditional quantile regression for MASE values given feature vectors, then describes adaptive-LASSO estimation and its linear-programming formulation.

A.3 Penalized linear quantile regression

  • The conditional τth quantile function fτ(xi) is defined for MASE values yi given feature vectors xi.The sample consists of {yi, xi} for i = 1, ..., n.Source: §A.3 · p. 31
  • Conditional quantile regression with the adaptive LASSO penalty is estimated by solving a minimization problem.Source: §A.3 · p. 33
  • Slack variables convert the convex optimization problem equivalently into a linear program solvable with standard optimization software.A comparison of existing algorithms is referenced to Peng and Wang (2015).Source: §A.3 · p. 33

A.4 Genetic Algorithms

Genetic algorithms search optimization problems through evolving populations of candidate solutions. Their process uses fitness evaluation and genetic operators, with convergence criteria governing when evolution stops.

A.4 Genetic Algorithms

  • Candidate chromosomes represent potential optimization solutions through a mapping into the decision-variable space.The method treats parameters as a population and evaluates fitness during the process.Source: §A.4 · p. 33
  • A genetic algorithm takes variable types, a fitness function, and convergence criteria as inputs, and evolves an initial population until convergence.The algorithm's output is an evolved population.Source: §A.4 · p. 33
  • Selection samples individuals with replacement according to survival probabilities, favoring qualified individuals and their offspring.Source: §A.4 · p. 33
  • Crossover combines genetic information from two parent chromosomes, while mutation randomly alters gene values in a parent chromosome.Source: §A.4 · p. 33
Loading 1903.02787v2…