Highly comparative feature-based time-series classification

Ben D. Fulcher; Nick S. Jones

Source-linked AI summary

Highly comparative feature-based time-series classification

Ben D. Fulcher, Nick S. Jones

arXiv:1401.3531v2Jan 15, 2014cs.LGcs.AIcs.DBphysics.data-anq-bio.QM

TL;DR

Time-series classification traditionally depends on representations and distances that may be difficult to choose. This paper instead compares thousands of scientific features, selects informative ones for classification, and reports strong performance with compact representations while noting library and training-cost limitations.

Problem
Researchers must choose representations and distance measures for time-series classification, while the best features may be unclear when data-generating mechanisms are poorly understood.
Method
The method computes a large set of scientific time-series features and automatically selects a reduced subset to construct feature-based classifiers.
Results
An average of 3.2 features represented the twenty studied UCR datasets, and feature-based classification was often superior to DTW, sometimes by a large margin.
Takeaways & Limitations
Selected features identify properties informative of dataset class structure, providing understanding beyond classification performance.
Takeaways & Limitations
Classifiers are limited by the features in the current library, and training costs remain open to optimization by exploiting redundancy.

Abstract

from arXiv · show

A highly comparative, feature-based approach to time series classification is introduced that uses an extensive database of algorithms to extract thousands of interpretable features from time series. These features are derived from across the scientific time-series analysis literature, and include summaries of time series in terms of their correlation structure, distribution, entropy, stationarity, scaling properties, and fits to a range of time-series models. After computing thousands of features for each time series in a training set, those that are most informative of the class structure are selected using greedy forward feature selection with a linear classifier. The resulting feature-based classifiers automatically learn the differences between classes using a reduced number of time-series properties, and circumvent the need to calculate distances between time series. Representing time series in this way results in orders of magnitude of dimensionality reduction, allowing the method to perform well on very large datasets containing long time series or time series of different lengths. For many of the datasets studied, classification performance exceeded that of conventional instance-based classifiers, including one nearest neighbor classifiers using Euclidean distances and dynamic time warping and, most importantly, the features selected provide an understanding of the properties of the dataset, insight that can guide further scientific investigation.

I. INTRODUCTION

The paper introduces an automated, highly comparative feature-based approach to time-series classification that replaces direct distance comparisons with data-driven selection of informative time-series properties. The selected features both support classification and reveal properties associated with class structure.

Time-series classification conventionally requires choosing both a representation and a dissimilarity measure, often by comparing sequential values directly.
Feature-based classification transforms time series into short vectors of derived properties, such as mean and variance, rather than comparing ordered measurements.
The proposed method computes thousands of features from scientific time-series analysis methods and automatically selects those with the best classification performance.
Feature selection is data-driven, so different classification problems can prioritize properties such as variance or entropy without requiring knowledge of the underlying dynamical mechanisms.
The selected features can identify properties informative of dataset class structure, providing understanding that may guide further investigation.

II. DATA AND METHODS

The paper constructs a broad feature-based representation of time series and describes the feature-selection and classification methodology used to learn from it.

The method represents time series using a large and diverse set of measured properties.
The paper organizes its methodology around datasets, feature-vector construction, and feature selection with classification.

A. Data

The evaluation uses twenty labeled, univariate UCR time-series datasets spanning varied lengths, dataset sizes, class counts, and application domains. Results use the source partitions without preprocessing, with partition sensitivity assessed separately.

Twenty labeled, univariate datasets from the UCR repository are analyzed using the source data and specified training–test partitions.
The datasets vary in length from N = 60 for Synthetic Control to N = 637 samples for Lightning.
Dataset sizes range from ntrain = 28 and ntest = 28 for Coffee to ntrain = 1 000 and ntest = 6 164 for Wafer.
The number of classes ranges from nclasses = 2 for Gun point to nclasses = 50 for 50 Words.
The datasets cover systems including silicon-wafer sensors, lightning strikes, Swedish leaves, and yoga poses.
Test-set classification rates are mostly similar across alternative partitions, so the main text reports the specified partitions for comparison with other studies.

B. Feature vector representation

The feature representation applies more than 9 000 algorithmic operations to each time series, producing scalar features that can be filtered into a smaller classification representation. This approach avoids pairwise distance calculations but requires handling invalid or poorly estimated operations.

The feature database contains over 9 000 time-series analysis operations spanning diverse scientific methods and properties.
Each operation maps a time series x = (x1, x2, ..., xN) to one real-valued output, called a feature.
Feature-based classification compares extracted feature sets, whereas instance-based methods compare ordered time-series values using distances such as dynamic time warping.
The resulting feature vector is filtered using labeled training data to obtain a reduced representation for learning a classification rule.

C. Feature selection and classification

The method represents each time series with thousands of features, then uses interpretable greedy forward selection and a linear classifier to retain a compact, discriminative subset. This avoids pairwise distance calculations and supports substantial dimensionality reduction.

Feature representation: Thousands of extracted features are reduced to a compact set by selecting those that best distinguish known classes.The initial feature vectors provide the basis for selecting a reduced representation.
Feature selection: Greedy forward selection incrementally adds the feature that yields the greatest improvement in linear classification rate.The procedure begins with the strongest individual feature and repeats until a termination criterion is reached.
Termination: Selection stops when the training-rate improvement falls below 3% or training misclassification reaches 0%.Results are reported as not highly sensitive to the 3% threshold.
Classification: A linear discriminant classifier estimates class boundaries from selected features, using pairwise boundaries and majority voting for multiclass problems.The classifier fits a multivariate normal density to each class with pooled covariance.
Baselines: The feature-based classifier is evaluated against Euclidean 1-NN, DTW 1-NN, and DTW 1-NN with a learned warping window.The comparison uses results from the UCR time-series classification source, with Euclidean results independently verified.
Dimensionality reduction: The representation reduces each ordered time series from N measurements to a single extracted feature in illustrative datasets.Feature distributions are shown separately for labeled training and test classes.

III. RESULTS

The results section illustrates the feature-based approach on selected datasets, compares it with instance-based methods across twenty datasets, and discusses computational complexity.

Results overview: The results demonstrate the approach using selected datasets before broader comparison and complexity analysis.The section covers illustrative examples, comparisons across all twenty datasets, and computational complexity.

A. Selected datasets

Selected datasets show that one or a few interpretable features can classify classes accurately while reducing time-series dimensionality. When individual features are insufficient, automatically selected combinations improve performance and reveal class distinctions.

Trace: 99% test accuracy on Trace matches DTW using one time-reversal asymmetry feature, avoiding pairwise time-series distance calculations.The feature-based representation reduces the time series to a single extracted feature.
Wafer: 99.98% test accuracy on Wafer slightly exceeds Euclidean 1-NN’s 99.5% using one pattern-frequency feature.The feature counts successive decrease-increase-decrease-increase patterns as a proportion of series length.
Synthetic Control: The first Synthetic Control feature distinguishes most classes but confuses uncorrelated random and cyclic series without an underlying trend.The second feature complements this weakness, although alone it has a 52.3% test misclassification rate.
Feature selection: The Synthetic Control classifier terminates after two features because the next feature improves training classification by less than 3%.The figure compares training and test misclassification rates as features are added.
Interpretability: The selected Synthetic Control features are interpretable because their values correspond to trend behavior and spectral concentration across classes.The complementary features show how time-series properties support classification rather than only providing opaque distance comparisons.
OSU Leaf: On OSU Leaf, five features achieve 16.5% test misclassification and outperform all instance-based classifiers despite reducing 427-sample series to five features.Using eleven features lowers test misclassification to 9%, while later additions show signs of modest over-fitting.

B. All results

Across twenty datasets, feature-based and instance-based classifiers often achieved comparable results, with each approach substantially outperforming the other on some datasets. Feature selection used very few interpretable properties and produced large dimensionality reduction, while performance depended on dataset characteristics.

Feature-based and 1-NN DTW classifiers produced broadly similar results for most datasets, but some showed large differences between methods.
Selected features covered autocorrelation, automutual information, symbolic motifs, spectral properties, entropy, stationarity, outliers, and scaling behavior.
Feature-based classification was less suited to some datasets, including cases with few heterogeneous examples, many classes, or unaligned patterns.
1.0% was the ECG feature-based test-set misclassification rate using one feature, versus 12.0% for the best Euclidean 1-NN and DTW 1-NN classifiers.
0% was the Coffee feature-based test-set misclassification rate using one feature, versus 17.9% for the best DTW instance-based classifiers.
Feature selection averaged nfeat = 3.2 features for time series averaging N = 282.1 samples, yielding orders-of-magnitude dimensionality reduction with often comparable or superior performance.

C. Computational complexity

Feature-based classification avoids time-domain comparisons during prediction by using a static representation, shifting computational effort toward one-time training. With suitable selected features, prediction scales independently of training-set size and can handle very long series efficiently.

Feature-based classification avoids time-domain comparisons by representing each time series with extracted features and a static classification rule.
Training requires computing thousands of features, but this one-off cost allows new time series to be classified independently of the training data.
O(Nnfeat) is the serial prediction complexity when selected features each scale as O(N), independent of training-dataset size.
The prediction process does not require training data in memory, unlike instance-based classification of large datasets.
Both demonstrated feature operations scaled approximately linearly with time-series length, as O(N), across time series of different lengths.
A 100 000-sample time series was converted to a single selected feature in under 5 ms using either trev(τ = 3) or a specified symbolic motif frequency.
The method’s computational burden depends on selected features: nonlinear operations can scale nonlinearly with N, while future searches could restrict features to O(N).
On Wafer, 0.1 s classified all 6 164 test series after loading them, with 6 163 classified correctly, or 99.98%.

IV. DISCUSSION

The discussion presents highly comparative feature-based classifiers as interpretable, compact alternatives to instance-based methods, while identifying computational, methodological, and interpretive limitations. Across twenty UCR datasets, classifiers used few selected features, often performed well against DTW, and enabled rapid classification after training.

Contribution: The method selects interpretable properties from a large feature library to construct feature-based classifiers for time series.Greedy forward selection and linear discriminant classification were chosen to demonstrate the approach clearly and produce interpretable results.
Interpretability: Features included outlier, entropy, local motif, and autocorrelation statistics that reveal properties differing between labeled classes.The selected features provide dataset-specific insight into class structure rather than relying only on direct time-series distances.
Results: 3.2 selected features represented time series averaging 282.1 samples, producing two orders of magnitude of dimensionality reduction.The compact representation was reported across the twenty UCR datasets studied.
Results: Despite losing direct pattern matching through time, feature-based representations achieved good performance, often exceeding DTW and sometimes by a large margin.The paper emphasizes that strong classification performance remained possible despite the substantial dimensionality reduction.
Computational trade-offs: Training incurs significant one-off computational expense, but subsequent classification is extremely rapid and independent of training-set size.The authors suggest exploiting redundancy in the massive feature set to reduce training cost.
Limitations: Classification performance and selected features depend on methodological choices, including greedy selection, class imbalance, feature-library coverage, and classifier complexity.Unequal class proportions bias total classification rate toward larger classes, while complex classifiers may overfit and useful features may lie outside the current library.

V. CONCLUSIONS

The paper demonstrates that highly comparative feature-based representations can produce competitive classifiers while reducing dimensionality and revealing class differences. The approach adapts feature and classification-rule selection to each labeled dataset without requiring domain knowledge.

The study applies an unprecedented diversity of scientific time-series analysis methods to classification problems.
Successful classifiers are produced by learning relevant features and classification rules from each dataset’s labeled structure.This avoids requiring domain knowledge about how the data were generated or measured.
Selected features reveal differences in properties between labeled classes, providing insights that can guide further scientific investigation.The feature-generation code is freely available.
Competitive classifiers remain achievable despite orders-of-magnitude dimensionality reduction from the original time-series representations.
After feature selection and rule learning, classifying new time series is rapid and can outperform instance-based classification.The intensive computation is concentrated in generating thousands of features, particularly when computation is not distributed.
The approach applies to variable-length time series and to series many orders of magnitude longer than those studied.