catch22: CAnonical Time-series CHaracteristics

Carl H Lubba; Sarab S Sethi; Philip Knaute; Simon R Schultz; Ben D Fulcher; Nick S Jones

Source-linked AI summary

catch22: CAnonical Time-series CHaracteristics

Carl H Lubba, Sarab S Sethi, Philip Knaute, Simon R Schultz, Ben D Fulcher, Nick S Jones

arXiv:1901.10200v2Jan 29, 2019cs.IRcs.LGstat.ML

TL;DR

Feature-based time-series analysis is hindered by the computational cost of searching thousands of candidate features. The paper distills hctsa features into catch22, a compact set of 22 characteristics, retaining similar classification accuracy while greatly reducing computation.

Problem
Selecting useful features from thousands of candidates is computationally expensive, and hctsa requires a Matlab license, limiting feature-based time-series analysis.
Method
The authors score individual features across 93 classification tasks and distill high-performing, minimally redundant subsets from the hctsa library.
Results
77.2% vs 71.7% mean class-balanced accuracy, reducing 4791 hctsa features to 22 catch22 characteristics.
Takeaways & Limitations
catch22 makes interpretable feature-based time-series analysis more computationally efficient and accessible across applications.
Takeaways & Limitations
The selected 22 features are tailored to typically short, phase-aligned UCR/UEA datasets and may differ for other application domains.

Abstract

from arXiv · show

Capturing the dynamical properties of time series concisely as interpretable feature vectors can enable efficient clustering and classification for time-series applications across science and industry. Selecting an appropriate feature-based representation of time series for a given application can be achieved through systematic comparison across a comprehensive time-series feature library, such as those in the hctsa toolbox. However, this approach is computationally expensive and involves evaluating many similar features, limiting the widespread adoption of feature-based representations of time series for real-world applications. In this work, we introduce a method to infer small sets of time-series features that (i) exhibit strong classification performance across a given collection of time-series problems, and (ii) are minimally redundant. Applying our method to a set of 93 time-series classification datasets (containing over 147000 time series) and using a filtered version of the hctsa feature library (4791 features), we introduce a generically useful set of 22 CAnonical Time-series CHaracteristics, catch22. This dimensionality reduction, from 4791 to 22, is associated with an approximately 1000-fold reduction in computation time and near linear scaling with time-series length, despite an average reduction in classification accuracy of just 7%. catch22 captures a diverse and interpretable signature of time series in terms of their properties, including linear and non-linear autocorrelation, successive differences, value distributions and outliers, and fluctuation scaling properties. We provide an efficient implementation of catch22, accessible from many programming environments, that facilitates feature-based time-series analysis for scientific, industrial, financial and medical applications using a common language of interpretable time-series properties.

1 Introduction

Time-series analysis supports applications across industry, finance, and medicine, but manually or highly comparatively selecting features from thousands of candidates is costly and limits adoption. The study develops a data-driven pipeline to identify reduced subsets of useful, complementary features from large feature libraries.

Motivation: Time-series analysis enables applications such as detecting production irregularities, identifying fraudulent transactions, and diagnosing pathological heartbeat patterns.The paper motivates time series as ordered measurements that reveal real-world system dynamics.
Motivation: Similarity-based clustering and classification can scale poorly, often quadratically with both the number of time series and their length.The stated scaling issue arises from computing pairwise distances between time series.
Problem: Manual feature selection is subjective, while systematic comparison across thousands of features creates a computational burden for large real-world datasets.The highly-comparative approach addresses uncertainty in manual selection but can become infeasible when selecting from thousands of candidates.
Contribution: The study develops a data-driven pipeline that distills reduced subsets of useful and complementary features from thousands of candidates.The approach scores each feature independently across a calibration set of 93 time-series classification problems, using an initial filtered pool of 4791 hctsa features.

2 Methods … 2.3 Performance-based selection

The methods evaluate hctsa-derived features across a diverse time-series classification calibration set, then use univariate performance and redundancy minimization to select a compact, interpretable subset. Features are filtered for general applicability, rescaled per dataset, and analyzed using an extendable Python-based pipeline.

2.1 Data: 147 198 time series across UEA/UCR classification datasets form the diverse calibration set used to measure feature usefulness.Dataset sizes range from 28 to 33 274 time series.
2.2 Time-series features: The hctsa toolbox supplies a comprehensive initial library of over 7500 features spanning statistics, correlations, autoregressive models, GARCH, wavelets, and network-based methods.Features were calculated in Matlab 2017a using hctsa v0.97.
2.2 Time-series features: Each feature was linearly rescaled to the unit interval separately for each dataset before selection.This preprocessing was applied to the feature values after calculation in hctsa v0.97.
2.2 Time-series features: 7658 hctsa features were initially filtered by removing 766 features sensitive to time-series means and variances, leaving 6892 features.The filtering reflects that most UCR/UEA series are z-score normalized, with four noted unnormalized datasets.
2.3 Performance-based selection: The selection pipeline evaluates each feature’s univariate classification performance on every task, combines scores across tasks, and seeks individually discriminatory, complementary features.Unlike typical selection methods focused on feature combinations, this pipeline targets interpretable features with individual discriminatory power.
2.3 Performance-based selection: The pipeline comprises performance filtering followed by redundancy minimization to produce a reduced set of generically useful features.The method can be extended to different classification-task sets or initial feature pools.
2.3 Performance-based selection: All analyses used Python 2.7 with scikit-learn, and reproducibility code was made accessible on GitHub.The cited repository is github.com/chlubba/op_importance.

2.4 Quantifying feature performance

Feature performance was quantified across 93 classification tasks using decision-tree classification with task-specific stratified cross-validation. Each of 4,791 features received a combined score based on mean class-balanced accuracy normalized relative to the other features on each task.

Performance scoring: 93 classification tasks were evaluated using decision trees with stratified cross-validation, with the fold count selected separately for each task.The pipeline scored each feature by its ability to distinguish labeled classes across the tasks.
Performance scoring: 4,791 features were scored on each task using mean class-balanced classification accuracy across folds.For feature i and task j, this accuracy was denoted ai,j.
Performance scoring: Task-normalized accuracy was computed by dividing each feature’s raw accuracy by the mean accuracy across all features on that task.This quantified performance relative to the other features in the library.
Performance scoring: The combined feature-accuracy score was calculated as the mean of normalized accuracies across the 93 tasks.This aggregated task-relative performance into a single score for each feature.

2.5 Statistical prefiltering

The study statistically prefiltered hctsa features by comparing observed classification accuracy with permutation-based null distributions. It combined task-level p-values across datasets and controlled multiple testing across features.

Permutation-based null model: 1000 shuffled-label classification repeats generated null accuracy distributions for each feature-task combination.The distributions were mostly unimodal and approximately normally distributed, motivating Gaussian fitting.
P-value estimation: P-values measured the probability that shuffled-label accuracy exceeded the observed accuracy.A Gaussian probability distribution was fitted to each null accuracy distribution before estimating this probability.
Multiple-testing correction: Fisher’s method combined p-values across classification tasks, while Holm-Bonferroni correction controlled multiple testing across features at significance level 0.05.This procedure produced a single cross-task significance assessment for each feature.

2.6 Selecting a canonical set of features · 2.7 Overall classification performance

The study selected a compact canonical feature subset by ranking significantly above-chance features and reducing redundancy based on classification-task performance patterns. It evaluated feature sets using class-balanced and unbalanced accuracy measures, including comparisons with alternative time-series classifiers.

2.6 Selecting a canonical set of features: 2.6 Selecting a canonical set of features: Significantly above-chance features were ranked by combined normalized accuracy an,c to identify high-performing candidates.The comparison considered subset sizes in the range 100 ≤ β ≤1000.
2.6 Selecting a canonical set of features: 2.6 Selecting a canonical set of features: Hierarchical clustering was used to reduce redundancy among top-performing features.Redundancy was defined by patterns of performance across classification tasks, using Pearson correlation distance.
2.7 Overall classification performance: 2.7 Overall classification performance: Different hctsa feature sets were compared using mean class-balanced accuracy.The evaluation also compared feature-based classification with alternative time-series classification methods.
2.7 Overall classification performance: 2.7 Overall classification performance: Mean class-balanced accuracy was computed across M tasks and NCV cross-validation folds.This measure was used to evaluate classification performance across feature sets.
2.7 Overall classification performance: 2.7 Overall classification performance: Feature sets were compared using two different accuracy measures.The measures supported both within-hctsa feature-set comparisons and comparisons with existing methods.
2.7 Overall classification performance: 2.7 Overall classification performance: Comparisons with existing methods used mean unbalanced classification accuracy across tasks.This followed Eq. (1) on the given train-test split to match the metric supplied with the UEA/UCR repository.

2.8 Execution times and scaling · 2.9 Selecting the two most informative features from a small subset

Section 2.8 motivates catch22’s computational efficiency and evaluates feature-set execution time and scaling using a diverse benchmark of 40 time series. Section 2.9 describes sequential forward selection for identifying two complementary features that best distinguish classes in a dataset.

2.8 Execution times and scaling: Quick computation is presented as a key merit of a small canonical feature set for time-series characterization.The section compares execution times across different feature sets.
2.8 Execution times and scaling: Execution-time comparisons use a benchmark of 40 time series drawn from diverse scientific and applied sources.Sources include simulated dynamical systems, financial data, medical recordings, meteorology, astrophysics, and bird sounds.
2.8 Execution times and scaling: The scaling analysis generates multiple versions of each benchmark series with different time-series lengths.This procedure estimates how feature computation changes with series length.
2.9 Selecting the two most informative features from a small subset: The visual analysis objective is to identify two features that together are most informative for distinguishing time-series classes.The selected pair is intended to support rapid inspection of a dataset in feature space.
2.9 Selecting the two most informative features from a small subset: Sequential forward selection first chooses the single feature with the best mean class-balanced accuracy across cross-validation folds.It then evaluates remaining features for addition to the selected feature.
2.9 Selecting the two most informative features from a small subset: The procedure iterates over remaining features to select the feature that performs best when combined with the first feature.This constructs a two-feature representation for class distinction.

3 Results

catch22 reduces 4791 candidate features to 22 canonical characteristics while retaining approximately 90% of the full pool’s classification performance. Across datasets, it achieves about 72% mean class-balanced accuracy versus about 77% for all features, with approximately 1000-fold faster computation and near-linear scaling.

Feature selection: 22 features were selected from 4791 candidates to form catch22, approximating 90% of the initial pool’s classification performance.The selected characteristics were manually simplified where possible to improve interpretability.
Classification performance: ∼72% was catch22’s mean class-balanced accuracy across datasets, compared with ∼77% for all 4791 features.This represents a small average reduction in accuracy after the dimensionality reduction.
Classification performance: r = 0.93 was the Pearson correlation between catch22 and the 16-feature tsfeatures package, whose mean accuracy was 69.4% versus catch22’s 71.7%.The comparison used generic UCR/UEA datasets.
Computational efficiency: Approximately 1000-fold acceleration was achieved relative to the full hctsa feature set in Matlab, with C catch22 taking ∼0.3 s on 10 000 samples.The C implementation was approximately 30 times faster than the Matlab implementation and had complexity O(N 1.16).
Dataset-specific behavior: 100% unbalanced accuracy was achieved by catch22 on ShapeletSim, exceeding the mean of 69.0 ± 18.7% for other classifiers except BOSS and Fast Shapelets.The feature representation succeeds because the classes differ in frequency content despite lacking consistent time-domain shape.
Dataset-specific behavior: 78.9% was catch22’s accuracy on CinCECGtorso, close to the average existing classifier’s 81.3 ± 13.3%.The results show that feature-based and shape-based approaches can have comparable performance on some datasets.

4 Discussion

The discussion presents catch22 as an efficient, accessible reduction of large time-series feature libraries, while emphasizing that canonical feature sets depend on the datasets and application domain. It also highlights the broader value of the methodology for generating tailored feature subsets.

Contributions: The methodology generates small, canonical feature subsets with high performance across task ensembles and complementary performance characteristics, reducing a library of 4791 features across 93 classification tasks.The tasks come from the UCR/UAE repository.
Contributions: An efficient reduced feature set such as catch22 can make feature-based time-series analysis more widely accessible than searching large libraries such as hctsa.The feature-reduction pipeline itself can also generate slightly different subsets for specific applications.
Limitations: The most useful time-series representation varies across datasets, with some problems favoring feature-based classification and others favoring shape-based classification.The selected 22 features are tailored to UCR/UEA datasets, which are typically short and phase-aligned.
Limitations: Different datasets emphasizing subtle dynamical variations in long time-series streams could yield canonical sets containing long-range automutual information and nonlinear-analysis measures rather than catch22’s simpler features.The feature-reduction method can be rerun as newer, more diverse datasets are added to the UCR/UEA repository.
Conclusion: The paper concludes that catch22 provides a concise, accessible feature-based summary for time-series classification and that tailored reduced sets could support diverse applications.The authors frame both ready-to-use representations and domain-specific feature generation as useful outcomes.

Supplement · S1 Insignificant features

The supplement identifies 145 features whose classification performance across tasks was consistent with a random-number generator. Examples include several tc3 features, lastimprovement, and frombefore.maxerrbar.

S1 Insignificant features: 1.000 SD was reported for tc3.normpatponmax.
S1 Insignificant features: 1.000 SD was reported for tc3.kspminfromext.
S1 Insignificant features: 1.000 SD was reported for tc3.ksphereonmax and tc3.ksiqrsfrommode.
S1 Insignificant features: 0.080 SP was reported for frombefore.maxerrbar.
S1 Insignificant features: 145 features exhibited classification performance consistent with a random-number generator.This result is stated in both the supplement text and Table S1.

S3 Performance comparison with tsfeatures

The automatically selected catch22 feature set performs comparably to the standard tsfeatures set on simple time series, as assessed by class-balanced accuracy across folds.

S3 Performance comparison with tsfeatures: catch22 performs as well as the standard tsfeatures feature set on simple time series.The comparison uses datasets contained in the tsfeatures package.
S3 Performance comparison with tsfeatures: Class-balanced accuracy is reported for both tsfeatures and catch22, with error bars showing standard deviation across folds.
S3 Performance comparison with tsfeatures: Datasets with the greatest accuracy differences are highlighted as labeled red circles relative to a gray dashed equality line.