PaperlayerView exact revision on arXiv

Source-linked AI summary

The Canonical Interval Forest (CIF) Classifier for Time Series Classification

Matthew Middlehurst, James Large, Anthony Bagnall

arXiv:2008.09172v1cs.LGeess.SP

TL;DR

Time series forest (TSF) uses three simple interval summaries, while whole-series catch22 classification is not particularly effective and recent methods have surpassed TSF. The paper combines catch22 with TSF in the Canonical Interval Forest (CIF), then replaces TSF with CIF in HIVE-COTE. The authors report that HC-CIF is more accurate on average than the current state of the art and that CIF extends to multivariate classification.

  • Problem

    TSF's three interval summaries leave open whether richer catch22 features can produce a more accurate interval classifier and improve HIVE-COTE.

    Source: §III · p. 3

  • Method

    CIF combines catch22 and TSF features over intervals, and the paper evaluates replacing TSF with CIF in HIVE-COTE.

    Source: §I · p. 1, §III · pp. 3–4

  • Results

    HC-CIF is significantly more accurate on average than the current state of the art, and averages 1.56% higher accuracy than InceptionTime.

    Source: D. CIF as a HIVE-COTE component · p. 7, §VIII · p. 9

  • Takeaways & Limitations

    CIF upgrades HIVE-COTE's interval-based component and is extended to multivariate classification.

    Source: §I · pp. 1–2

Abstract

from arXiv · show

Time series classification (TSC) is home to a number of algorithm groups that utilise different kinds of discriminatory patterns. One of these groups describes classifiers that predict using phase dependant intervals. The time series forest (TSF) classifier is one of the most well known interval methods, and has demonstrated strong performance as well as relative speed in training and predictions. However, recent advances in other approaches have left TSF behind. TSF originally summarises intervals using three simple summary statistics. The `catch22' feature set of 22 time series features was recently proposed to aid time series analysis through a concise set of diverse and informative descriptive characteristics. We propose combining TSF and catch22 to form a new classifier, the Canonical Interval Forest (CIF). We outline additional enhancements to the training procedure, and extend the classifier to include multivariate classification capabilities. We demonstrate a large and significant improvement in accuracy over both TSF and catch22, and show it to be on par with top performers from other algorithmic classes. By upgrading the interval-based component from TSF to CIF, we also demonstrate a significant improvement in the hierarchical vote collective of transformation-based ensembles (HIVE-COTE) that combines different time series representations. HIVE-COTE using CIF is significantly more accurate on the UCR archive than any other classifier we are aware of and represents a new state of the art for TSC.

I. INTRODUCTION

Interval classifiers can focus on phase-dependent regions that carry class information, avoiding confounding parts of a series. The paper introduces CIF by combining catch22 features with TSF and reports stronger results for CIF and HIVE-COTE with CIF.

I. INTRODUCTION

  • Interval classifiers use randomly selected intervals held constant across series to target class-discriminative locations and mitigate confounding regions.In the EthanolLevel example, the high-variation visible spectrum can confound whole-series methods, while an interval into the near infrared contains discriminatory information for ethanol concentration.Source: §I · p. 1, Figure 1 · p. 2
  • TSF is fast, robust to parameter settings, and supports interval-importance analysis, but is the weakest HIVE-COTE component in predictive power on average..Source: §I · p. 2
  • TSF summarizes each interval with mean, standard deviation, and slope, whereas catch22 offers 22 diverse features that are not particularly useful alone as a whole-series transform.The paper also evaluates eight TSC algorithms proposed after the earlier bake-off using the expanded UCR archive.Source: §I · p. 2
  • CIF embeds catch22 features in an adaptation of TSF; it is significantly more accurate than TSF and improves HIVE-COTE when replacing TSF.The authors also report HC-CIF as significantly more accurate than TS-CHIEF and InceptionTime, and extend CIF to multivariate classification.Source: §I · p. 2

II. TIME SERIES CLASSIFICATION

The paper frames time series classification as predicting labels for new series from labeled training data. It surveys approaches that represent different discriminatory patterns, including ensembles that combine representations.

II. TIME SERIES CLASSIFICATION

  • A time series is an ordered vector of continuous values, and TSC predicts labels for new series using models derived from labeled training data.The main results concern univariate equal-length series; multivariate datasets and an unequal-length case study are also considered.Source: §II · p. 2

A. Current State of the Art

  • Proximity forest (PF) ensembles trees built from randomized selections of time-series distance functions and is described as the most accurate distance-based approach.It beats the previous best, the Elastic Ensemble.Source: A. Current State of the Art · p. 2
  • WEASEL and S-BOSS classify from distributions of discretized sliding-window words and are state of the art among dictionary-based methods.Both are significantly more accurate than BOSS.Source: A. Current State of the Art · p. 2
  • Shapelet transform classifier STC searches under a maximum time limit and, with one hour per dataset, finds a transform not significantly worse than full enumeration.The shapelet transform avoids unnecessary full enumeration, which often leads to overfitting; RISE extracts spectral features from intervals.Source: A. Current State of the Art · pp. 2–3
  • Time-domain interval methods are the paper's focus; Time Series Bag of Features and LSF are no more accurate than TSF on average and are considerably slower.These methods use interval-derived features, whereas the broader taxonomy also includes distance, shapelet, dictionary, and frequency representations.Source: A. Current State of the Art · p. 3
  • HIVE-COTE ensembles five representation-specific classifiers, while TS-CHIEF combines distance, dictionary, and spectral features within a tree ensemble.InceptionTime [4] is identified as the strongest deep-learning classifier; many deep-learning methods appear to perform poorly on the UCR archive, which may include datasets too small for effective training.Source: A. Current State of the Art · p. 3

B. Time Series Forest (TSF)

TSF builds a forest from features extracted on randomly sampled intervals, while catch22 provides a compact set of descriptive time-series features selected from a much larger collection.

B. Time Series Forest (TSF)

  • For each tree, TSF samples k intervals, summarizes each with mean, standard deviation, and slope, and builds a decision tree on the concatenated 3k features.New cases are classified by majority vote across the forest.Source: B. Time Series Forest (TSF) · p. 3
  • TSF's computational complexity is O(nlog(n) · m · r), where r is the number of trees.The time series tree has interpretability advantages over the random tree used in the bake-off implementation, with no accuracy difference reported.Source: B. Time Series Forest (TSF) · p. 3

C. Canonical Time Series Characteristics (catch22)

  • catch22 is a set of 22 descriptive features selected as a concise, informative subset of the 7658 features in the hctsa toolbox.Its selection and experimentation were primarily based on classification with the UCR archive.Source: C. Canonical Time Series… · p. 3
  • The catch22 selection process filtered features for calculability and performance, then reduced redundancy by clustering correlated features and choosing one from each of 22 clusters.Features sensitive to mean and variance were first removed because most UCR data had been normalized; selection also considered accuracy, computational efficiency, and interpretability.Source: C. Canonical Time Series… · p. 3
  • catch22 covers concepts including basic statistics, linear correlations, and entropy, with computation complexity O(n · m^1.16).The reported catch22 results used a decision tree, while the default code implementation uses a random forest.Source: C. Canonical Time Series… · p. 3

III. CANONICAL INTERVAL FOREST CLASSIFIER (CIF)

CIF develops an interval classifier by replacing TSF's three summary features with catch22 features, then retaining TSF features and subsampling from the combined feature set. The design also addresses computation costs and extends interval sampling to multivariate series.

III. CANONICAL INTERVAL FOREST CLASSIFIER (CIF)

  • The initial hybrid replaces TSF's mean, standard deviation, and slope with all 22 catch22 features to test whether richer interval features improve classification.The motivation is that whole-series catch22 classifiers perform significantly worse than alternative feature-based approaches, while TSF provides fast interval diversity.Source: §III · p. 3
  • CIF normalizes intervals only for the positive and negative DN OutlierInclude features because those features can take a long time on extreme-valued unnormalized data.The other catch22 features are calculated on unnormalized intervals, where absolute values may carry useful discriminatory information.Source: §III · pp. 3–4
  • CIF retains the three TSF features alongside catch22 and randomly samples from the resulting 25 features for each tree; the default sample size is 8.The approach adds feature-space diversity and improves time efficiency; it uses the time series tree rather than the random tree in the open-source Java implementation.Source: §III · p. 4
  • For multivariate CIF, each interval is drawn from a random dimension as well as random positions and lengths, and the number of intervals per tree is k = d · √m.Here d is the number of dimensions and m is series length.Source: §III · p. 4
  • CIF's stated defaults are 500 trees, k = d · √m intervals per tree, and a = 8 attributes subsampled per tree.The parameters are the tree count r, intervals per tree k, and attributes subsampled per tree a.Source: §III · p. 4
  • The training procedure samples attribute indices and computes catch22 features when an index is at most 22, otherwise using a TSF feature before building a time series tree.The feature choice is applied to each selected interval and case.Source: §III · p. 4

IV. EXPERIMENTAL METHODS

The experiments use UCR univariate datasets and UEA multivariate datasets, with resampling for univariate evaluation and default splits for multivariate evaluation.

IV. EXPERIMENTAL METHODS

  • 112 UCR datasets are evaluated after excluding unequal-length or missing-value data and Fungi; univariate scores average 30 stratified resamples.Fold 0 remains the original train/test split, and dataset fold indices seed resampling and classifier parameters for reproducibility.Source: §IV · p. 4
  • The multivariate experiments use 26 equal-length UEA datasets and their default train/test splits.The paper provides the adapted toolkits and code used to reproduce its experimental results on an accompanying website.Source: §IV · p. 4
  • Classifier comparisons use pairwise Wilcoxon signed-rank tests, and multi-classifier results use critical-difference diagrams with Holm-corrected cliques.Performance is assessed with accuracy, and individual experiments are limited to seven days of computation.Source: §IV · p. 4

V. RESULTS

The results section examines recent TSC algorithms, CIF as a standalone classifier, multivariate performance, HIVE-COTE with CIF, and efficiency.

V. RESULTS

  • The results section frames comparisons of eight recently proposed classifiers on the expanded UCR archive and asks whether CIF improves on TSF and matches strong single-representation classifiers.It also sets out tests of CIF on multivariate data and whether replacing TSF improves HIVE-COTE.Source: §V · p. 4
  • Figure 2 presents accuracy rankings for eight recently proposed TSC algorithms, TSF, and DTWCV.The section also plans to quantify CIF time and space efficiency and propose a more efficient test-accuracy estimation mechanism for HIVE-COTE.Source: §V · pp. 4–5, Figure 2 · p. 5

A. Recent TSC Algorithms

CIF improves on TSF and catch22 in accuracy, while matching leading single-representation classifiers on accuracy and outperforming some on AUROC. The results also place DTWCV behind modern TSC methods.

A. Recent TSC Algorithms

  • 7% more accuracy per problem for S-BOSS and 10.5% for HIVE-COTE, on average, compared with DTWCV.DTWCV is not competitive with modern TSC algorithms; TSF is significantly better than DTWCV and catch22 but worse than the single-representation clique of S-BOSS, WEASEL, PF, and STC.Source: A. Recent TSC Algorithms · p. 5

B. CIF as a standalone classifier

C. CIF on multivariate data

CIF performs significantly better than the compared multivariate classifiers, and replacing TSF with CIF makes HIVE-COTE significantly more accurate than the three state-of-the-art classifiers evaluated. HC-CIF also averages higher accuracy than InceptionTime.

C. CIF on multivariate data

  • CIF and S-BOSS vary substantially across problems, with CIF winning on 59, tying on 1, and losing on 47.The authors attribute this diversity to the representations specializing on different data characteristics, and identify it as a reason HIVE-COTE performs well.Source: Figure 7 · p. 6, D. CIF as a HIVE-COTE component · p. 6
  • CIF performs significantly better than the other multivariate algorithms in the analysis, although the comparison is not exhaustive.DTW remains a good multivariate benchmark, while TSF and cBOSS outperform univariate DTW but do not carry that advantage into the multivariate case.Source: Figure 8 · p. 6, C. CIF on multivariate data · p. 6

D. CIF as a HIVE-COTE component

VI. CIF EFFICIENCY AND USABILITY

CIF is faster than the hybrid while retaining accuracy comparable to single-representation classifiers, and its memory footprint is lower than that of the dictionary-based methods discussed. Reduced and contracted variants offer additional training-time controls.

A. Time and space efficiency

  • CIF is approximately eight times faster than the hybrid and is the fastest displayed single-representation classifier, with no significant accuracy difference among those classifiers.The comparison plots average accuracy difference from the hybrid against average run time across UCR problems.Source: A. Time and space efficiency · p. 7, Figure 12 · p. 7
  • TSF is the fastest but least accurate extreme, while TS-CHIEF is the most accurate but slowest.All experiments used a single thread to make comparisons fair.Source: A. Time and space efficiency · p. 7
  • CIF-Fast reduces trees and intervals per tree for large problems with little accuracy loss, while contracted CIF can add trees incrementally under a fixed training-time limit.The fixed-time option is intended for time-sensitive training requirements such as cluster job limits.Source: A. Time and space efficiency · p. 7
  • TS-CHIEF and S-BOSS use 241% and 231% of CIF’s memory, respectively, while the hybrid uses slightly more memory than CIF.The dictionary methods store whole-dataset representations; the hybrid’s larger feature space accounts for its slightly higher use.Source: A. Time and space efficiency · p. 7

B. Model Visualisation

CIF adapts temporal-importance curves to show which intervals contribute to classification. On EthanolLevel, the most important region matches the region identified by a domain expert, though accuracy, computational cost, and interpretability can conflict.

B. Model Visualisation

  • CIF temporal-importance curves sum information gain from tree splits over the intervals covered by each feature, displaying the top three features by maximum gain and the mean across features.The curves plot time-point importance for classification.Source: B. Model Visualisation · pp. 7–8
  • For EthanolLevel, the most important classification region matches the region identified by a domain expert.The figure marks the expert-specified region with a dotted box; the displayed curves use no feature subsampling.Source: B. Model Visualisation · p. 8, Figure 14 · p. 8
  • Curves remain meaningful with feature subsampling, but randomly restricting features may promote weaker features and may not give the best representation for higher-level analysis.The authors note that maximizing accuracy, minimizing computational cost, and maximizing interpretability can conflict.Source: B. Model Visualisation · p. 8

C. Out-of-bag error estimates

In the asphalt case study, CIF is the best single-representation classifier, while InceptionTime is best overall; CIF also beats HC-CIF on two of the three problems. For HIVE-COTE’s test-accuracy estimates, bagging offers a faster compromise than cross-validation when the final CIF is trained on all data.

C. Out-of-bag error estimates

  • Bagging estimates have lower error than cross-validation’s 2.86% average underestimation and are approximately five times faster, while full-data CIF prediction causes no significant overall HIVE-COTE change.Bagging estimates the weight, but CIF trained on the full data predicts new cases; the actual test accuracy of the bagged classifier is significantly lower.Source: C. Out-of-bag error estimates · p. 8, Table I · p. 8

VII. CASE STUDY

  • The case study uses 100 Hz smartphone motion data converted to acceleration magnitude to classify three road-surface conditions.The datasets concern road regularity, pavement type, and obstacles.Source: §VII · pp. 8–9
  • Because the asphalt series have unequal lengths, zero-padding to the longest series may introduce confounding features; the authors believe interval-based CIF has inherent tolerance to padding.The experiments use the methods described in the paper’s experimental methodology.Source: §VII · p. 9
  • CIF is the best single-representation classifier on the asphalt datasets, InceptionTime is best overall, and CIF beats HC-CIF on two of the three problems.The authors attribute HC-CIF’s reduced accuracy to its RISE component and omit EE because it could not finish within the time constraints.Source: §VII · p. 9
  • The HC-CIF asphalt comparison omits EE, and TS-CHIEF failed to complete two problems within the seven-day limit.The authors also report that many tested classifiers cannot handle unequal-length series directly.Source: §VII · p. 9

VIII. CONCLUSION

CIF combines TSF and catch22 and is significantly more accurate than either; replacing TSF with CIF in HIVE-COTE yields significantly higher accuracy and a new UCR-archive state of the art.

VIII. CONCLUSION

  • HC-CIF is significantly more accurate than HIVE-COTE, TS-CHIEF, and InceptionTime, and represents a new UCR-archive state of the art in classification accuracy.The reported change is replacing TSF with CIF in HIVE-COTE.Source: §VIII · p. 9
  • CIF is significantly more accurate than TSF and catch22, is not significantly worse than leading single-representation classifiers, and is generally faster and less memory-intensive than them.The compared classifiers are WEASEL, S-BOSS, PF, and STC.Source: §VIII · p. 9
  • The authors replace only TSF with CIF in HIVE-COTE, avoiding other possible changes that could add variation and obscure CIF's contribution.They report that this single change produces significantly higher average accuracy than the current state of the art.Source: §VIII · p. 9
Loading 2008.09172v1…