PaperlayerView exact revision on arXiv

Source-linked AI summary

CLUES-WEASEL: No additional clues required to choose your time series clustering algorithm

Johann Faouzi

arXiv:2609.07606v1cs.LG

TL;DR

Time series clustering is unsupervised, and existing methods balance clustering performance against runtime. CLUES-WEASEL combines unsupervised WEASEL 2.0 feature extraction, PCA, and k-means; the authors report significantly better clustering than existing algorithms while running much faster. They also report that the architecture works with other feature extractors, with memory use remaining a limitation, especially for large datasets.

  • Problem

    Time series clustering methods often trade clustering performance for faster runtimes, while dictionary-based approaches have not been much used for clustering.

    Source: §2.5.2 · p. 9

  • Method

    CLUES-WEASEL applies unsupervised WEASEL 2.0 feature extraction, PCA dimensionality reduction, and k-means clustering to the reduced features.

    Source: §1 · p. 2

  • Results

    CLUES-WEASEL is reported to be significantly better than existing time series clustering algorithms while being much faster, and its architecture also achieves state-of-the-art performance with RDST and MultiROCKET features.

    Source: §1 · p. 3, §7 · pp. 27–28

  • Takeaways & Limitations

    The reported results support using the CLUES-WEASEL architecture with feature extractors beyond WEASEL 2.0 for time series clustering.

    Source: §7 · p. 28

    CLUES-WEASEL has a non-negligible memory footprint, especially on large datasets, because PCA complexity is quadratic in the number of features.

    Source: §7 · p. 28

Abstract

from arXiv · show

Time series data is very common in many real-world applications and in numerous domains, with increasing interest for automated information extraction using machine learning. One of these subfields is time series clustering, which consists in identifying clusters among a set of time series in an unsupervised fashion. Most time series clustering algorithms suffer from the same balancing act: they trade clustering performance for faster runtimes or vice versa. We present a novel time series clustering algorithm that we call CLUES-WEASEL, which stands for CLustering with the UnsupervisEd Second version of Word ExtrAction for time SEries cLassification. CLUES-WEASEL extracts features using the unsupervised version of the transformation step of WEASEL 2.0, which is a time series classification algorithm, then reduces these features using principal component analysis, and finally performs clustering with the $k$-means algorithm using these reduced extracted features. Through extensive experiments, we prove that CLUES-WEASEL is significantly better than any other existing time series clustering algorithm while being (much) faster than any state-of-the-art one. We also show that the architecture of CLUES-WEASEL can work well with other time series feature extraction algorithms. Our findings highlight the relevance of CLUES-WEASEL for time series clustering.

1 Introduction

Time series clustering groups similar time series without labels, but existing approaches balance clustering performance against runtime. The paper introduces CLUES-WEASEL and reports strong clustering performance alongside faster runtimes.

1 Introduction

  • Time series are ordered sequences of real-valued observations sampled at equal frequency, and their use is growing across fields as sensors and Internet of Things devices become more available.The paper cites applications in econometrics and finance, climate science, and geology.Source: §1 · p. 2
  • Time series clustering groups similar time series without labels and has applications including anomaly detection and indexing.Existing approaches include adapting clustering algorithms to time-series distances or extracting features before standard clustering.Source: §1 · p. 2
  • CLUES-WEASEL extracts features with unsupervised WEASEL 2.0, reduces them with PCA, and applies k-means to the reduced features.The evaluation compares it with state-of-the-art algorithms on the UCR time series classification archive.Source: §1 · p. 2
  • The authors report that CLUES-WEASEL significantly outperforms existing time series clustering algorithms while running much faster than state-of-the-art alternatives.They also report that WEASEL 2.0’s unsupervised transformation yields the best results among the feature-extraction algorithms investigated.Source: §1 · p. 3

2 Related work

The related work groups time-series methods by how they represent data and perform clustering or classification. It notes comparatively limited use of feature-based, interval-based, and dictionary-based methods for clustering, and motivates adapting unsupervised transformations developed for classification.

2 Related work

  • The section surveys selected classification and clustering methods rather than providing a systematic review, and excludes neural-network-based clustering approaches.It directs readers to separate recent reviews for broader coverage.Source: §2 · p. 3

2.1 Distance-based approaches

  • Distance-based clustering adapts initialization, distance, assignment, or averaging choices; methods include DTW barycenter averaging, Shape-DBA, and MSM-based approaches.MSM satisfies the triangle inequality, unlike DTW; KASBA combines MSM with a distance-based initialization and barycenter stochastic gradient descent.Source: §2.1.2 · pp. 4–5

2.2 Feature-based approaches

  • Feature-based methods compute whole-series statistics, but the paper finds these methods have seen little use in time series clustering.FeatTS applies k-medoids to TSFresh features; other identified methods are often data-specific and were not evaluated on the UCR archive.Source: §2.2 · p. 5, §2.2.2 · p. 5

2.3 Interval-based approaches

  • Interval-based classifiers extract statistics from subseries, with TSF, CIF, DrCIF, and QUANT representing several approaches.The authors know of no published time series clustering algorithm based on an interval-based classification algorithm at the time of writing.Source: §2.3 · p. 5, §2.3.1 · p. 6, §2.3.2 · p. 6

2.4 Shapelet-based approaches

  • Shapelet methods compare subsequences with time series; clustering examples include U-Shapelets’ unsupervised search and USLM’s joint optimization of transformation and distances.Classification methods include STC and RDST, whose dilation allows nonconsecutive shapelet entries and whose features include minimum-distance position and occurrence count.Source: §2.4 · p. 6, §2.4.1 · p. 6, §2.4.2 · p. 7

2.5 Dictionary-based approaches

  • Dictionary methods convert subsequences into symbolic words and represent each series by word frequencies; SAX uses time-domain reduction, while SFA uses Fourier coefficients.Classification examples include BOSS and WEASEL; WEASEL uses supervised selection, while WEASEL 2.0 adds dilation mapping and less supervised SFA selection.Source: §2.5 · p. 7, §2.5.1 · pp. 8–9

2.5.2 Time series clustering

  • Dictionary-based methods have seen little use in clustering, and the authors know of no published clustering algorithm based on SFA or an SFA-derived method.Existing SAX clustering approaches include Lloyd’s algorithm, agglomerative hierarchical clustering, and an ensemble of trees.Source: §2.5.2 · pp. 9–10

2.6 Convolution-based approaches

  • Convolution-based classifiers such as ROCKET, MiniROCKET, and MultiROCKET use random kernels and extracted convolution features; Hydra combines dictionary and convolution approaches.Clustering methods include R-Clustering, which applies PCA and k-means to MiniROCKET features, and RandomNet, which combines block-level clusterings.Source: §2.6.1 · p. 10, §2.6.2 · pp. 10–11

3 Methodology

CLUES-WEASEL combines unsupervised WEASEL 2.0 feature extraction, PCA, and k-means. The authors focus hyperparameter optimization on PCA while retaining default transformation settings and a single k-means++ initialization.

3.1 Time series clustering algorithm

  • CLUES-WEASEL extracts WEASEL 2.0 features, reduces their dimensionality with PCA, and clusters the reduced features using k-means.Potential constant features are removed, and features are standardized before PCA.Source: §3.1 · p. 11, §3.1.1 · p. 12, §3.1.2 · p. 13

3.1.1 Feature extraction

  • The feature-extraction step must be unsupervised for clustering, so the authors use WEASEL 2.0’s unsupervised transformation and replace supervised chi-squared selection with random selection when needed.They cap the feature count at 30,000; if more features are extracted, selection is random.Source: §3.1.1 · p. 12

3.1.2 Dimensionality reduction

  • PCA reduces the feature space because unsupervised clustering cannot use supervised feature weighting and high-dimensional distances can become less informative.The authors choose PCA as a linear technique that reports explained variance; they standardize features to zero mean and unit variance beforehand.Source: §3.1.2 · pp. 12–13
  • PCA’s decomposition has complexity proportional to the number of samples times the square of the number of features, so the 30,000-feature cap keeps runtime and memory reasonable on modern hardware.The authors note that incremental PCA could reduce complexity in the number of samples but do not investigate it.Source: §3.1.1 · p. 12, §3.1.2 · p. 13

3.1.3 Clustering on reduced extracted features

  • The authors select k-means because it is standard and assigns unseen observations to the nearest cluster centroid.They also note its prior use with extracted features and time-series-specific metrics.Source: §3.1.3 · p. 13

3.2 Hyperparameter values and optimization

  • The authors focus optimization on PCA, retaining WEASEL 2.0 defaults except for random feature selection and using one k-means++ initialization to reduce complexity.They assume the transformation defaults are effective because WEASEL 2.0 performs strongly in classification; PCA is treated as the most critical of the three tunable steps.Source: §3.2 · p. 13
  • PCA component selection is based on total explained variance rather than a raw component count or a per-component cutoff.The tested thresholds range from 5% to 100%; the minimum number of components is raised from one to two because clustering in one dimension might be too difficult.Source: §3.2 · pp. 13–14

4 Experimental setup

The experiments use 112 UCR univariate data sets and compare CLUES-WEASEL with selected time-series clustering algorithms. Hyperparameters are selected on a development subset, with evaluation on separate data sets using clustering metrics and statistical comparisons.

4.1 Data sets and algorithms

  • 112 UCR univariate data sets were included after excluding variable-length series and series with missing values; each series was standardized independently, and results averaged 10 random-seed runs.The default training and test splits were used unless stated otherwise.Source: §4.1 · p. 14
  • Six clustering algorithms were selected for the main comparisons, with less extensive comparisons to RandomNet and deep-learning methods because their experimental setups differed.The main set includes KASBA, MSM, PAM-MSM, DBA, Shape-DBA, and R-Clustering.Source: §4.1 · pp. 14–15

4.2 Metrics, hyperparameter optimization and comparisons

  • Four clustering metrics were used when available, and architecture selection used mean ranks across 41 development data sets before evaluation on the remaining 71.The metrics were clustering accuracy, adjusted Rand index, adjusted mutual information, and normalized mutual information.Source: §4.2 · p. 15
  • Algorithms were compared using modified critical-difference diagrams, multiple-comparison matrices, and pairwise scatter plots; exploratory feature visualizations used UMAP.The diagrams use pairwise Wilcoxon signed-rank tests with Holm correction at significance level 0.05.Source: §4.2 · p. 15

4.3 Experiments, implementation and reproducibility

  • The implementation used aeon for WEASEL 2.0 transformations, NumPy for PCA explained-variance calculations, and scikit-learn for feature processing, PCA, and k-means; source code and reproduction instructions are public.The experiments ran on a MacBook Air (M1, 2020) with 16 GB RAM using Python 3.13.11.Source: §4.3 · pp. 15–16

5 Results

CLUES-WEASEL ranked ahead of existing clustering algorithms in the reported comparisons, and the experiments also assessed runtime, alternative transformations, and feature-count settings.

5.1 Hyperparameter optimization

  • 5.280 mean rank and 6 wins made the 20% explained-variance ratio the selected setting for the final architecture on 41 development sets.Source: Table 1 · p. 17, §5.1 · p. 17

5.2 Comparisons to other time series clustering algorithms with the same setup

  • With and without cross-validation, CLUES-WEASEL had significantly lower mean ranks than all competitors across four metrics; with cross-validation, it also led three direct comparisons in wins and median scores.Source: §5.2.1 · pp. 17–18, Figure 4 · p. 18, §5.2.2 · p. 18

5.3.1 RandomNet

  • CLUES-WEASEL significantly outperformed RandomNet on adjusted Rand index across both the 61 shared evaluation sets and all 112 sets.Source: §5.3.1 · p. 20, Figure 7 · p. 21

5.3.2 Deep learning

  • Across four configurations, CLUES-WEASEL significantly outperformed every selected trained neural network.The comparison used neural-network results with a code issue: k-means was trained on test sets rather than using training-set centroids, which might overestimate performance.Source: §5.3.2 · pp. 21–22, Figure 8 · p. 22

5.4 Runtimes

  • 2.6 times faster than KASBA, CLUES-WEASEL's slowest run took 0.68% more time than its fastest, versus 11.94% for KASBA.Source: §5.4 · p. 22, Table 2 · p. 23

5.5.1 Use of other unsupervised transformations

5.5.2 Impact of the maximum feature count

  • The 5k-feature, 30% version had the lowest mean ranks and significantly beat every tested configuration except the 30k-feature, 20% version.Source: §5.5.2 · p. 26

6 Exploratory analyses

The exploratory analyses visualize reduced features in two dimensions and report class-related clustering patterns for Coffee and Fish. They also examine feature-count and variance settings across development data sets.

6 Exploratory analyses

  • Table 4 reports clustering accuracy across variance ratios and maximum feature counts on 41 development data sets, highlighting each transformation’s lowest mean rank.Source: Table 4 · p. 27
  • UMAP projects the reduced test-set features into two dimensions for visualization, with training on the default training sets.An additional reduction step may be needed when the feature space has more than two dimensions.Source: §6 · pp. 26–27
  • On Coffee, same-class samples tend to be closer, but the samples do not form two distinct groups for any algorithm.The set has two classes, with 28 training and 28 test samples.Source: §6 · p. 27
  • On Fish, all algorithms except TSFresh, and QUANT to a lesser extent, form groups correlated with the true classes.The set has seven classes, with 175 training and 175 test samples.Source: §6 · p. 27
  • Additional visualizations for other data sets appear in Figures B9, B10, and B11.Source: §6 · p. 27

7 Conclusion

CLUES-WEASEL combines unsupervised WEASEL 2.0 feature extraction, PCA, and k-means, and the paper reports strong clustering performance with faster runtimes. The conclusion also notes memory costs for large data sets and the need for benchmarking beyond the UCR archive.

7 Conclusion

  • On the UCR archive, CLUES-WEASEL is reported significantly better than existing time series clustering algorithms and much faster than state-of-the-art algorithms.The conclusion describes it as state-of-the-art in clustering performance, with relatively high memory footprint for large data sets.Source: §7 · pp. 27–28
  • The architecture also achieves state-of-the-art clustering performance with RDST and MultiROCKET feature extraction.Source: §7 · p. 28
  • PCA creates a non-negligible memory footprint on large data sets; the authors report similar clustering performance with at most 5,000 rather than 30,000 extracted features.The study selects PCA components by total variance explained ratio and notes that the current implementation performs the full PCA before selecting the number of components.Source: §7 · p. 28
  • The authors identify benchmarking on data sets beyond the UCR archive as a further need, while noting this also applies to other time series clustering algorithms.Source: §7 · p. 28
  • Figure 12 varies maximum feature counts and total variance explained ratios, while Figure 13 presents pairwise accuracy comparisons across those configurations on 71 data sets.Figure 13 reports mean differences, wins/ties/losses, and unadjusted p-values for one-sided Wilcoxon sign rank tests.Source: Figure 12 · p. 28, Figure 13 · p. 29

8 Declarations

The authors make the study materials available for reuse and report no funding or relevant conflicts of interest. The supplied figures show UMAP visualizations of reduced features for Coffee and Fish.

8 Declarations

  • The data sets, source code, and results are publicly available, with reproduction instructions and a BSD 3-Clause License for the repository.Source: §8 · p. 29
  • The authors report no funding, relevant financial or non-financial interests, or conflicts of interest.Source: §8 · p. 29
  • Figures 14 and 15 visualize reduced test-set features in two-dimensional UMAP space, with colors marking true classes for Coffee’s two classes and Fish’s seven classes.Source: Figure 14 · p. 30, Figure 15 · p. 31

A.1.1 With cross-validation

The appendix compares CLUES-WEASEL with clustering algorithms under different evaluation protocols. The captions specify how many data sets contribute to the mean ranks and identify omissions caused by runtime.

A.1.1 With cross-validation

  • The cross-validation comparison with five state-of-the-art clustering algorithms reports mean ranks over 101 of 112 data sets; some results are missing because of runtime.All algorithms use default training and test sets.Source: Figure A1 · p. 32

A.1.2 Without cross-validation

  • Without cross-validation, the comparison with five state-of-the-art clustering algorithms reports mean ranks over 93 of 112 data sets, with some results missing because of runtime.Algorithms are trained and evaluated on merged training and test sets.Source: Figure A2 · p. 33

A.2.1 Deep learning

  • The deep-learning comparison uses selected top-ranked algorithms and computes mean ranks over all available data sets.The caption describes selection as the union of the top three rankings by mean scores and mean ranks.Source: Figure A3 · p. 34

A.3.1 Use of other unsupervised transformations

The appendix compares WEASEL 2.0 and other unsupervised transformations, then compares their clustering architectures with state-of-the-art clustering algorithms. The figures specify dataset coverage and statistical comparison procedures but provide no result values in these passages.

A.3.1 Use of other unsupervised transformations

  • WEASEL 2.0 is evaluated against five state-of-the-art time-series classification transformation steps using mean ranks across 112 datasets.The algorithms use default training and test sets.Source: Figure A4 · p. 35
  • The six unsupervised transformations are compared on clustering accuracy across 112 datasets using pairwise mean differences, win/tie/loss counts, and unadjusted one-sided Wilcoxon tests.Bold values mark differences significant at the 0.05 confidence level.Source: Figure A5 · p. 35
  • Six clustering algorithms using different transformations are compared with three state-of-the-art clustering algorithms, with mean ranks over 101 datasets because results are missing for 11 datasets due to runtime.All use default training and test sets.Source: Figure A6 · p. 36

A.3.2 Impact of the maximum feature count

The appendix evaluates CLUES-WEASEL configurations with different maximum feature counts and variance-explained ratios, and visualizes reduced features for three datasets using UMAP.

A.3.2 Impact of the maximum feature count

  • CLUES-WEASEL configurations varying maximum feature count and total variance explained ratio are compared by mean rank and clustering accuracy across all 112 datasets.The comparison matrix reports pairwise mean differences, win/tie/loss counts, and unadjusted one-sided Wilcoxon p-values; bold values indicate significance at 0.05.Source: Figure A7 · p. 37, Figure A8 · p. 37

Appendix B Exploratory analyses

Loading 2609.07606v1…