Source-linked AI summary
MINIROCKET: A Very Fast (Almost) Deterministic Transform for Time Series Classification
Angus Dempster, Daniel F. Schmidt, Geoffrey I. Webb
TL;DR
Accurate time series classification methods have remained computationally expensive and difficult to scale. The paper reformulates Rocket as MiniRocket, using a faster, almost deterministic transform based on convolutional kernels and transformed features for linear classification. MiniRocket achieves essentially the same accuracy while being up to 75 times faster, and supports training and testing across all 109 UCR datasets in less than 10 minutes.
Problem
Accurate time series classification methods remain constrained by high computational complexity and limited scalability.
Method
MiniRocket reformulates Rocket with a small, fixed set of convolutional kernels and an almost deterministic transform whose features train a linear classifier.
Results
75 times faster on larger datasets, MiniRocket maintains essentially the same accuracy as Rocket and enables training and testing on all 109 UCR datasets in less than 10 minutes.
Takeaways & Limitations
MiniRocket is suggested as the default variant of Rocket because it offers a favorable accuracy-to-computational-cost trade-off.
Takeaways & Limitations
Fully deterministic transformation requires additional computational cost and is unlikely to be practical for larger datasets.
Abstract
from arXiv · showhide
Until recently, the most accurate methods for time series classification were limited by high computational complexity. ROCKET achieves state-of-the-art accuracy with a fraction of the computational expense of most existing methods by transforming input time series using random convolutional kernels, and using the transformed features to train a linear classifier. We reformulate ROCKET into a new method, MINIROCKET, making it up to 75 times faster on larger datasets, and making it almost deterministic (and optionally, with additional computational expense, fully deterministic), while maintaining essentially the same accuracy. Using this method, it is possible to train and test a classifier on all of 109 datasets from the UCR archive to state-of-the-art accuracy in less than 10 minutes. MINIROCKET is significantly faster than any other method of comparable accuracy (including ROCKET), and significantly more accurate than any other method of even roughly-similar computational expense. As such, we suggest that MINIROCKET should now be considered and used as the default variant of ROCKET.
1 INTRODUCTION
MiniRocket reformulates Rocket into a much faster, almost deterministic transform while maintaining essentially the same accuracy. Across 109 UCR datasets, it combines competitive accuracy with substantially lower computational cost.
- High computational complexity and limited scalability remain persistent problems for accurate time series classification methods.
- 75 times faster on larger datasets is the reported maximum speedup of MiniRocket over Rocket.The reformulation maintains essentially the same accuracy and can optionally be made fully deterministic with additional computational expense.
- MiniRocket reformulates Rocket using a small, fixed set of kernels and an almost entirely deterministic transform while retaining dilation and PPV pooling.It uses transformed features to train a linear classifier, as Rocket does.
- MiniRocket is marginally more accurate than Rocket on average, though slightly less accurate than the most accurate current methods.The comparison uses 30 resamples of 109 UCR benchmark datasets.
- 30 times faster than Rocket on average is MiniRocket’s reported transform-time advantage across the same 109 UCR datasets.On a single CPU core, total compute time was 2 hours 2 minutes for Rocket versus 8 minutes for MiniRocket.
- MiniRocket is significantly faster than methods with comparable accuracy and significantly more accurate than methods with roughly similar computational expense.
2 RELATED WORK
Related work establishes a trade-off between accuracy and computational cost: leading methods are often slow, while scalable methods can be less accurate. MiniRocket retains Rocket’s convolutional-transform design while using fixed kernels and simpler optimisations.
- 2.1 Current State of the Art: State-of-the-art methods such as HIVE-COTE variants, TS-CHIEF, InceptionTime, and Rocket improve accuracy, but computational complexity and scalability remain persistent problems.
- 2.1 Current State of the Art: cBOSS is more scalable than most competing methods but significantly less accurate, whereas several alternatives require days or weeks to train on UCR datasets.
- Rocket transforms time series with random convolutional kernels, applies global max pooling and PPV pooling, and trains a linear classifier on the resulting features.
- Dilation sampled on an exponential scale and PPV are identified as Rocket’s two most important accuracy-related components.
- MiniRocket differs from Rocket by using a small, fixed set of kernels rather than random kernels, while retaining full-precision inputs and convolution outputs.
- MiniRocket’s optimisations reduce convolution operations through fixed kernels and direct use of their convolution outputs rather than learned linear combinations.
- Unlike most convolutional neural-network approaches, MiniRocket produces independent transform features without hidden layers or nonlinear activations before linear classification.
3 METHOD
MiniRocket reformulates ROCKET with a small, fixed kernel set and near-deterministic bias sampling, preserving its core dilation and PPV features while reducing transform cost. Its design uses constrained kernels, selected dilations, and bias values drawn from convolution outputs to support efficient classification.
- 3.1 Removing Randomness: MiniRocket replaces ROCKET’s random kernels with a small, fixed set while retaining dilation and PPV pooling.The fixed kernel set and reduced randomness are intended to accelerate the transform without sacrificing the key ROCKET features.
- 3.1.2 Weights: PPV counts the proportion of convolution outputs exceeding bias values, making the feature equivalent to an empirical cumulative distribution function.Because biases are drawn from convolution outputs, weight scale matches input scale and input normalization is unnecessary.
- 3.1.2 Weights: The zero-sum weight constraint makes convolution outputs invariant to adding or subtracting a constant from the input.The constraint is expressed as β = −2α; MiniRocket uses α = −1 and β = 2.
- 3.1.1 Kernels: 84 length-9 kernels use three weights valued 2 and six weights valued −1, producing a compact kernel set with high classification accuracy.The paper states that other kernel subsets and lengths achieve similar accuracy, so this set is not necessarily unique.
- 3.1.3 Bias: MiniRocket is minimally random because only training-example selection for bias sampling is stochastic; an optional variant samples biases from the entire training set deterministically.The fully deterministic variant has negligible accuracy difference but adds computational cost that is unlikely to be practical for larger datasets.
- 3.1.4 Dilation: MiniRocket limits each kernel’s maximum number of dilations to 32 because longer series create more possible dilations and reduce transform efficiency.The default cap has little effect on accuracy and is intended to remain unchanged.
3.2 Optimising the Transform
MiniRocket accelerates its transform through four optimisations that exploit fixed two-valued kernels and PPV. These optimisations avoid redundant convolutions and multiplications while reusing intermediate results across features and kernels.
- Four key optimisations: Four optimisations accelerate MiniRocket: paired-kernel PPV computation, convolution-output reuse, multiplication avoidance, and near-simultaneous processing of kernels.These are presented as the key mechanisms for reducing transform time.
- Computing PPV for W and −W: PPV and PNV are complementary, so computing PPV for kernel W implicitly provides the equivalent feature for inverted kernel −W.Consequently, MiniRocket computes only one kernel from each matched pair.
- Reusing convolution output: A single convolution output for kernel W and dilation d can be reused to compute multiple PPV features with different bias values.The additional PPV calculations cost substantially less than repeating the convolution.
- Avoiding multiplications: Restricting kernel weights to two values lets MiniRocket factor multiplications out of convolution and perform the remaining operation using addition.For each input series, the weighted products are precomputed once and reused across kernels.
- Avoiding multiplications: The convolution equals the column sums of an aligned matrix whose rows represent weighted, dilation-shifted input series.This matrix view enables precomputed weighted inputs to replace repeated multiplications.
- Computing kernels at once: For MiniRocket’s α = −1 and β = 2 kernels, the final output is formed as C = C_α + C_γ with γ = 3, enabling shared computation across kernels.The method computes a common C_α and adjusts it using selected contributions from C_γ.
3.3 Classifiers
MiniRocket produces transformed features that are used to train a linear classifier, using the same classifier choices as Rocket.
- Classifier pipeline: MiniRocket uses transformed features to train a linear classifier, matching Rocket’s classification pipeline.The available classifiers are ridge regression and, for larger datasets, logistic regression trained with Adam.
- Classifier choice: When training examples exceed approximately 10,000, the authors suggest switching from ridge regression to logistic regression.This recommendation applies when there are more training examples than features.
3.4 Complexity
MiniRocket retains linear scalability in kernels or features, examples, and input time-series length.
- Computational complexity: MiniRocket’s computational complexity is O(k · n · l_input), linear in the number of kernels or features, examples, and input length.Using fewer kernel/dilation combinations and reusing computations does not change this proportional relationship.
3.5 Memory
MiniRocket uses additional temporary memory, and its fully deterministic variant can require substantially more storage on large datasets.
- Temporary storage: MiniRocket temporarily stores up to 13 additional vectors compared with Rocket, equivalent to about 52MB for a time series of length 1 million.These vectors include precomputed inputs and intermediate convolution outputs.
- Deterministic variant: The deterministic variant stores convolution output for a kernel/dilation combination across the training set, equivalent to one additional copy of the entire training set.This storage requirement is impractical for larger datasets, so the method avoids it by default.
4 EXPERIMENTS
Experiments on UCR datasets show that MiniRocket preserves accuracy while substantially reducing training and transform time, including on large datasets and long series. Parameter studies identify settings that retain accuracy while enabling these efficiency gains.
- 4.1 UCR Archive: MiniRocket is marginally more accurate than Rocket and not significantly less accurate than TS-CHIEF or HIVE-COTE/TDE.The comparison uses 30 resamples of 109 UCR datasets, with statistical significance assessed using a corrected Wilcoxon signed-rank test.
- 4.1 UCR Archive: MiniRocket is more accurate than Rocket on 61 datasets and less accurate on 45, with mostly small differences.A Rocket variant using MiniRocket’s bias-sampling method has a 50/6/53 win/draw/loss record against MiniRocket.
- 4.2.1 Training Set Size: 75 times faster on MosquitoSound, 66 times faster on FruitFlies, and 43 times faster on InsectSound.Across these large datasets, MiniRocket’s accuracy is slightly higher on one dataset and slightly lower on two.
- 4.2.2 Time Series Length: Both methods scale linearly with time series length, but MiniRocket is considerably faster for a given length.With only 50 training examples, bias-sampling overhead is a significant part of MiniRocket’s total training time.
- 4.3 Parameter Sensitivity: Kernels of length 9 are most accurate, while lengths 7 and 11 are not significantly less accurate; the 9{3} subset is nearly as accurate as the full length-9 set.Sampling bias from the convolution output is critical to matching Rocket’s accuracy, and PPV alone is at least as accurate as adding global max pooling.
5 CONCLUSION
MiniRocket reformulates ROCKET into a mostly deterministic procedure that is much faster while achieving essentially the same accuracy. The authors characterize it as a significant advance in accuracy relative to computational cost and recommend it as ROCKET’s default variant.
- MiniRocket is up to 75 times faster on larger datasets than ROCKET while achieving essentially the same accuracy.
- MiniRocket is much faster than methods of comparable accuracy and more accurate than methods with roughly similar computational expense.
- The authors suggest that MiniRocket should be considered and used as the default variant of ROCKET.
A LOGISTIC REGRESSION TRAINING
The training procedure uses a single shuffled training order and progressively larger subsets, with fixed hyperparameters shared across datasets.
- The training set is shuffled once, and training uses increasingly large subsets of that shuffled data.
- Training uses a validation set of 2,048 examples and a minibatch size of 256 across datasets.
- The initial learning rate is 10^-4 and is halved when validation loss fails to improve after 50 updates.
B ALGORITHM
The algorithm accepts time series and allocates dilations, features, and biases or quantiles for the MiniRocket transform. Its procedure uses fixed kernel-weight indices and computes dilation schedules from series length and kernel limits.
- Inputs and outputs: The algorithm takes time series together with dilations, features per dilation, and either biases or quantiles as inputs.
- Inputs and outputs: It outputs dilations, features per dilation, and biases after processing the input time series.
- Dilation schedule: Maximum dilation is derived from log2(length(X) − 1)/8 and used to construct a schedule across the allowed number of dilations.
- Feature allocation: The algorithm apportions any remaining features across the initial dilation groups after computing their base allocation.
- Kernel construction: The kernel construction uses fixed indices of β weights, including combinations spanning positions 0 through 8.