PaperlayerView exact revision on arXiv

Source-linked AI summary

RTSM: Adapting Recursive Models to Multivariate Time-Series Classification

Georg Tenzing, Anna Tegon, Luca Benini, Thorir Mar Ingolfsson

arXiv:2610.23353v1cs.LG

TL;DR

Multivariate time-series classification requires extracting temporal patterns and combining evidence across a sequence, while recursive approaches must choose a temporal representation and latent-state organization. RTSM combines a learned temporal tokenizer, single-state weight-tied refinement, and pooled readout. Across the 30 UEA datasets, it records the highest observed mean accuracy among the evaluated deep-learning models.

  • Problem

    Multivariate time-series classification must extract temporal patterns and combine evidence across diverse sequences, while adapting recursive refinement raises choices about temporal representation and latent-state organization.

    Source: §1 · p. 1

  • Method

    RTSM uses a learned convolutional temporal tokenizer, one weight-tied latent state refined repeatedly, and a pooled sequence-level classification readout.

    Source: §1 · pp. 1–2, §3 · p. 4

  • Results

    72.81% mean accuracy across all 30 UEA datasets was the highest observed among 17 evaluated deep-learning models, 1.60 percentage points above MambaSL.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Eight-dataset ablations favor the proposed tokenizer and single-state formulation, while the refinement module reuses parameters across computational depth.

    Source: §5 · p. 9

Abstract

from arXiv · show

Accurate multivariate time-series classification requires extracting temporal features and integrating information across a sequence. Weight-tied refinement offers a way to repeatedly process these features without allocating new parameters at each stage. Applying this principle to time series requires choosing both a temporal representation and a latent-state organization suited to classification. We introduce the Recursive Time-Series Model (RTSM), which combines a learned convolutional temporal tokenizer, single-state weight-tied refinement, and a pooled classification readout. Across all 30 UEA datasets, RTSM achieves 72.81% mean accuracy under validation-based configuration and checkpoint selection, the highest observed mean among the 17 evaluated deep-learning models and 1.60 percentage points above MambaSL. Ablations on eight datasets favor the proposed tokenizer over a simple convolutional embedding and the single-state model over the tested two-state variants. The complete classifier reaches 86.10% mean accuracy versus 84.79% for a separately trained tokenizer-only classifier. These findings identify temporal encoding and state organization as key design choices for adapting recursive architectures to time series. The resulting classifier combines competitive accuracy with a shared refinement module whose parameter count is independent of computational depth.

1 INTRODUCTION

The paper adapts weight-tied recursive refinement to multivariate time-series classification, where both temporal representation and latent-state organization are design questions. RTSM pairs a learned temporal tokenizer with single-state refinement and a pooled readout, and achieves the highest observed mean accuracy among the evaluated deep-learning models.

1 INTRODUCTION

  • The design targets classification across UEA datasets that differ in sequence length, dimensionality, and training-set size, using parameters shared across computational depth.The adaptation examines temporal encoding and latent-state organization together.Source: §1 · p. 1
  • RTSM combines a convolutional temporal tokenizer, one evolving latent state with shared refinement, and a pooled classification readout.Temporal order enters through the tokenizer; the refinement module uses no positional encoding.Source: §1 · pp. 1–2, Figure 1 · p. 2
  • 72.81% mean accuracy across all 30 UEA datasets is the highest observed among 17 evaluated deep-learning models and 1.60 percentage points above MambaSL.Configuration and checkpoint selection are validation-based.Source: §1 · p. 2
  • Across eight datasets, ablations favor the proposed tokenizer over simple convolutional embedding and single-state refinement over tested two-state variants.The complete classifier improves on a separately trained tokenizer-only classifier by 1.31 percentage points.Source: §1 · p. 2

2 BACKGROUND AND RELATED WORK

The background frames time-series classification as a choice of how to extract temporal features and aggregate them into a sequence-level prediction. It then describes recursive architectures and inference-time computation as context for RTSM’s adaptation.

2.1 MULTIVARIATE TIME-SERIES CLASSIFICATION

  • Time-series classifiers use different feature extractors and sequence-level readouts, including fixed-filter methods and learned convolutional, state-space, and attention-based approaches.ROCKET and MultiRocket use fixed convolutional filters and pooled summaries; neural approaches learn representations with the prediction objective.Source: §2.1 · pp. 2–3

2.2 RECURSIVE ARCHITECTURES

  • Recursive architectures repeatedly apply shared update modules to latent states, with encoders and readouts mapping inputs to predictions.HRM uses separate networks for its auxiliary- and main-state updates, whereas TRM shares one network; the single-state formulation omits the auxiliary state.Source: §2.2 · p. 3

2.2.1 INFERENCE-TIME COMPUTATION

  • Recursive models combine gradient schedules, intermediate supervision, and halting; inference-time computation can also vary by depth or stochastic breadth.HRM uses Q-learning for halting, while TRM trains a scalar halt score to estimate correctness; EqR and PTRM explore inference-time depth and breadth in different ways.Source: §2.2 · p. 3, §2.2.1 · p. 3

3 RECURSIVE TIME-SERIES MODEL

RTSM converts multivariate sequences into temporally informed tokens, repeatedly refines one token-aligned latent state with shared attention, and pools the result for sequence-level classification. Its training uses intermediate supervision and learned halting, while benchmark inference uses a fixed refinement budget.

3 RECURSIVE TIME-SERIES MODEL

  • RTSM adapts recursive computation to sequence classification with a convolutional tokenizer, single latent state, and learned pooled readout.It retains intermediate supervision and learned halting during training.Source: §3 · p. 4, Table 1 · p. 4

3.1 TIME-SERIES TOKENIZER

  • The tokenizer combines channels at each time point, then uses gated residual and dilated convolutions with dense connections to extract temporal features before forming tokens.Patch length and stride set token resolution and are selected per dataset under the evaluation protocols.Source: §3.1 · p. 4

3.2 SINGLE-STATE REFINEMENT

  • A single state holds one vector per token; each shared refinement cycle adds the input tokens and applies the same attention-based module.The reference step uses three cycles, and attention mixes information across token states while the input remains available.Source: §3.2 · p. 5

3.3 SEQUENCE-LEVEL READOUT

  • Learned weighted pooling produces sequence-level classification and halt outputs, with separate pooling parameters for the two heads.The halt score estimates whether the current prediction is correct and controls training duration; benchmark inference uses a fixed budget.Source: §3.3 · p. 5

3.4 TRAINING AND INFERENCE

  • Training supervises the class prediction and estimated correctness at each step, detaches carried states between steps, and may halt adaptively up to 16 steps.The reference configuration tracks gradients through the final cycle; evaluation instead uses 16 fixed steps, or 48 applications of the shared module.Source: §3.4 · pp. 5–6
  • Increasing evaluation depth reuses the same parameters, while computation also depends on token count and the truncated gradient schedule.At benchmark depth 16, the reference configuration performs 96 attention–MLP block applications; preceding cycles still incur forward computation.Source: §3.4 · p. 6

4 EXPERIMENTS

Across the UEA archive, RTSM leads the evaluated deep-learning models under validation-based selection; ablations favor its temporal tokenizer and single-state design, while inference-budget tests show only modest accuracy changes with added computation.

4.1 CLASSIFICATION ON THE UEA ARCHIVE

  • The primary evaluation trains separate models on an 80/20 stratified split of each official training partition and selects configurations and checkpoints by validation accuracy.The official test set is reserved for final evaluation; the benchmark compares complete model-and-training configurations.Source: §4.1 · p. 6
  • 72.81% mean accuracy across 30 UEA datasets is the highest observed among 17 evaluated models, 1.60 points above MambaSL.Each dataset receives equal weight under validation-based configuration and checkpoint selection.Source: §4.1 · p. 6, Figure 2 · p. 7

4.2 ARCHITECTURAL AND TRAINING ABLATIONS

  • Across eight datasets, the proposed tokenizer improves mean accuracy wherever both tokenizers are evaluated, and the single-state model reaches 86.10% versus 84.98% and 84.29% for tested two-state variants.The variants retain the TRM training recipe, and the comparisons involve configurations with different parameter counts and computational costs.Source: §4.2 · p. 7, Table 2 · p. 7
  • 86.10 ± 0.31% for RTSM exceeds 84.79 ± 0.41% for the tokenizer-only classifier by 1.31 points across the eight-dataset subset.RTSM has higher mean accuracy on six datasets, ties on Heartbeat, and is lower on NATOPS; the overall difference is positive for all three seeds.Source: §4.2 · p. 8, Figure 3 · p. 8
  • 86.14% with mean pooling and 86.10% with learned weighted pooling yield similar observed mean accuracies.This readout comparison is reported in Appendix B.3.Source: §4.2 · p. 8
  • Across tested depths H = 1–5, accuracy ranges from 85.60% to 86.10%; at H = 3, adaptive supervision reaches 86.10%, versus 85.98% for one step and 85.05% when all 16 steps are forced.State-reset variants reach 86.00% at H = 1 and 86.09% at H = 3, close to the 86.10% reference mean; increasing depth does not produce a monotonic accuracy gain.Source: §4.2 · p. 8, Table 3 · p. 8

4.3 PREDICTION BEHAVIOR ACROSS INFERENCE BUDGETS

  • 72.42% at step 1 rises to 72.61% at step 16 and 72.84% at step 64, while mean RMS state-update magnitude falls from 1.4172 to 0.0067.The measurements characterize state stabilization but do not establish convergence to a fixed point.Source: §4.3 · p. 9
  • At D = 64, B = 512, tested stochastic selectors yield 72.66%, 72.76%, and 72.74%, below the deterministic trajectory’s 72.84%; RTSM reuses a 6.82-million-parameter module 48 times.The stochastic selectors are halting score, majority vote, and minimum residual; untied applications would require 327.16 million refinement parameters, excluding tokenizer, readout, and activations.Source: §4.3 · p. 9

5 CONCLUSION

RTSM adapts recursive computation to multivariate time-series classification with a learned temporal tokenizer, single-state weight-tied refinement, and pooled readout. It achieves the highest observed mean accuracy among evaluated deep-learning models on the 30-dataset UEA archive, while ablations favor the proposed tokenizer and single-state formulation.

5 CONCLUSION

  • RTSM combines a learned temporal tokenizer, single-state weight-tied refinement, and pooled sequence-level readout for multivariate classification.The refinement module reuses its parameters across computational depth.Source: §5 · p. 9
  • Across 30 UEA datasets, RTSM achieves 72.81% average accuracy, the highest observed mean among evaluated deep-learning models.Eight-dataset ablations favor the proposed tokenizer and single-state formulation; the complete model exceeds a separately trained tokenizer-only classifier by 1.31 points on that subset.Source: §5 · p. 9
  • Depth and state-reset ablations identify several refinement schedules with similar mean accuracy, alongside parameter reuse across computational depth.The conclusion identifies temporal encoding and state organization as design choices for adapting recursive architectures to time series.Source: §5 · p. 9

A.1 PROTOCOL AND IMPLEMENTATION SPECIFICATIONS

The appendix distinguishes the MambaSL and validation protocols by their data use and model-selection procedures. It also documents training implementation, preprocessing, and basic settings.

A.1 PROTOCOL AND IMPLEMENTATION SPECIFICATIONS

  • The appendix documents the experimental environment and preprocessing, and RTSM uses a separate pipeline with persistent batch slots and per-sample trajectory accounting.The basic training settings are summarized for both protocols.Source: §A.1 · p. 14, Table 4 · p. 15
  • The slot-buffer scheduler tracks each sample’s completed trajectories and refills slots so every sample completes exactly the target number of epochs.Samples that have not halted retain their slot and latent state; newly assigned samples receive the default latent state.Source: §A.1 · p. 14
  • The MambaSL protocol trains on the full official training partition and uses the official test partition for model selection and early stopping.The validation protocol instead uses an 80/20 stratified train-validation split, selects configurations and checkpoints on validation accuracy, and reserves the official test set for final evaluation without early stopping.Source: §A.1 · p. 14
  • RTSM’s historical normalization fits statistics either before or after the 80/20 split, while the shared baseline loader standardizes each partition independently.Neither RTSM normalization implementation fits its normalizer on the official test set.Source: §A.1 · p. 14

A.2 SHARED BENCHMARK SPECIFICATION

The shared benchmark specification defines the UEA datasets, comparison methods, and reporting metrics, then documents the hyperparameter searches used across protocols.

A.2 SHARED BENCHMARK SPECIFICATION

  • Reported measures include mean accuracy, mean per-dataset rank, top-N counts, pairwise win/draw and loss/draw counts, and one-sided Wilcoxon signed-rank tests.Ranks use accuracies rounded to two decimal places, with ties receiving the minimum occupied rank.Source: §A.2 · p. 15
  • The benchmark covers 30 multivariate UEA datasets and compares RTSM with deep-learning model families and, under the MambaSL protocol, five non-deep-learning methods.The 10-dataset subset is commonly used in recent TSC benchmarking.Source: §A.2 · p. 15, Table 5 · p. 16

A.3 MODEL HYPERPARAMETERS FOR GRID SEARCH

  • Baseline search spaces are retained across protocols and usually contain more than 200 candidate configurations per model-dataset pair, while DTWD has no learnable hyperparameters.RTSM is listed with 240 configurations; the appendix also records model-specific search settings.Source: §A.3 · pp. 17, 20

B.1 FULL RESULTS ON THE UEA BENCHMARK UNDER MAMBASL PROTOCOL

This section presents the complete per-dataset accuracy results under the MambaSL protocol and identifies the scope of its accompanying aggregate comparison.

B.1 FULL RESULTS ON THE UEA BENCHMARK UNDER MAMBASL PROTOCOL

  • Figure 4 compares RTSM, 16 other deep-learning models, and five non-deep-learning baselines using equal-weight average accuracy across the 30-dataset archive and 10-dataset subset.The MambaSL protocol uses official test-set performance for configuration and checkpoint selection.Source: Figure 4 · p. 21
  • Table 6 reports classification accuracy for models across all 30 UEA datasets under the MambaSL protocol, highlighting the best and second-best results.The baseline results are reused from the original MambaSL benchmark, while RTSM is evaluated under the same training and selection procedure.Source: §B.1 · p. 21, Table 6 · pp. 22–23

B.2 FULL RESULTS ON THE UEA BENCHMARK UNDER VALIDATION PROTOCOL

This section reports per-dataset accuracy for deep-learning models under the validation protocol, with configuration and checkpoint selection based only on validation data.

B.2 FULL RESULTS ON THE UEA BENCHMARK UNDER VALIDATION PROTOCOL

  • Configurations and checkpoints are selected exclusively on the validation split, while the official test set is reserved for final evaluation.The table presents the complete per-dataset results for the evaluated deep-learning models.Source: §B.2 · p. 24
  • Table 7 reports classification accuracy for deep-learning models on the 30 UEA datasets under the validation protocol, marking the best and second-best results.The official test set is reserved for final evaluation.Source: Table 7 · p. 24, §B.2 · pp. 24–25

B.3 RECURSIVE ARCHITECTURE AND TRAINING ABLATIONS

The ablations compare architectural and training choices for RTSM, with configuration names encoding those choices. They test the reference configuration against tokenizer-only and two-state alternatives across eight datasets.

B.3 RECURSIVE ARCHITECTURE AND TRAINING ABLATIONS

  • H, L, and G encode outer cycles, inner cycles, and gradient-tracked cycles; shared and separate distinguish one reasoning network from distinct two-state networks.The separate-network variant uses the TRM training recipe and is not an evaluation of HRM itself.Source: §B.3 · p. 26
  • Suffixes identify modifications including tokenizer-only prediction, latent-state resets, forced full training budgets, temporal MLPs, and alternative pooling.The rima and pdc suffixes denote learned EMA-style latent-state reweighting and progressive depth curriculum, respectively.Source: §B.3 · p. 26
  • The ablations report three-seed accuracy comparisons across eight datasets for the reference configuration, tokenizer-only baseline, and shared two-state variant.Table 8 covers architecture and training ablations; Tables 9–11 compare the reference configuration and tokenizer-only baseline with the shared two-state variant.Source: Table 8 · p. 26, Table 9 · p. 27, Table 10 · p. 27

B.4 RECURSIVE DYNAMICS AND INFERENCE-TIME SCALING ANALYSIS

The appendix documents an inference-time probe of fixed RTSM checkpoints, reporting how stochastic depth and breadth relate to oracle coverage and state changes. Across the tested probe, deeper trajectories stabilize state residuals, while accuracy shows little consistent improvement.

B.4.1 INFERENCE-TIME PROBE

  • The probe records deterministic and stochastic trajectories from fixed checkpoints across 28 UEA datasets, excluding EigenWorms and InsectWingbeat for compute-budget reasons.Stochastic rollouts use randomized initial states and noise between steps; smaller depth and breadth settings are evaluated as prefixes and subsets of the maximum-budget run.Source: §B.4.1 · p. 28

B.4.2 FULL DEPTH–BREADTH GRID

  • Mean stochastic residual falls from approximately 1.409 at D = 1 to 0.0193 at D = 8, then settles near 0.0167 for D ≥16; breadth has little effect at fixed depth.Across the full grid, depth has the larger measured effect on average state-update magnitude.Source: §B.4.2 · p. 29, Table 12 · p. 29, Figure 5 · p. 29
  • 77.71% oracle accuracy occurs at D = 1, B = 512, the highest value in the evaluated depth–breadth grid.At B = 512, oracle accuracy falls to 75.23% at D = 2 and 74.03% at D = 4, then varies without a consistent trend.Source: Table 12 · p. 29, Figure 5 · p. 29, §B.4.2 · p. 29

B.4.3 DETERMINISTIC RECURSIVE DYNAMICS

  • 72.42% accuracy is reached after the first deterministic step and fluctuates within roughly half a percentage point thereafter, although residual decreases to 0.0067 by step 64.Cross-entropy increases overall across the trajectory and is reported as a diagnostic, without further conclusions drawn from it.Source: §B.4.3 · p. 29, Table 13 · p. 30, Figure 6 · p. 30

B.4.4 REACHABLE HEADROOM AND TRAJECTORY SELECTION

  • Positive headroom is concentrated in PhonemeSpectra and EthanolConcentration, whose deterministic accuracies are among the lowest of the 28 datasets.The macro table compares the oracle upper bound with three label-free selectors; the per-dataset distribution shows the macro average is not representative of a typical dataset.Source: §B.4.4 · pp. 30–31, Table 14 · p. 30, Table 16 · p. 31

B.4.5 SCOPE AND RELATION TO PRIOR INFERENCE-TIME SCALING

  • The probe tests inference-only perturbations on checkpoints trained without them, so whether EqR-style perturbation training induces similar time-series scaling remains untested.The probe is closer to PTRM but differs by also randomizing the initial state.Source: §B.4.5 · p. 31

C.1 TOKENIZER COMPUTATIONS

  • For these checkpoints and perturbations, greater breadth adds little oracle coverage at the largest depth, and the tested selectors do not improve accuracy.The tokenizer uses channel projection, gated residual temporal filtering, densely connected dilated convolutions, and a final patch projection.Source: §B.4.5 · p. 31, §C.1 · p. 32

C.2 TRAINING SETTINGS

The training settings specify the benchmark optimizer and stopping protocol, while the component comparison situates RTSM alongside HRM and TRM.

C.2 TRAINING SETTINGS

  • The benchmark uses AdamW with learning rate 10^-4, weight decay 1, constant learning rate, zero dropout, and a nominal 100-epoch budget.Batch size is selected per dataset, with protocol-specific early stopping and checkpoint selection described separately.Source: §C.2 · p. 32
  • Training initializes the halt logit as negative and uses an exploration rule to suppress early halting on some trajectories; benchmark inference always uses 16 steps.The halt projection starts with wq = 0 and bq = −5; with probability 0.1, training draws a minimum depth uniformly from 2 through 16.Source: §C.2 · p. 32

C.3 COMPONENT COMPARISON

  • The component comparison covers HRM, TRM, and RTSM, focusing on input representation, state formulation, and sequence-level readout.It follows released HRM and TRM implementations; the HRM-style ablation changes state structure and network sharing within the TRM training recipe.Source: §C.3 · p. 33, Table 17 · p. 33

Related papers

Loading 2610.23353v1…