PaperlayerView exact revision on arXiv

Source-linked AI summary

Label-Efficient Time Series Classification at Scale: A Dual-Stream OSSE-LSTM with Counterfactual Attribution

Nguyen Ho, Bach Tung Tran, Trung Ky Nguyen, Zhenchang Xia, Bolong Zheng, Long Van Ho

arXiv:2610.02704v1cs.AI

TL;DR

With few labels, prototype estimates can be affected by irrelevant variation, while explanations should reflect class separation rather than an isolated score. The paper proposes a dual-stream episodic metric learner with prototype-margin attribution; on 19 univariate UCR datasets it leads across support sizes and remains within a 0.36-point accuracy band.

  • Problem

    With few support examples, irrelevant variation can displace prototypes, and explanations for prototype decisions should reflect separation from competing classes rather than an isolated score.

    Source: §I · p. 2

  • Method

    Dual-Stream OSSE-LSTM combines multi-scale Omni-Scale CNN features and Bidirectional LSTM context for prototype learning, while C-IG attributes the target-opponent prototype margin and guides test-time prototype refinement.

    Source: §I · p. 2

  • Results

    96.36% was OSSE-LSTM’s lowest average accuracy across K = 2 to K = 8 on 19 univariate UCR datasets, exceeding the best result any baseline reached at any K, 93.99%.

    Source: §VI · p. 10

  • Takeaways & Limitations

    C-IG-guided test-time refinement gives small but consistent gains where unguided noise augmentation degrades accuracy, without updating the trained encoder.

    Source: §I · p. 2, §VI · p. 10

Abstract

from arXiv · show

Time series are produced continuously at enormous scale by industrial equipment, wearables, power grids, and clinical monitors, yet annotation remains manual, expensive, and expert-dependent. The binding constraint in large-scale time series analytics is therefore not data volume but label volume, and the question facing a practitioner is concrete: how many examples per class must be labeled before a classifier becomes usable? We study this question directly, in a regime where the label space is fixed and known in advance and the decision rule must be constructed from only K labeled examples per class. We propose Dual-Stream OSSE-LSTM, an episodic metric-learning framework that pairs an Omni-Scale CNN with Squeeze-and-Excitation recalibration, for multi-scale motif extraction without per-dataset kernel tuning, with a Bidirectional LSTM for global temporal context. The two streams are independently normalized and fused into a prototype-oriented embedding. Because decisions taken from a few labels must also be explainable, we introduce Counterfactual Integrated Gradients (C-IG), which attributes the prototype margin between target and opposing classes rather than an isolated classifier logit, and reuses the resulting maps as soft masks for test-time prototype refinement without updating the encoder. On 19 univariate UCR datasets, OSSE-LSTM attains the highest average accuracy and per-dataset win count at every support size, and its accuracy remains within a 0.36-point band (96.36-96.72%) across that range. Its weakest configuration still exceeding the best result any compared baseline achieves at any K (93.99%).

I. INTRODUCTION

Large time-series archives outgrow manual labeling, so the paper studies classification from K examples per class in a fixed label space and proposes a prototype-oriented, dual-stream framework with contrastive explanations.

I. INTRODUCTION

  • Manual labels accumulate more slowly than signals, making the practical question how many examples per class are needed for a usable classifier in a fixed, known label space.The paper frames this as the annotation-cost regime of large operational archives, where only K labeled examples per class may be available.Source: §I · pp. 1–2
  • With only a few support examples, noise, drift, and phase shifts can displace prototypes, and poorly organized embeddings mean adding examples may move prototypes farther from class structure.Thus label efficiency depends on each labeled example contributing usefully to the prototype.Source: §I · pp. 1–2
  • Integrated Gradients typically explains an isolated class score, but prototype decisions are contrastive and should be explained through evidence separating target and competing classes.C-IG attributes the target-opponent prototype margin and uses its maps to refine test-time prototypes without updating the encoder.Source: §I · p. 2
  • Dual-Stream OSSE-LSTM combines an Omni-Scale CNN with Squeeze-and-Excitation for multi-scale motifs and a Bidirectional LSTM for global temporal context, then fuses normalized streams for prototype classification.The omni-scale design avoids dataset-specific receptive-field tuning across heterogeneous signal morphologies.Source: §I · p. 2
  • On 19 univariate UCR datasets, OSSE-LSTM had the highest average accuracy and per-dataset win count at every support size; its accuracy varied by under 0.4 points.At two examples per class it reached 96.36%, while convolutional baselines lost up to 12 points from 2- to 8-shot; C-IG-guided refinement gained consistently where unguided augmentation degraded accuracy.Source: §I · p. 2

A. Time Series Representation Learning

  • Classical time-series methods often use hand-crafted distances or costly searches, while convolutional models capture local patterns and hybrids combine convolution with recurrent context.Pooling may weaken global ordering; the paper situates hybrid representations in episodic metric learning so prototypes can be formed from few labels.Source: A. Time Series Representation Learning · pp. 2–3, §I · p. 2

B. Few-Shot Time Series Classification

The paper places its approach among few-shot time-series methods that integrate local and global representations, then motivates explanations aligned with prototype comparisons and defines episodic prototype classification.

B. Few-Shot Time Series Classification

  • Few-shot methods for time series include TapNet, TS2Vec, and DPSN, but existing approaches typically emphasize either local motifs or global sequence embeddings.The paper’s OSSE-LSTM is designed to integrate both within one framework.Source: B. Few-Shot Time Series Classification · p. 3

C. Interpretability for Prototype-Based Time Series Models

  • Prototype explanations should reflect relative distance to competing classes; C-IG integrates gradients between target and opposing class representatives, and its maps guide test-time support augmentation.Other cited approaches include Grad-CAM-style maps and Integrated Gradients, whose baseline choice is non-trivial for time series.Source: C. Interpretability for… · p. 3

A. Time Series and Episodic Few-Shot Classification

  • In an episode, examples from sampled classes are split into K labeled support examples and R queries per class, and queries must be classified using only the support set.The task is described as an N-way K-shot task.Source: A. Time Series and Episodic Few-Shot… · p. 3

B. Prototypical Metric Learning

  • A prototype network embeds each sample, represents each class by the mean support embedding, and assigns queries to the nearest prototype using squared Euclidean distance.Class probabilities use a softmax over negative distances, and training minimizes average query negative log-likelihood.Source: B. Prototypical Metric Learning · p. 3

C. Omni-Scale Convolution and Receptive Fields

The OS-CNN uses prime-sized kernels to cover a broad range of temporal scales without dataset-specific kernel-size tuning. Squeeze-and-Excitation recalibrates the resulting feature channels to emphasize informative ones.

C. Omni-Scale Convolution and Receptive Fields

  • Prime-sized 1D kernels can combine to realize essentially any temporal span, covering varied discriminative-pattern lengths without dataset-specific kernel-size tuning.Combining prime sizes in the first two stages with {1, 2} in the third covers even and adjacent odd receptive-field lengths.Source: C. Omni-Scale Convolution and… · p. 4
  • The kernel-size sets P(j) define the available kernels at each OS stage, and their combinations determine the receptive-field lengths along a three-stage path.For the first two stages the sets use prime kernel sizes; the third uses {1, 2}.Source: C. Omni-Scale Convolution and… · p. 4

D. Channel Recalibration

  • SE averages each feature channel over time, then uses a two-layer bottleneck, ReLU, and sigmoid to produce weights that rescale the channels.The bottleneck uses reduction ratio rse; the resulting channel weights lie in (0, 1).Source: D. Channel Recalibration · p. 4

E. Bidirectional Recurrent Temporal Modeling

The method combines local multi-scale convolutional features with bidirectional temporal context in an episodically trained embedding, then uses prototype-aware attribution to explain class separation. Its OSSE stream applies parallel multi-kernel convolutions followed by channel recalibration.

E. Bidirectional Recurrent Temporal Modeling

  • The BiLSTM reads each series in both temporal directions and concatenates directional hidden states to represent context beyond finite convolutional receptive fields.The concatenated states are averaged over time to produce a fixed-length representation, then fused with the convolutional representation.Source: E. Bidirectional Recurrent Temporal… · p. 4

F. Integrated Gradients for Input Attribution

  • Integrated Gradients attributes a scalar output along a straight path from a baseline to the input, but prototype classification requires a distance-based scalar rather than a fixed classifier logit.The paper introduces a scalar function reflecting the distance-based decision rule for Prototypical Networks.Source: F. Integrated Gradients for Input… · p. 4

IV. METHODOLOGY

  • OSSE-LSTM has a two-phase workflow: episodic few-shot training learns a prototype-oriented embedding, followed by C-IG interpretation of target–opponent class separation.An optional test-time strategy uses C-IG maps to guide support augmentation without updating the trained encoder.Source: §IV · p. 4

A. Phase I: Dual-Stream Episodic Learning

  • Phase I fuses OSSE local features and BiLSTM temporal context into a normalized embedding trained with an episodic objective.The embedding function maps an input series from R^C×L to R^d.Source: A. Phase I: Dual-Stream Episodic… · p. 4

1) Omni-Scale Squeeze-and-Excitation (OSSE) Convolutional Stream:

  • The OSSE stream stacks layers of parallel multi-kernel convolution, batch normalization, ReLU, and Squeeze-and-Excitation recalibration.The parallel kernels capture features across multiple receptive-field sizes.Source: 1) Omni-Scale Squeeze-and-Excitation… · p. 4

2) Bidirectional Recurrent Stream:

The recurrent stream summarizes past-to-future and future-to-past dependencies, then fuses its representation with the convolutional stream in a normalized embedding for prototype-based classification.

2) Bidirectional Recurrent Stream:

  • The BiLSTM representation summarizes dependencies in both temporal directions that local convolutional receptive fields may not capture.Dropout is applied to recurrent outputs.Source: 3) Feature Fusion and Embedding… · p. 5

3) Feature Fusion and Embedding Projection:

  • Each stream output is L2-normalized before concatenation because prototype classification uses Euclidean distances.The concatenated vector receives layer normalization followed by L2 normalization.Source: 3) Feature Fusion and Embedding… · p. 5
  • The final embedding feeds the class prototypes, query probabilities, and episodic loss.Both stream parameter sets are collected in ϕ.Source: 3) Feature Fusion and Embedding… · p. 5

4) Episodic Optimization:

Episodic training updates the shared encoder from support-set prototypes and query predictions, while inference keeps the encoder fixed. C-IG attributes target–opponent prototype separation along a path from an opposing-class reference to the target input.

4) Episodic Optimization:

  • Each training episode samples N classes with K support and R query examples per class, then updates the shared encoder using the episodic objective.At inference, the encoder remains fixed and queries are classified using prototypes computed from the support set.Source: 4) Episodic Optimization · p. 5

B. Phase II: Prototype-Aware Counterfactual Attribution

  • C-IG explains decisions in the input time domain by attributing prototype separation, identifying regions that move an input closer to its target prototype and farther from opposing ones.The procedure returns a class-level attribution map after processing opposing classes and fusing their maps.Source: B. Phase II: Prototype-Aware… · p. 5

1) Prototype-Margin Score:

  • The prototype-margin score compares target and opposing prototype distances; larger scores mean the input is closer to the target than to the opponent.Attributing this score identifies regions supporting the target class over that opponent.Source: 1) Prototype-Margin Score · pp. 5–6

2) Counterfactual Interpolation Path:

  • C-IG uses a representative opposing-class input as its reference and interpolates from that signal toward the target input.Uniformly sampled interpolation coefficients approximate the path integral.Source: 2) Counterfactual Interpolation Path · p. 6

3) Pairwise C-IG Attribution:

Pairwise C-IG attributes the target–opponent prototype margin along a counterfactual path, then fuses opponent-specific maps to highlight discriminative intervals. The maps also guide test-time prototype refinement without updating the encoder.

3) Pairwise C-IG Attribution:

  • C-IG averages prototype-margin gradients along a counterfactual path for each target–opponent pair, then smooths and normalizes the raw map.Temporal smoothing suppresses isolated spikes, while normalization makes maps comparable between opponents.Source: 3) Pairwise C-IG Attribution · p. 6

4) Multi-Opponent Fusion:

  • Closer opponents receive larger weights in the fused class-level map because they represent harder class boundaries.The resulting map highlights regions separating the target from its most relevant opponents; its temporal importance curve is attribution evidence, not a conventionally mined shapelet.Source: 4) Multi-Opponent Fusion · p. 6

C. Attribution-Guided Test-Time Prototype Refinement

  • Attribution maps are smoothed, normalized over time, and broadcast across channels to form soft masks for support refinement.The mask preserves high-attribution regions and perturbs low-attribution ones to reduce prototype sensitivity to irrelevant background variation.Source: C. Attribution-Guided Test-Time… · p. 6
  • The multi-opponent margin increases as a sample moves farther from opposing prototypes and closer to its own.The support-specific attribution uses a counterfactual path from an episode-local baseline formed by averaging support samples from other classes.Source: C. Attribution-Guided Test-Time… · p. 6

M TTA

At test time, attribution-guided masks generate augmented support samples by perturbing low-attribution regions, and the augmented samples refine class prototypes without changing the encoder.

M TTA

  • The mask preserves high-attribution regions and directs noise mainly into low-attribution regions when generating augmented support samples.The mask is scaled using each support sample’s per-channel temporal standard deviation, and β controls perturbation strength.Source: M TTA · p. 6
  • The refined prototype is the mean embedding over the original and augmented support samples.For each class, the augmented samples are added to its support set before computing this mean.Source: M TTA · pp. 6–7
  • Refined inference recomputes support-derived prototypes for query probabilities without updating encoder parameters.It uses attribution-guided augmentations in place of episodic training updates.Source: M TTA · p. 7

V. EXPERIMENTS

The experiments assess limited-data classification, ablations, interpretability, and attribution-guided refinement using 19 univariate UCR datasets. They compare six modeling baselines under an episodic protocol with varying labeled examples per class.

V. EXPERIMENTS

  • Experiments examine limited-data classification, component ablations, prototype-aware interpretability, and C-IG-guided test-time refinement.Source: §V · p. 7

1) Datasets:

  • 19 univariate UCR datasets are used, with no handcrafted features or domain-specific preprocessing.Sequences are padded to consistent shapes within episodic batches when necessary.Source: 1) Datasets · p. 7

2) Baselines:

  • Six baselines span recurrent-convolutional, convolutional-transform, prototype-learning, shapelet, self-supervised, and multi-scale convolutional approaches.DPSN is described as the only competitor targeting both label efficiency and interpretability; TapNet is identified as the closest metric-learning baseline.Source: 2) Baselines · p. 7

3) Evaluation Protocol:

  • The episodic evaluation varies training examples per class across K ∈ {2, 4, 6, 8} and reports Top-1 accuracy (%) ± standard deviation over repeated test episodes.Support samples form class prototypes, and queries are classified by distance to those prototypes.Source: 3) Evaluation Protocol · p. 7

4) Implementation Details:

  • The OSSE branch caps its maximum receptive-field range at min(⌊L/4⌋, 89), while the BiLSTM uses 16 hidden units per direction and dropout 0.5.All evaluations use prototype-based inference without test-time parameter updates.Source: 4) Implementation Details · p. 7

B. Effect of Support Set Size on Classification Results

OSSE-LSTM maintains the strongest aggregate results across 2- to 8-shot settings, although dataset-level responses to more support examples vary. The ablation and interpretability analyses show component trade-offs and class-relevant temporal evidence.

B. Effect of Support Set Size on Classification Results

C. Ablation Study

  • At ρ = 0.9, full OSSE-LSTM achieves the best average accuracy and best-count (11), though its recurrent branch can increase support-set sensitivity in low-data settings.The ablation compares OS-CNN, OS-CNN+SE, OS-CNN+BiLSTM, and the full model on a 14-dataset subset.Source: C. Ablation Study · pp. 7–8, Table III · p. 9

D. Interpretability Analysis

  • For ECG200, explanations highlight early deflections and recovery in both heartbeat classes, with class-specific regions reflecting differences in peak morphology, trough depth, and recovery.The normal heartbeat includes a sharp peak and following negative deflection; the myocardial-infarction event uses different evidence boundaries and types.Source: D. Interpretability Analysis · p. 8, Figure 2 · p. 9
  • On SonyAIBORobotSurface1, explanations distinguish carpet’s impact-to-trough transition and damped recovery from cement’s stronger, less-damped response.The selected regions include a short interval around the initial impulse and longer-range vibration patterns.Source: D. Interpretability Analysis · pp. 8–9, Figure 3 · p. 9

E. Attribution-Guided Test-Time Prototype Refinement

C-IG-guided test-time refinement uses attribution maps to preserve important regions while perturbing low-attribution regions. The paper reports consistent but dataset-dependent gains, alongside strong benchmark results for OSSE-LSTM across support sizes.

E. Attribution-Guided Test-Time Prototype Refinement

  • C-IG-guided TTA perturbs mainly low-attribution support regions while preserving high-attribution regions; both TTA variants use the same trained checkpoint.Each TTA result is compared with the standard baseline from the same run because episodic sampling varies standard accuracy.Source: E. Attribution-Guided Test-Time… · p. 9, Table IV · p. 10
  • C-IG-guided TTA avoids random-noise degradation and gives small, consistent gains, with the largest gain on ArrowHead and protection against severe degradation on Wine.Gains on ECG200 and ItalyPowerDemand are smaller because their standard prototypes are already relatively stable.Source: E. Attribution-Guided Test-Time… · p. 10

VI. CONCLUSION

  • 96.36–96.72% is OSSE-LSTM’s accuracy band across 2- to 8-shot, with the highest average accuracy and win count at every support size across 19 UCR datasets.Its weakest configuration exceeds the best result any baseline reaches at any K (93.99%).Source: §VI · p. 10
  • C-IG attributes the target–opponent prototype margin and reuses the maps for test-time refinement, rather than attributing an isolated classifier logit.The refinement is auxiliary, not a universal improvement.Source: §VI · p. 10, E. Attribution-Guided Test-Time… · p. 10
Loading 2610.02704v1…