PaperlayerView exact revision on arXiv

Source-linked AI summary

A Survey on Time-Series Pre-Trained Models

Qianli Ma, Zhen Liu, Zhenjing Zheng, Ziyang Huang, Siying Zhu, Zhongzhong Yu, James T. Kwok

arXiv:2305.10716v2cs.LGcs.AI

TL;DR

Large-scale well-labeled time-series datasets are difficult to construct, motivating alternatives to deep learning approaches reliant on labeled data. This survey organizes TS-PTMs by pre-training technique and experimentally analyzes them; it reports dataset-dependent classification performance and points to patch-based pre-training for forecasting and anomaly detection.

  • Problem

    Constructing large, well-labeled time-series datasets is difficult, despite deep learning models' reliance on massive labeled data for TSM.

    Source: C. Why Pre-Trained Models? · p. 4

  • Method

    The survey categorizes TS-PTMs by supervised, unsupervised, and self-supervised pre-training and compares methods in a uniform setting across 27 methods, 434 datasets, and 679 transfer-learning sets.

    Source: §I · p. 2

  • Results

    Transfer learning-based TS-PTMs perform poorly on small UCR classification datasets but excellently on other publicly available large datasets; patch-based pre-training is identified as a future focus for forecasting and anomaly detection.

    Source: §I · p. 2

  • Takeaways & Limitations

    The taxonomy and systematic review are intended to guide understanding, application, and future study of TS-PTMs.

    Source: §I · p. 2

    Current Transformer-based TS-PTMs mainly focus on classification and forecasting, leaving their performance on other downstream tasks for further exploration.

    Source: 1) Reconstruction-based PTMs · p. 9

Abstract

from arXiv · show

Time-Series Mining (TSM) is an important research area since it shows great potential in practical applications. Deep learning models that rely on massive labeled data have been utilized for TSM successfully. However, constructing a large-scale well-labeled dataset is difficult due to data annotation costs. Recently, pre-trained models have gradually attracted attention in the time series domain due to their remarkable performance in computer vision and natural language processing. In this survey, we provide a comprehensive review of Time-Series Pre-Trained Models (TS-PTMs), aiming to guide the understanding, applying, and studying TS-PTMs. Specifically, we first briefly introduce the typical deep learning models employed in TSM. Then, we give an overview of TS-PTMs according to the pre-training techniques. The main categories we explore include supervised, unsupervised, and self-supervised TS-PTMs. Further, extensive experiments involving 27 methods, 434 datasets, and 679 transfer learning scenarios are conducted to analyze the advantages and disadvantages of transfer learning strategies, Transformer-based models, and representative TS-PTMs. Finally, we point out some potential directions of TS-PTMs for future work.

I. INTRODUCTION

The survey reviews time-series pre-trained models (TS-PTMs), classifies them by pre-training technique, and experimentally analyzes transfer strategies and representative models. It situates this work against the challenge of learning useful time-series representations when large labeled datasets are difficult to obtain.

A. Time-Series Mining Tasks

  • TSM fundamentally seeks appropriate time-series representations, since downstream mining tasks operate on those representations.Traditional representations such as shapelets can be time-consuming because they rely heavily on domain or expert knowledge.Source: §I · p. 1

I. INTRODUCTION

  • Deep learning models have been used for TSM, but constructing large, well-labeled datasets is difficult because of data acquisition and annotation costs.Time-series augmentation must account for temporal and multi-scale dependencies, and its design generally relies on expert knowledge.Source: §I · p. 1
  • The survey groups TS-PTMs into supervised classification- and forecasting-based methods, unsupervised reconstruction-based methods, and self-supervised consistency- and pseudo-labeling-based methods.Models using multiple pre-training tasks are classified by their core task.Source: §I · p. 2
  • Transfer learning-based TS-PTMs perform poorly on small UCR classification datasets but perform excellently on other publicly available large datasets.For forecasting and anomaly detection, the survey identifies patch-based pre-training as a future research focus.Source: §I · p. 2
  • The survey discusses future directions by considering datasets, deep learning models, time-series properties, adversarial attacks, noisy labels, and pre-trained large language models.It also compares 27 methods, 434 datasets, and 679 transfer-learning sets in a uniform experimental setting.Source: §I · p. 2

1) Time-Series Classification:

  • In time-series classification, a classifier is trained on labeled samples and then used to assign labels to unseen samples.Each sample may be univariate or multivariate, with a corresponding one-hot label vector.Source: 1) Time-Series Classification · p. 2

2) Time-Series Forecasting:

The section defines several time-series mining tasks and surveys recurrent and convolutional model families used to address them.

2) Time-Series Forecasting:

  • Forecasting uses historical observations to predict future values over a specified horizon.Models generally account for trend, seasonality, and correlations among observed values.Source: 2) Time-Series Forecasting · p. 2

3) Time-Series Clustering:

  • Time-series clustering partitions samples into groups that maximize within-cluster similarity and between-cluster dissimilarity.Temporal and multi-scale dependencies make time-series clustering more difficult than clustering static data.Source: 3) Time-Series Clustering · pp. 2–3

4) Time-Series Anomaly Detection:

5) Time Series Imputation:

  • Time-series imputation replaces missing values with realistic estimates to facilitate downstream mining tasks.The predicted values are generated by the imputation technique.Source: 5) Time Series Imputation · p. 3

6) Time Series Extrinsic Regression:

  • Time-series extrinsic regression learns a relationship between a time series and a continuous scalar output.Unlike forecasting, it predicts an external variable rather than future values in the series.Source: 6) Time Series Extrinsic Regression · p. 3

1) Recurrent Neural Networks:

  • RNNs use recurrent hidden layers; cited applications include forecasting and clustering.Examples combine dynamic attention with sequence-to-sequence modeling or use dilated RNNs to extract multi-scale temporal dependencies.Source: 1) Recurrent Neural Networks · p. 3

2) Convolutional Neural Networks:

  • CNNs aggregate local information across nearby time steps, while TCNs use causal convolutions that prevent future-to-past information leakage.The cited CNN approaches capture multi-scale temporal features; TCNs have been reported as more accurate, simpler, and more efficient than recurrent networks across diverse sequence tasks.Source: 2) Convolutional Neural Networks · p. 3

3) Transformers:

This section surveys deep-learning architectures for time-series mining and explains why pre-training is used when labeled data are scarce. It organizes TS-PTMs by pre-training technique and reviews supervised classification strategies, including transfer, alignment, and model reprogramming.

3) Transformers:

  • Transformers use self-attention to model long-range time-series dependencies, while graph neural networks represent intra-sample or inter-sample relationships as graphs.The section also notes a Transformer forecasting model with O(L log L) time and space complexities.Source: 3) Transformers · p. 3, 4) Graph Neural Networks · p. 3

C. Why Pre-Trained Models?

  • Limited labeled data and costly annotation motivate pre-training, which can provide better initialization and representations with less reliance on expert knowledge.The paper notes that PTMs generally result in better generalization performance on downstream tasks.Source: C. Why Pre-Trained Models? · pp. 3–4

III. OVERVIEW OF TS-PTMS

  • The proposed TS-PTM taxonomy systematically classifies models according to their pre-training techniques.Fig. 3 presents the taxonomy.Source: §III · p. 4, Figure 3 · p. 4

A. Supervised PTMs

  • Classification-based PTMs include universal encoders, aligned encoders, and model reprogramming; transfer can help or hurt depending on source-target similarity.The early supervised transfer strategy adapts models trained on labeled source datasets to target datasets.Source: 1) Classification-based PTMs · p. 4, A. Supervised PTMs · p. 4

1) Classification-based PTMs:

  • Aligned encoders map source and target data into a shared feature space to learn domain-invariant representations, using MMD or adversarial learning.Time-series alignment remains challenging because of complex dynamics, timestamp distribution shifts, and associations among variables.Source: 1) Classification-based PTMs · pp. 5–6
  • Voice2Series reprograms a large acoustic model with transformed time-series inputs and mapped labels, while Time-LLM applies reprogramming to transfer LLM knowledge to time-series tasks,.Voice2Series uses voice data for pre-training; its input transformation includes zero-padding and learned parameters for aligning source and target distributions.Source: 1) Classification-based PTMs · p. 6
  • Universal encoders are jointly pre-trained across labeled source datasets and transferred to target tasks, but typically require many labeled source samples.Transfer may be positive for similar source and target datasets and negative for dissimilar ones; finding a suitable large labeled time-series dataset remains an open challenge.Source: 1) Classification-based PTMs · pp. 4, 6

2) Forecasting-based PTMs:

Forecasting-based TS-PTMs use observed time-series values to supervise predictions and learn temporal representations. The section describes autoregressive and task-adaptive approaches, while noting sampling-bias and accumulated-error challenges.

2) Forecasting-based PTMs:

  • Forecasting uses observed past values to predict future values, making future observations supervisory information without manual labels.A PTM is pre-trained on a source dataset and its base-model weights are then fine-tuned on the target dataset.Source: 2) Forecasting-based PTMs · pp. 6–7
  • Autoregressive models, including RNNs, TCNs, and Transformers, predict future values from historical observations.The RNN example maps observations [x1, x2, x3, x4, x5] to future values [x6, x7, x8].Source: 2) Forecasting-based PTMs · p. 6, Figure 7 · p. 7
  • Task-adaptive pre-training uses meta-learning to seek initialization that can adapt to new tasks with few target samples.The described meta-learning procedure learns shared knowledge across tasks to provide initial predictor parameters for new tasks.Source: 2) Forecasting-based PTMs · p. 7, Figure 8 · p. 7
  • Forecasting-based PTMs can capture temporal dependencies, but avoiding biased subseries sampling remains challenging and one-step RNN forecasts may accumulate error.Transformer models that generate all predictions in one forward operation are presented as an alternative to one-step regression.Source: 2) Forecasting-based PTMs · p. 7

B. Unsupervised PTMs

Unsupervised TS-PTMs learn from unlabeled time series, often by reconstructing inputs through an encoder-decoder. Reconstruction methods include autoencoders, denoising strategies, and Transformer encoders, but their transferability and downstream scope remain open concerns.

B. Unsupervised PTMs

  • Unsupervised TS-PTMs avoid the labeled-sample requirement by pre-training on unlabeled data, often through reconstruction.An encoder maps the time series to a latent representation, and a decoder reconstructs the input, commonly using mean square error.Source: B. Unsupervised PTMs · p. 8, 1) Reconstruction-based PTMs · p. 8

1) Reconstruction-based PTMs:

  • Autoencoder-based Timenet encodes a univariate series into a fixed-dimensional vector and reconstructs a sequence; its use beyond classification is less explored.The encoder produces representation H from X, and the decoder reconstructs X from H.Source: 1) Reconstruction-based PTMs · p. 8
  • Transformer reconstruction methods use masking or denoising to learn time-series representations across multivariate, traffic, tabular, and speech data.Examples include masked-token prediction, recovering masked frames, and reconstructing altered speech data.Source: 1) Reconstruction-based PTMs · pp. 8–9
  • DAE-based TS-PTMs corrupt inputs with noise or masks and train the model to reconstruct the original series, a strategy gaining attention over vanilla autoencoders.Denoising reconstruction is used to learn more robust time-series representations.Source: 1) Reconstruction-based PTMs · pp. 8–9
  • DAE design for time series remains exploratory, and domain variation can impair transfer; Transformer TS-PTMs have mainly addressed classification and forecasting.The survey identifies reconstruction mechanisms that work across domains and other downstream tasks as areas needing further study.Source: 1) Reconstruction-based PTMs · p. 9

C. Self-supervised PTMs

Self-supervised TS-PTMs derive supervisory signals from the data, including consistency between augmented views and pseudo-labels. Consistency-based methods form positive and negative pairs in several ways, with contextual approaches reported across multiple downstream tasks but open design challenges remaining.

1) Consistency-based PTMs:

  • Consistency-based learning brings representations of positive views from the same sample closer while separating negative views from different samples.These methods commonly use contrastive learning and must construct pairs that capture time-series dynamics.Source: 1) Consistency-based PTMs · p. 9
  • Four consistency strategies form pairs from nested subseries, adjacent subseries, transformed views, or same-timestamp contexts.The survey distinguishes subseries, temporal, transformation, and contextual consistency.Source: 1) Consistency-based PTMs · pp. 9–10
  • Other approaches include SelfTime’s inter-sample and intra-temporal relations, CoST’s temporal and frequency-domain representations, and BTSF’s iterative temporal-spectral fusion.The cited work also reports contrastive methods for change-point detection and joint local-subseries and global-instance representations.Source: 1) Consistency-based PTMs · p. 10
  • TS2Vec uses timestamp masking and random cropping to create same-timestamp positive contexts, with hierarchical temporal and instance-level contrasts.The survey reports strong downstream performance on classification, forecasting, and anomaly detection.Source: 1) Consistency-based PTMs · p. 10
  • False positive pairs can arise from subseries sampling bias, while uniform time-series augmentations and multi-scale consistency remain challenges.Contextual consistency uses masking or dropout to capture temporal dependencies and can alleviate sampling bias among subsequences.Source: 1) Consistency-based PTMs · p. 10

2) Pseudo-labeling PTMs:

Pseudo-labeling creates self-supervisory targets from transformed time-series samples or expert features, but incorrect pseudo-labels remain a concern. The evaluation compares transfer strategies on classification datasets and reports that outcomes vary with dataset setting and source-data size.

2) Pseudo-labeling PTMs:

  • Pseudo-labeling assigns targets from subsequence temporal distance or expert features, but incorrect pseudo-labels may negatively affect learning.One described auxiliary task predicts which transformation was applied to an input sample.Source: 2) Pseudo-labeling PTMs · p. 11

IV. EXPERIMENTAL RESULTS AND ANALYSIS

  • The evaluation covers classification, forecasting, and anomaly detection, using UCR and UEA archives, four transfer scenarios, nine forecasting benchmarks, and univariate and multivariate anomaly datasets.Appendix B is identified for dataset, baseline, and implementation details.Source: §IV · p. 11

A. Performance of PTMs on Time-Series Classification

  • For UCR and UEA classification, training and test sets are merged before five-fold cross-validation divides data 60%-20%-20%; average test accuracy is the metric.The protocol addresses archives without specified validation sets and datasets whose test sets may exceed training sets.Source: A. Performance of PTMs on Time-Series… · p. 11

1) Comparison of Transfer Learning PTMs based on Supervised Classification and Unsupervised Reconstruction:

  • Across UCR target datasets, supervised classification transfer had the best average accuracy and positive-transfer counts, while the symmetric FCN decoder was better suited than the asymmetric RNN decoder.On the smallest targets, supervised transfer and symmetric-FCN reconstruction were not significantly different; overall positive-transfer counts were unsatisfactory.Source: 1) Comparison of Transfer Learning… · p. 11
  • On four independent scenarios, supervised classification transfer helped neural-stage detection and device diagnosis, while unsupervised RNN-decoder transfer helped activity recognition; all strategies transferred negatively for physical-status monitoring.The authors relate the latter result possibly to the small number of EMG samples.Source: 1) Comparison of Transfer Learning… · p. 11, Table II · p. 12

2) Comparison of TS-PTMs for Classification:

The classification, forecasting, and anomaly-detection comparisons show that TS-PTM performance varies by task and dataset. The survey also identifies limited dataset scale as a challenge for pre-training.

2) Comparison of TS-PTMs for Classification:

  • TS2Vec and GPT4TS rank first and second on 128 UCR datasets, while GPT4TS ranks first on 30 UEA datasets; TS2Vec is not significantly different from GPT4TS on UCR.On UEA, direct FCN classification ranks second and TS2Vec trails in average accuracy and rank; GPT4TS takes longer to fine-tune on UCR but trains faster than most compared methods on UEA.Source: 2) Comparison of TS-PTMs for… · p. 12

B. Performance of PTMs on Time-Series Forecasting

  • iTransformer and GPT4TS generally outperform other forecasting methods, while CoST leads on Weather and ILI.The comparison covers average test results across prediction lengths and nine public datasets; CoST’s trend and seasonal feature modeling is credited for its performance on Weather and ILI.Source: B. Performance of PTMs on Time-Series… · pp. 12–13, Table IV · p. 13

C. Performance of PTMs on Time-Series Anomaly Detection

  • On Yahoo and KPI, TS2Vec and Anomaly Transformer lead on F1, precision, and recall, while DOUNT performs better overall on the other six metrics.On the UCR anomaly archive, TS2Vec leads DOUNT on F1, F1-PA-10, F1-PA-50, and F1-PA-90, while DOUNT leads on the other six metrics; DCdetector has the best average across all twelve. On seven multivariate datasets, DCdetector leads on five, with TS2Vec and GPT4TS each leading on one.Source: C. Performance of PTMs on Time-Series… · p. 13, Table V · p. 13, Table VI · p. 14

A. Large-scale Time Series Datasets

  • Most UCR and UEA datasets have only hundreds or a few thousand samples, limiting their usefulness for pre-training; speech-domain and synthesized datasets are among the larger-scale alternatives discussed.Voice2Series transfers from a large speech-domain sequence dataset to general time-series classification, and studies have also combined datasets across domains to form pre-training corpora.Source: A. Large-scale Time Series Datasets · pp. 13–14

B. Inherent Properties of Time Series

The survey identifies unresolved challenges in transferring time-series representations, selecting architectures, and applying pre-trained models to robustness, noisy labels, and language-model settings. Its conclusion highlights patching and Transformer-based approaches for some tasks while pointing to other architectures as future directions.

B. Inherent Properties of Time Series

  • Time-series representations can capture temporal, multiscale, contextual, and frequency-domain structure, but their transferability across domains remains weakly interpretable.Unlike text, time-series data lack a unified semantic sequence dataset whose subsequences have consistent meanings across scenarios, making transferable segments a research challenge.Source: B. Inherent Properties of Time Series · p. 14

C. Deep Learning Models in Time Series

  • Transformers capture long-term dependencies and have been used for forecasting, but classification may favor discriminative subseries or multiscale features; CNNs, Mamba, and GNNs also merit study.The survey cites consistency-based PTMs with TCNs and patching strategies as strong on UCR and UEA classification, and identifies patching and iTransformer as promising avenues across tasks.Source: C. Deep Learning Models in Time Series · p. 14

D. Adversarial Attacks on Time Series

  • Experiments on 42 UCR datasets show vulnerability to adversarial examples, and improving time-series model robustness with TS-PTMs is identified as a direction for exploration.The survey notes that self-supervised learning improved robustness in other settings, while generating adversarial examples can be difficult when clusters lie far from classification boundaries.Source: D. Adversarial Attacks on Time Series · p. 15

E. Pre-Training Models for Time-Series Noisy Labels

  • Noisy labels are a concern because low-cost automatic labeling can introduce noise and deep models can fit noisy labels, while TS-PTMs for time-series noisy labels remain unstudied in the cited discussion.The passage reports only a few studies of time-series noisy labels, despite PTMs having been studied for noisy-label problems in computer vision.Source: E. Pre-Training Models for Time-Series… · p. 15

F. Pre-Trained LLMs for Time-Series Mining

  • The rationale for successful LLM fine-tuning on time series remains unclear, and one cited study finds pre-trained LLMs do not outperform models trained from scratch for forecasting.The survey also notes that a simple patching-and-attention encoder can achieve comparable results, and suggests textual descriptions or multimodal pre-training as directions to explore.Source: F. Pre-Trained LLMs for Time-Series… · p. 15

VI. CONCLUSION

  • Across the reviewed experiments, LLM fine-tuning with patching and Transformer-based models shows potential for classification and forecasting, while consistency-based PTMs with patching show promise for anomaly detection.The conclusion also identifies selecting suitable architectures, including CNNs or Mamba, as a promising direction for future TS-PTMs.Source: §VI · p. 15

APPENDIX A RELATED SURVEY ANALYSIS

The appendix compares this survey’s coverage and experiments with related surveys, then documents datasets and baseline methods used in its analyses. It describes the survey’s broader coverage of pre-training methods and transfer strategies alongside extensive experiments.

APPENDIX A RELATED SURVEY ANALYSIS

  • The survey covers six tasks and supervised, unsupervised, and self-supervised pre-training, while examining fine-tuning transfer strategies and experiments; prior surveys focus on narrower representation-learning or LLM-transfer topics.Its review spans RNN, CNN, TCN, Transformer, and GNN architectures and emphasizes the time-series domain.Source: Appendix A · p. 21, Table VII · p. 22
  • The experiments include 27 methods, 434 datasets, and 679 transfer-learning experiments, reproduced under consistent conditions for more objective comparisons.The passage describes the transfer experiments as classification-focused and calculated from source-to-target dataset migrations plus four independent time-series scenarios.Source: Appendix A · p. 21, Table VIII · p. 22

A. Source Datasets, Code and TS-PTMs

B. Experimental Datasets

  • For classification, UCR and UEA data are repartitioned with five-fold cross-validation into 60% training, 20% validation, and 20% test sets; forecasting and anomaly detection use separate named datasets.The forecasting set includes ETTh1, ETTh2, ETTm1, ETTm2, Electricity, Traffic, Weather, Exchange, and ILI; anomaly detection uses Yahoo, KPI, UCR, and seven multivariate datasets.Source: B. Experimental Datasets · pp. 22–23

C. Baselines

  • The baseline comparisons include supervised classification and reconstruction-based transfer strategies, plus task-specific TS-PTMs and conventional methods for classification, forecasting, and anomaly detection.The listed forecasting TS-PTMs include TS2Vec and CoST; anomaly detection compares TS2Vec with methods including SPOT, DONUT, and Anomaly Transformer.Source: C. Baselines · p. 23

1) Time-Series Classification:

The classification experiments compare direct supervised models with pre-trained models and report test accuracy under a common dataset split and evaluation setup. The section also documents the implementations and configurations used for the compared methods.

1) Time-Series Classification:

  • Classification experiments compare FCN, TCN, and Transformer direct classifiers with pre-trained methods including T-Loss, SelfTime, TS-TCC, TST, and TS2Vec across UCR and UEA datasets.The comparison uses test accuracy, mean-imputes missing values, and runs experiments on eight 1080Ti GPUs and two 3090 GPUs.Source: 1) Time-Series Classification · p. 23
  • The methods span CNN, TCN, Transformer, and other architectures, with the paper summarizing TS-PTMs in a dedicated table.The table is identified as a summary of Time-Series Pre-Trained Models.Source: 1) Time-Series Classification · p. 24, Table XI · p. 24
  • The implementation descriptions cover the selected TS-PTMs and baselines, including their backbones, training strategies, and code sources.For example, TS2Vec uses a TCN representation with an SVM classifier, while TimesNet is trained as a fully supervised backbone for this comparison.Source: 1) Time-Series Classification · p. 25

2) Time-Series Forecasting:

The forecasting section describes supervised baselines and pre-trained or representation-learning approaches; the anomaly-detection section lists evaluated methods and shared experimental settings.

2) Time-Series Forecasting:

  • CoST learns disentangled seasonal-trend representations with contrastive learning, and LongTrans uses convolutional self-attention and LogSparse attention for long-range forecasting.TS2Vec uses a TCN backbone to predict the next H timestamps from the last T observations.Source: 2) Time-Series Forecasting · p. 25
  • DLinear serves as a supervised forecasting baseline, while TCN, Informer, Autoformer, and TimesNet provide convolutional and Transformer-based alternatives.TimesNet is used as a backbone for fully supervised training to analyze its potential in time-series pre-training.Source: 2) Time-Series Forecasting · pp. 25–26
  • GPT4TS and TEMPO fine-tune pre-trained language models for forecasting, while PatchTST contributes patch-based and channel-independent strategies.The experiments use PatchTST's patch strategy with a Transformer trained in a fully supervised manner.Source: 2) Time-Series Forecasting · pp. 25–26

3) Time-Series Anomaly Detection:

  • The anomaly-detection comparisons include streaming TS2Vec, patch-based DCdetector, TimesNet, and GPT4TS.TimesNet is directly trained as a backbone; GPT4TS applies patch and channel-independent strategies to fine-tune language models.Source: 3) Time-Series Anomaly Detection · p. 26
  • Baseline training uses uniform random seeds and consistent dataset partitions, with model hyperparameters following the original authors' settings.The listed anomaly methods also include SPOT and DSPOT, LSTM-VAE, DONUT, and Spectral Residual.Source: 3) Time-Series Anomaly Detection · p. 26

D. Implementation Details

The authors describe their transfer-learning evaluation and implementation controls, then detail an FCN-based comparison of supervised classification and unsupervised reconstruction pre-training.

D. Implementation Details

  • The experiments compare pre-training strategies on downstream classification and target-dataset pre-training across classification, forecasting, and anomaly detection.The authors use UCR and four scenario datasets to examine transfer-learning performance, aiming to inform pre-training paradigms for different tasks.Source: D. Implementation Details · pp. 26–27
  • Classification experiments use z-score normalization, FCN and TCN models, uniform early stopping, and task-specific test-accuracy averaging across seeds.Forecasting and anomaly-detection preprocessing and hyperparameters follow TS2Vec; the comparative methods are reproduced from original authors' code.Source: D. Implementation Details · p. 27

A. Comparison of Transfer Learning PTMs based on Supervised Classification and Unsupervised Reconstruction

  • FCN is chosen as the transfer-learning backbone after considering training time and classification performance against TCN and Transformer on 128 UCR datasets.The comparison pre-trains the FCN encoder using supervised classification or unsupervised reconstruction with a symmetric FCN or asymmetric RNN decoder.Source: A. Comparison of Transfer Learning… · pp. 27–28
  • The FCN transfer study selects 15 largest-sample UCR datasets as sources and 45 of the remaining datasets as targets, stratified by target sample size.For each strategy and target sample-size group, 15 sources and 15 targets yield 225 transfer results; Fig. 12 encodes accuracy differences, with positive values indicating positive transfer and negative values negative transfer.Source: A. Comparison of Transfer Learning… · p. 28, Figure 12 · p. 28

B. Comparison of TS-PTMs on Classification

The section reports classification comparisons and visualizations of learned time-series features, alongside forecasting test-result scope and experimental configurations.

B. Comparison of TS-PTMs on Classification

  • Classification accuracy comparisons for Transformer-based and contrastive-learning PTMs cover 128 UCR and 30 UEA datasets, with results presented in Tables XVI and XVII.The reported table descriptions omit standard deviations for dataset-level classification performance.Source: B. Comparison of TS-PTMs on… · pp. 28–29

1) Visualization on the classification task:

C. Details of Results on Time-Series Forecasting

  • Forecasting results are reported across prediction lengths and datasets, with train-validation-test configurations drawn from TSVec, CoST, and TimesNet settings.The experiments use a unified maximum-epoch setting for several listed methods and otherwise follow each model's default parameters.Source: C. Details of Results on Time-Series… · pp. 29–30

D. Details of Results on Time-Series Anomaly Detection

The anomaly-detection results are organized across UCR anomaly subdatasets and multivariate datasets, with metric-specific tables and stated exclusions and evaluation constraints.

D. Details of Results on Time-Series Anomaly Detection

Loading 2305.10716v2…