PaperlayerView exact revision on arXiv

Source-linked AI summary

DTAAD: Dual Tcn-Attention Networks for Anomaly Detection in Multivariate Time Series Data

Lingrui Yu

arXiv:2302.10753v3cs.LG

TL;DR

Multivariate time-series anomaly detection must contend with scarce labels, complex data, and demands for rapid detection and diagnosis. DTAAD combines dual TCNs and Transformer attention in an autoregressive–autoencoder design with feedback. The authors report improved benchmark performance and reduced training time relative to baselines.

  • Problem

    Multivariate time-series anomaly detection faces scarce labels, high-dimensional dependencies, and demands for fast inference and anomaly-source tracking.

    Source: §1 · pp. 1–2

  • Method

    DTAAD combines an autoregressive–autoencoder design with local and global TCNs, Transformer attention, and a feedback mechanism.

    Source: §1 · pp. 2–3

  • Results

    DTAAD outperformed state-of-the-art methods on nine benchmark datasets, improving F1 and F1* scores by 8.38% and 8.97%, respectively.

    Source: §6 · p. 12

  • Takeaways & Limitations

    The authors describe DTAAD as a promising unsupervised anomaly-detection solution for high-dimensional data, with up to 99% less training time than baseline methods.

    Source: §6 · p. 12

    The formulation assumes a fully observed multivariate time series as input.

    Source: §3.1 · p. 4

Abstract

from arXiv · show

Anomaly detection techniques enable effective anomaly detection and diagnosis in multi-variate time series data, which are of major significance for today's industrial applications. However, establishing an anomaly detection system that can be rapidly and accurately located is a challenging problem due to the lack of anomaly labels, the high dimensional complexity of the data, memory bottlenecks in actual hardware, and the need for fast reasoning. In this paper, we propose an anomaly detection and diagnosis model, DTAAD, based on Transformer and Dual Temporal Convolutional Network (TCN). Our overall model is an integrated design in which an autoregressive model (AR) combines with an autoencoder (AE) structure. Scaling methods and feedback mechanisms are introduced to improve prediction accuracy and expand correlation differences. Constructed by us, the Dual TCN-Attention Network (DTA) uses only a single layer of Transformer encoder in our baseline experiment, belonging to an ultra-lightweight model. Our extensive experiments on seven public datasets validate that DTAAD exceeds the majority of currently advanced baseline methods in both detection and diagnostic performance. Specifically, DTAAD improved F1 scores by $8.38\%$ and reduced training time by $99\%$ compared to the baseline. The code and training scripts are publicly available on GitHub at https://github.com/Yu-Lingrui/DTAAD.

1 INTRODUCTION

The introduction frames multivariate time-series anomaly detection as challenging because labels are scarce, data are complex, and applications need rapid detection and diagnosis. DTAAD combines dual TCNs with Transformer attention, feedback, and an autoregressive–autoencoder design; the paper reports strong benchmark performance and reduced training time.

1 INTRODUCTION

  • The paper identifies data complexity, scarce labels, inference speed, and anomaly-source tracking as challenges for multivariate time-series anomaly detection.Source: §1 · p. 1
  • Classical unsupervised methods often neglect temporal correlation and dependencies between high-dimensional variables, while recurrent models can be slow and struggle with long-term trends.Source: §1 · p. 2
  • The authors introduce TCN into multivariate time-series anomaly detection and combine it with MAML to help maintain performance with limited data.Source: §1 · pp. 2–3
  • DTAAD combines an autoregressive and autoencoder design with local and global attention, dual TCNs, and feedback to reflect anomaly differences.Source: §1 · pp. 2–3
  • The design uses a single Transformer encoder layer and minimum dilation depth for the global receptive field, with training time reduced by 99% versus baseline methods.Source: §1 · p. 3
  • DTAAD obtained the most advanced anomaly-detection results in benchmark tests for most real applications.Source: §1 · p. 3

2 RELATED WORKS

The related work surveys deep unsupervised and weakly supervised anomaly-detection approaches, then reviews Transformer-based methods. It positions DTAAD as addressing performance bottlenecks while reducing training time.

2.1 Deep Unsupervised & Weakly Supervised Anomaly Detection

  • Deep unsupervised and weakly supervised methods include distance-, clustering-, and one-class-classification approaches, each with noted limits such as computation costs or sensitivity to clustering quality.Source: §2.1 · p. 3

2.2 Transformers for Deep Anomaly Detection

  • Transformer anomaly-detection work uses attention associations and anomaly-attention strategies, while TranAD applies a serial attention encoder for temporal trends and fast inference.Source: §2.2 · p. 3
  • The paper compares DTAAD with established baselines and describes its aim as alleviating anomaly-detection performance bottlenecks while significantly reducing training time.Source: §2.2 · pp. 3–4

3 BACKGROUND

The background formulates anomaly detection and diagnosis as future-step tasks over observed multivariate time series. It then describes standardization, noise augmentation, sliding windows, and anomaly-score thresholding, alongside the model's dual-TCN and Transformer architecture.

3.1 Problem Definition

  • The problem formulation assumes a fully observed multivariate time series and predicts future steps from the observed series.Source: §3.1 · p. 4
  • The formulation separates detecting anomalies at future steps from diagnosing them through dimension-wise analysis of anomaly timing and severity.Source: §3.1 · p. 4

3.2 Data Preprocessing

  • Preprocessing standardizes the data, adds white noise at 50 dB signal-to-noise ratio, and slices time series into sliding windows.Source: §3.2 · p. 4
  • The architecture pairs a TCN-based approximate autoregression layer with a Transformer encoder-decoder layer; local-attention predictions feed back to the global TCN.Source: Figure 1 · p. 5
  • The pipeline scales values using training-series minima and maxima, uses copy fill for initial windows, and selects anomaly thresholds with Peak Over Threshold.Source: §3.2 · p. 5

4 METHODOLOGY

DTAAD couples local causal and global dilated TCNs with a Transformer encoder and dual encoder-decoder reconstruction path. Training combines the two decoder errors, while test-time scoring applies dimension-wise thresholds.

4.1 Model Framework

  • DTAAD combines local causal and global dilated TCNs with a Transformer-based encoder-decoder path in an approximate autoregressive architecture.Local and global attention outputs feed separate decoders, and local prediction results are fed back to the global TCN.Source: §4.1 · pp. 5, 7
  • Causal convolutions provide shallow local detection, while dilated convolutions expand the global receptive field without adding much computational cost.The design uses the minimum number of dilated layers needed to cover the input history.Source: §4.1 · p. 6
  • The baseline uses a single Transformer encoder layer whose multi-head attention operates on queries, keys, and values derived from temporal convolutions and positional encoding.The encoder also includes a feed-forward sublayer, residual connections, and layer normalization.Source: §4.1 · p. 7, Figure 4 · p. 7

4.2 Training Procedure

  • Training minimizes a weighted combination of the two decoders’ reconstruction losses, with MSE used to measure each output against the original window.MAML is used for data augmentation because low anomaly rates can leave training under-fitted.Source: §4.2 · pp. 7–8

4.3 Anomaly Detection and Diagnosis

  • At test time, decoder 1’s reconstruction loss supplies anomaly scores, and POT dynamically selects a threshold for each dimension.The offline threshold uses extreme value theory to fit the score tail with a Generalized Pareto Distribution.Source: §4.2 · p. 8, §4.3 · pp. 8–9
  • In WADI, the dual-attention analysis reports extremely high attention to peaks and noise and describes dimension-specific anomaly detection using whole-series context.The number of attention heads is set to the dataset dimensionality to consider cross-dimensional features.Source: §4.3 · p. 9, Figure 6 · p. 9

5 EXPERIMENTS

The experiments compare DTAAD with baseline methods across nine datasets, using full-data and limited-data detection metrics, diagnostic measures, timing comparisons, and ablations. DTAAD generally outperforms most baselines, while the ablations indicate that both TCN components contribute to performance.

5.1 Datasets

  • The nine datasets were NAB, UCR, MBA, SMAP, MSL, SWaT, WADI, SMD, and MSDS, spanning sensor, spacecraft, industrial, and distributed-system data.The dataset descriptions include real-world traces, telemetry, water-treatment sensors, and application logs, metrics, and traces.Source: §5.1 · pp. 9–10

5.2 Performance Metrics

  • Evaluation included AUC* and F1* after training on 20% of the data, plus HitRate@P% and NDCG@P% for ranking anomalous dimensions.HitRate counts true anomalous dimensions among top-ranked priorities; NDCG discounts lower-ranked items.Source: §5.2 · p. 10

5.3 Results and Analysis

  • F1 improved by 8.38%, F1* by 8.97%, AUC by 1%, and AUC* by 9.98% versus the mean of comparison baselines across the nine-dataset union.The comparisons cover complete datasets and training with 20% of the data.Source: §5.3 · p. 10, Table 3 · p. 11, Table 4 · p. 12
  • DTAAD’s diagnostic detection range was 40.87%–62.35%; diagnostic scores rose by 7.5% on SMD and 18% on MSD versus the baseline, while training and inference time was 62%–99% below baseline methods.Timing was measured in seconds per epoch on the same machine and GPU with the same random seeds.Source: §5.3 · p. 11

5.4 Ablation Experiment

  • On 20% training data, the complete model outperformed all ablated versions, especially on larger datasets such as WADI and SMD.Removing TCN_Global and TCN_Local caused the largest F1* and AUC* drops on most datasets, particularly WADI and SMD.Source: §5.4 · pp. 11–12

6 CONCLUSION

DTAAD combines Transformer and Dual TCN components in an anomaly detection model that integrates autoregressive and autoencoder structures. It reports improved F1 scores on nine benchmark datasets and substantially lower training time than baseline methods.

6 CONCLUSION

  • DTAAD combines an autoregressive model and autoencoder with Transformer sequence modeling and local and global TCN components.The design also uses a feedback mechanism and loss ratio to improve detection accuracy and expand association differences.Source: §6 · p. 12
  • F1 improved by 8.38% on complete datasets and F1* by 8.97% on small datasets across nine benchmark datasets.The conclusion describes DTAAD as a promising solution for unsupervised anomaly detection on high-dimensional data.Source: §6 · p. 12
  • DTAAD is presented as suitable for varied devices, with up to 99% less training time than baseline methods.The authors identify industrial and embedded systems requiring accurate, fast anomaly prediction as target settings.Source: §6 · p. 12

APPENDIX A HYPER-PARAMETERS

The appendix reports optimization, thresholding, data-split, and training-stop settings, alongside captions identifying the diagnostic, timing, and ablation tables.

APPENDIX A HYPER-PARAMETERS

  • Training used AdamW with an initial learning rate of 0.01, weight decay of 10^-5, and a step scheduler with step size 5 and decay rate 0.9.The passage also specifies bias correction by default.Source: Appendix A · pp. 12–13
  • The appendix table captions identify comparisons of diagnostic performance, per-epoch training and inference time, and ablation F1* and AUC* scores.The captions describe Table 5, Table 6, and Table 7, respectively.Source: Table 5 · p. 13, Table 6 · p. 13, Table 7 · p. 13
  • The POT parameter was q = 10^-4 for all datasets; low quantiles were 0.07 for SMAP, 0.01 for MSL, and 0.001 for the others.The number of multi-head attention heads was set to the dataset size.Source: Appendix A · p. 13
  • Each series was split 80% for training and 20% for testing; the unsupervised training process used no labels and predicted only the next timestamp.Early stopping followed declining validation accuracy, with moving-average validation performance used to smooth fluctuations.Source: Appendix A · p. 13

APPENDIX B VISUALIZATION

The visualization analysis reports a window-size trade-off and contrasts global and local attention patterns.

APPENDIX B VISUALIZATION

  • A window size of 10 balances F1 scores and training time; windows that are too small or large can lose context or obscure local anomalies.With TCN_Global added, the final score rebounded as window size increased.Source: Appendix B · p. 13
  • The attention matrix visualization shows global attention distributed more evenly over the overall trend, while local attention focuses on anomalous regions.Source: Appendix B · p. 14

Related papers

Loading 2302.10753v3…