PaperlayerView exact revision on arXiv

Source-linked AI summary

FirstDiff: One-Step Diffusion-Based Anomaly Detection for Multivariate Time Series via Initial Noise Prediction

Ali Boudaghi, Alireza Nemati, Hadi Zare

arXiv:2608.15727v1cs.LGcs.AIstat.ML

TL;DR

Existing diffusion-based anomaly detectors typically wait for the full reverse process and final reconstruction, leaving intermediate denoising information underused. FirstDiff detects anomalies from the predicted noise at the initial reverse-diffusion step, achieving a 92.94% average F1-score across five benchmarks versus 87.82% for the previous best GAD model while reducing inference to one denoising evaluation.

  • Problem

    Existing diffusion-based anomaly detectors typically rely on final reconstruction, while the intermediate denoising process remains largely unexplored as an anomaly-information source.

  • Method

    FirstDiff models the predicted-noise distribution from normal validation data and scores test deviations after a single initial denoising-network evaluation.

  • Results

    92.94% average F1-score across five benchmarks, versus 87.82% for the previous best GAD model.

  • Takeaways & Limitations

    Initial predicted noise provides a strong, complementary anomaly signal while substantially reducing diffusion inference latency.

Abstract

from arXiv · show

Diffusion models have recently shown strong potential for multivariate time-series anomaly detection by learning the distribution of normal data through iterative denoising. Existing diffusion-based approaches, however, typically perform anomaly detection after completing the reverse diffusion process, relying primarily on the final reconstructed signal and overlooking informative representations produced during denoising. This design incurs substantial computational cost and limits the use of intermediate diffusion information for anomaly detection. In this paper, we propose FirstDiff, a diffusion-based anomaly detection framework based on the observation that the predicted diffusion noise at the initial reverse-diffusion evaluation already contains sufficient information for accurate anomaly detection. FirstDiff models the statistical distribution of predicted diffusion noise under normal behavior using validation data, enabling anomaly inference from a single denoising-network evaluation rather than completing the reverse diffusion trajectory. To model complex temporal and inter-sensor dependencies, FirstDiff employs a Diffusion Transformer as the denoising backbone. Extensive experiments on five public benchmark datasets demonstrate that FirstDiff achieves state-of-the-art performance while reducing diffusion inference from the full reverse trajectory to a single denoising-network evaluation.

I. INTRODUCTION … B. Deep Learning Methods

The paper motivates FirstDiff by the limitations of error-based anomaly scores and existing diffusion detectors, then presents one-step predicted-noise inference with statistical scoring and a Diffusion Transformer. Related work covers classical baselines and deep learning methods based on forecasting or reconstruction.

  • I. INTRODUCTION: Multivariate time-series anomaly detection supports reliable operation of cyber-physical systems, where high-dimensional temporal dynamics make manual monitoring infeasible.The application areas include industrial control, cloud infrastructure, power grids, healthcare monitoring, and financial platforms.
  • I. INTRODUCTION: Prediction- and reconstruction-based deep learning methods primarily derive anomaly scores from observable forecasting or reconstruction errors.Prediction methods estimate future observations, whereas reconstruction methods learn the distribution of normal data.
  • I. INTRODUCTION: Error-based detection can degrade when anomalies produce subtle reconstruction discrepancies or remain close to the learned normal-data manifold.This limitation motivates anomaly representations beyond conventional prediction or reconstruction errors.
  • I. INTRODUCTION: FirstDiff infers anomalies from predicted diffusion noise at the initial reverse-diffusion evaluation instead of completing the reverse trajectory.It models the statistical distribution of predicted noise using normal validation data and scores test samples by deviation from that reference.
  • I. INTRODUCTION: A Diffusion Transformer uses global self-attention to jointly model temporal and inter-sensor dependencies across timestamps and variables.This contrasts with CNNs focused primarily on local temporal dependencies and graph networks using predefined sensor structures.
  • I. INTRODUCTION: Predicted diffusion noise provides information complementary to reconstruction errors, motivating a hybrid inference strategy that jointly uses both representations.The paper also evaluates statistical scoring strategies including Mahalanobis distance.
  • A. Classical Methods: Classical anomaly-detection baselines include statistical, density-based, dimensionality-reduction, kernel, clustering, ensemble, signal-decomposition, and forecasting methods.Examples include ECOD, PCA, One-Class SVM, CBLOF, k-Means, Isolation Forest, wavelet transforms, and ARIMA.
  • B. Deep Learning Methods: Deep learning methods are categorized into forecasting-based and reconstruction-based approaches that learn temporal dynamics or normal-data distributions.Representative methods include LSTM-AD and MTAD-GAT for forecasting, plus LSTM-VAE, OmniAnomaly, InterFusion, and TranAD for reconstruction.

C. Diffusion-Based Anomaly Detection · III. OVERVIEW AND METHODOLOGY · A. Problem Formulation and Framework Overview

FirstDiff reframes diffusion-based multivariate time-series anomaly detection around the predicted noise from the first reverse-diffusion step rather than the final reconstruction. It operates on overlapping normal-data windows and uses a Diffusion Transformer to model temporal and cross-sensor dependencies.

  • C. Diffusion-Based Anomaly Detection: Diffusion-based methods typically learn normal-data distributions and detect anomalies from reconstruction errors after completing the reverse diffusion process.These models are motivated by stable optimization and strong reconstruction capability.
  • III. OVERVIEW AND METHODOLOGY: FirstDiff uses predicted diffusion noise at the first reverse-diffusion step, comparing it with a normal-validation reference distribution instead of completing the reverse trajectory.Statistical distance measures enable anomaly inference directly from the predicted noise.
  • C. Diffusion-Based Anomaly Detection: Existing work improves diffusion anomaly detection through masking, imputation, guided denoising, decomposition, graph attention, and conditioning strategies.Examples include ImDiffusion, DiffAD, D3R, Graph-Attention Diffusion, and ICDiffAD.
  • A. Problem Formulation and Framework Overview: The unsupervised setting provides only normal data for training and validation, while anomalies appear exclusively during testing.The framework therefore learns characteristics of normal behavior without anomalous training examples.
  • A. Problem Formulation and Framework Overview: Input multivariate time series are partitioned into overlapping windows of length W = 96, and the objective is timestamp-level anomaly-sequence estimation.Each window contains W timestamps and K sensor variables.
  • A. Problem Formulation and Framework Overview: Conventional inference iteratively denoises XT through T reverse-diffusion steps to obtain ˆX0 and derive an anomaly score.FirstDiff instead performs anomaly inference from ϵθ(XT, T) at the first step.
  • A. Problem Formulation and Framework Overview: FirstDiff employs a Diffusion Transformer denoising backbone whose global self-attention jointly models dependencies across all sensors and timestamps within each window.Given a noisy window and diffusion timestep, the DiT predicts injected Gaussian noise.

B. Diffusion-Based Denoising

FirstDiff uses DDPM to learn normal multivariate time-series windows by progressively adding Gaussian noise and training a denoising network to predict it. During inference, reverse diffusion removes predicted noise step by step, while conventional anomaly scoring compares the input with the fully reconstructed sample.

  • Forward diffusion and training: DDPM learns the distribution of normal multivariate time-series windows by progressively adding Gaussian noise and training a denoising network to predict the injected noise.The denoising objective is the standard DDPM objective, with the DiT network predicting Gaussian noise.
  • Forward diffusion and training: The forward process corrupts a clean window X0 ∈ RW ×K according to a predefined noise schedule over T diffusion steps, with noisy samples obtainable directly in closed form.The sampled noise follows ϵ ∼N(0, I).
  • Reverse diffusion: During inference, reverse diffusion progressively removes predicted noise through sequential steps, using z ∼N(0, I) for t > 1 and z = 0 at the final step.Each reverse step transforms the noisy sample toward the learned data distribution.
  • Reconstruction-based anomaly scoring: Conventional diffusion reconstruction methods score anomalies after the complete reverse process by measuring the discrepancy between input window X0 and reconstructed sample ˆX0.This discrepancy is commonly defined as a timestamp-level reconstruction score.

C. Predicted Noise Distribution Modeling

FirstDiff models normal predicted-noise behavior from the first reverse-diffusion step and detects anomalies without completing the remaining denoising trajectory. It approximates normal latent distributions with Gaussian reference models and scores deviations using complementary distance metrics.

  • First-step predicted-noise modeling: FirstDiff uses predicted diffusion noise from the first reverse-diffusion step as an anomaly indicator instead of waiting for the final reconstruction.Normal samples produce consistent denoising directions, whereas anomalous samples yield predicted-noise distributions that deviate from normal behavior.
  • First-step predicted-noise modeling: The predicted noise reflects the direction guiding noisy samples toward higher-probability regions of the learned normal distribution.This relationship follows from the connection between optimal noise prediction and the score function of the noisy data distribution.
  • Normal distribution estimation: Only normal validation windows are used to estimate predicted-noise and denoising-residual behavior at the first reverse-diffusion step.Gaussian noise is added through the forward diffusion process, and the trained Diffusion Transformer predicts the noise for each validation window.
  • Normal distribution estimation: FirstDiff approximates the empirical latent distributions with Gaussian reference models whose statistics are estimated from sample means and covariances.Predicted noise and residuals are reshaped into timestamp-level vectors in R^K and aggregated over the validation set.
  • Inference and scoring: During inference, anomaly scores are computed from the initial reverse-diffusion evaluation for either predicted noise or denoising residuals, without further denoising iterations.The investigated distances are Euclidean, cosine, and Mahalanobis, capturing magnitude, direction, and covariance-aware statistical deviation.

D. Hybrid Inference Strategy

FirstDiff evaluates hybrid anomaly detection by combining one-step predicted-noise signals with reconstruction-based scores or alternative distance measures. Binary detector outputs are fused with a parameter-free OR rule, classifying a timestamp as anomalous when any constituent detector identifies it.

  • Hybrid signal combinations: Hybrid inference combines one-step predicted-noise anomaly scores with conventional reconstruction-based scores or predicted-noise scores using Euclidean, cosine, and Mahalanobis distances.These combinations test whether anomaly signals from different representations or statistical measures provide complementary information.
  • Decision fusion: Binary anomaly decisions are combined using a simple OR rule, so a timestamp is anomalous if at least one detector identifies it.The parameter-free strategy enables evaluation of complementarity among different anomaly signals.
  • Evaluation: The effects of different hybrid combinations on detection performance are evaluated in the ablation study.The ablation study provides the complete evaluation of the hybrid combinations.

IV. EXPERIMENT … C. Baseline Methods

The experiment evaluates FirstDiff across five diverse multivariate time-series benchmarks using Precision, Recall, F1-score, and Average Detection Delay, and compares it with classical, deep-learning, and diffusion-based baselines. Baseline implementations follow official releases and paper recommendations where available, while methods without public implementations are excluded for reproducibility.

  • A. Datasets: FirstDiff is evaluated on SWaT, PSM, SMD, MSL, and SMAP, covering industrial control, cloud monitoring, and spacecraft telemetry.The benchmarks vary in dimensionality, sequence length, anomaly frequency, and system dynamics.
  • A. Datasets: The datasets span substantially different anomaly conditions, including SMD’s 4.16% anomaly ratio and PSM’s 27.76% anomaly ratio.The monitored-variable count ranges from 25 to 55 across different application domains.
  • B. Evaluation Metrics: Evaluation uses Precision, Recall, F1-score, and Average Detection Delay (ADD).ADD measures how quickly anomalies are detected after onset, with lower values indicating more timely detection.
  • B. Evaluation Metrics: Precision measures detected-anomaly reliability, Recall measures anomalous-event identification, and F1-score balances both.The metric definitions use true positives, false positives, and false negatives.
  • C. Baseline Methods: The comparison covers representative methods from classical machine learning, deep learning, and recent diffusion-based anomaly detection.This selection evaluates FirstDiff against conventional and state-of-the-art methodological paradigms.
  • C. Baseline Methods: Classical baselines are PCA, Isolation Forest (IForest), CBLOF, and ECOD, selected as widely adopted unsupervised methods.These methods are used for their simplicity, computational efficiency, and broad anomaly-detection performance.
  • C. Baseline Methods: Deep-learning baselines include GDN, MAD-GAN, OmniAnomaly, MSCRED, and TranAD, representing graph, adversarial, variational, convolutional, and transformer approaches.TranAD uses self-attention to model long-range temporal dependencies.
  • C. Baseline Methods: Diffusion-based comparisons include DiffAD, ImDiffusion, and Graph-Attention Diffusion (GAD), while DGM and MRDDM are excluded because public implementations are unavailable.Official implementations are used where available, and hyperparameters follow corresponding paper recommendations for fair comparison.

D. Implementation Details · V. RESULTS AND DISCUSSION · A. Overall Anomaly Detection Performance

FirstDiff standardizes and windows normal training data, then detects anomalies from predicted diffusion noise at the first reverse step. Across five benchmarks, it achieves a 92.94% average F1-score, exceeding prior methods while avoiding full reverse diffusion.

  • D. Implementation Details: All datasets are standardized with training-set statistics and segmented into overlapping windows of length W = 96; only normal training data are optimized.Training uses stride 64, while validation and testing use stride 96.
  • D. Implementation Details: The denoising network is a DiT with 6 Transformer blocks, hidden dimension 256, 8 attention heads, and T = 100 diffusion timesteps.It is trained for 15 epochs with Adam, batch size 128, and the standard noise-prediction objective.
  • D. Implementation Details: After training, validation noise estimates Gaussian statistics, and FirstDiff performs inference using predicted noise from the first reverse diffusion step.Reconstruction-based and hybrid variants are also evaluated to assess the one-step signal’s effectiveness and complementarity.
  • A. Overall Anomaly Detection Performance: 92.94% average F1-score makes FirstDiff the best method across five benchmarks, surpassing GAD’s 87.82% average F1-score.Table II compares precision, recall, and F1-score against competing anomaly detection methods.
  • A. Overall Anomaly Detection Performance: 96.66% F1-score on SWaT exceeds GAD’s 86.46%, while MSL reaches 93.01% F1-score, with 92.43% precision and 93.59% recall.FirstDiff is particularly strong on MSL, SWaT, PSM, and SMAP.
  • A. Overall Anomaly Detection Performance: 97.26% F1-score on PSM exceeds ImDiffusion’s 96.20% and DiffAD’s 95.71%; SMAP reaches 96.28%, while SMD records 81.48%.On SMAP, precision is 96.59% and recall is 95.98%; on SMD, FirstDiff is competitive with GAD’s 81.91% and ImDiffusion’s 81.77%.
  • A. Overall Anomaly Detection Performance: FirstDiff extracts effective anomaly-discrimination information after one reverse diffusion step, unlike ImDiffusion, GAD, and DiffAD, which use full denoising.The results indicate that completing the reverse trajectory is unnecessary for effective anomaly inference.

B. Detection Delay … A. Effect of One-Step Inference

FirstDiff’s first-step predicted-noise representation enables competitive or leading detection delays while substantially reducing inference time. Ablation results show that one-step variants can match or exceed full reconstruction on most datasets, although performance remains dataset-dependent.

  • B. Detection Delay: 26.17 and 110.14 are FirstDiff’s lowest ADD values among compared methods on PSM and SWaT, respectively.Lower ADD indicates earlier anomaly detection and better temporal responsiveness.
  • B. Detection Delay: 65.36 on MSL and 142.06 on SMAP are competitive with the strongest reported delays, while improving substantially over several baselines on SMAP.FirstDiff is close to ImDiffusion’s 62.59 on MSL, competitive with GAD’s 143.65 on SMAP, and below DiffAD’s 214.74 and TranAD’s 322.34.
  • B. Detection Delay: 35.90 on SMD is less favorable than CBLOF’s 24.05 and PCA’s 26.06, although the difference is described as modest.FirstDiff nevertheless maintains strong overall SMD detection performance as reflected by its F1-score.
  • C. Inference Time Analysis: 1.12–1.60 ms per window is required for one-step Mahalanobis inference, averaging approximately 1.23 ms, versus approximately 99–101 ms for full-step reconstruction.The full-step version represents an approximately 80× increase in inference time; measurements use synchronized GPU timing over 100 runs after 10 warm-ups.
  • C. Inference Time Analysis: Approximately 1.23 ms makes one-step FirstDiff faster than Imdiffusion at 203–208 ms, DiffAD at 264–273 ms, and GAD at 371–374 ms.The one-step formulation reduces computational cost by approximately two orders of magnitude relative to these diffusion-based reconstruction methods.
  • C. Inference Time Analysis: Approximately 99.74 ms for full-step FirstDiff is below Imdiffusion’s 204.55 ms, DiffAD’s 268.34 ms, and GAD’s 372.91 ms.These correspond to reductions of approximately 51.2%, 62.8%, and 73.2%, respectively.
  • A. Effect of One-Step Inference: 88.32% is the reconstruction-based variant’s average F1-score, while one-step variants span 78.61% to 92.94% across representations and statistical distances.The ablation evaluates one-step denoising, anomaly representation, statistical distance, and complementary anomaly-signal combinations.
  • A. Effect of One-Step Inference: 98.30%, 93.01%, 97.26%, and 96.65% are the strongest one-step F1-scores on PSM, MSL, SWaT, and SMAP, exceeding reconstruction results of 96.36%, 88.41%, 89.47%, and 76.85%.On SMD, reconstruction reaches 90.53% while the strongest one-step configuration reaches 83.52%, demonstrating dataset-dependent benefits.

B. Effect of Statistical Distance · C. Effect of Hybrid Anomaly Signals · VII. CONCLUSION

FirstDiff’s one-step predicted-noise signal supports effective anomaly detection, with Mahalanobis distance strongest individually and hybrid signals improving performance. The conclusion emphasizes complementary information, computational efficiency, and the Diffusion Transformer’s ability to model multivariate dependencies.

  • B. Effect of Statistical Distance: Cosine distance performs particularly well on PSM, reaching a 98.30% F1-score.
  • B. Effect of Statistical Distance: Mahalanobis distance achieves the strongest average individual performance, with a 92.94% F1-score, and becomes the primary FirstDiff configuration.It accounts for covariance structure in the predicted-noise distribution.
  • C. Effect of Hybrid Anomaly Signals: Reconstruction+ϵ improves average F1-score from 92.07% for ϵ alone to 93.18%.
  • C. Effect of Hybrid Anomaly Signals: Reconstruction+Mah(ϵ) achieves a 93.54% average F1-score and the highest MSL F1-score, reaching 94.20%.The results indicate that first-step predicted noise provides information complementary to final reconstruction.
  • C. Effect of Hybrid Anomaly Signals: Cos(ϵ)+Mah(ϵ) records the highest average F1-score among evaluated variants at 93.67% and the best SWaT result at 97.26%.Other distance combinations also improve average performance, supporting complementary statistical views of predicted noise.
  • C. Effect of Hybrid Anomaly Signals: FirstDiff’s main contribution remains one-step anomaly detection using predicted diffusion noise and its learned normal distribution, while hybrid configurations are extensions.
  • VII. CONCLUSION: The initial reverse-diffusion predicted noise provides a highly informative anomaly-detection representation across diverse benchmark datasets, with Mahalanobis distance particularly robust.
  • VII. CONCLUSION: FirstDiff remains computationally efficient with the complete diffusion trajectory, while its one-step formulation provides a substantially larger inference-latency reduction.The Diffusion Transformer captures complex temporal and inter-sensor dependencies in multivariate time series.

APPENDIX · A. Relationship Between Noise Prediction and the Score Function

The appendix derives the relationship between the optimal diffusion noise predictor and the score function used in Section III. It connects the result to the forward diffusion process, Fisher’s identity, and DDPM denoising training.

  • A. Relationship Between Noise Prediction and the Score Function: The appendix establishes a relationship between the diffusion noise predictor and the score function.The derivation specifically treats ϵθ(Xt, t) as the optimal diffusion noise predictor.
  • A. Relationship Between Noise Prediction and the Score Function: The derivation begins with the forward diffusion process and the conditional distribution of the noisy sample.The conditional distribution is then expressed in Gaussian form.
  • A. Relationship Between Noise Prediction and the Score Function: Taking the logarithm of the Gaussian conditional distribution and differentiating with respect to Xt produces its score expression.The derivation notes that C is independent of Xt before differentiation.
  • A. Relationship Between Noise Prediction and the Score Function: Using the forward diffusion equation transforms the conditional score into an expression involving the diffusion noise.This substitution is the step that connects the forward-process formulation to noise prediction.
  • A. Relationship Between Noise Prediction and the Score Function: The unconditional score equals the conditional score averaged over the clean sample given Xt, as stated through Fisher’s identity.The appendix explicitly gives ∇Xt log pt(Xt) = E [∇Xt log p(Xt|X0) | Xt].
  • A. Relationship Between Noise Prediction and the Score Function: DDPM trains the denoising network by minimizing a least-squares objective whose optimal solution is a conditional expectation.Replacing that conditional expectation with the optimal predictor yields the stated predictor–score relationship.
  • A. Relationship Between Noise Prediction and the Score Function: In practice, the trained denoising network approximates the optimal predictor, producing the relationship used throughout the work.The appendix presents this approximation as the practical form of the derived result.
Loading 2608.15727v1…