Revisiting VAE for Unsupervised Time Series Anomaly Detection: A Frequency Perspective

1 INTRODUCTION

• Web systems use time-series monitoring to detect anomalies promptly and initiate diagnosis and remediation; rare anomalies make labeling difficult.Online advertising systems are one example of web systems monitored with time-series data.Source: §1 · p. 1
• Existing VAE methods disregard anomalies during reconstruction but fail on some normal points, including heterogeneous periodic shapes and detailed within-window trends.Timestamp-conditioned CVAE is also unsatisfactory for heterogeneous periods; missing frequency components hinder reconstruction of detailed patterns.Source: §1 · p. 2
• FCVAE conditions a CVAE on global and local frequency features, while target attention selects useful sub-window frequencies to reduce noisy conditioning.Directly conditioning on all frequencies from a window yields numerous sub-frequencies that add noise and obstruct reconstruction.Source: §1 · pp. 2–3
• ∼40% on public datasets and 10% in a real-world web system in terms of F1 score: FCVAE substantially surpasses state-of-the-art methods.The authors also report that VAE-based methods can inherently handle mixed anomaly-normal training data, unlike prediction-based methods.Source: §1 · p. 3

2.1 Problem Statement

• Given a univariate time series and its label series, the task is to use preceding observations to predict each point’s anomaly label.The series values are real-valued, and labels are binary.Source: §2.1 · p. 3

2.2 VAEs and CVAEs

• VAEs maximize ELBO for reconstruction and generation; CVAEs condition the objective, while M-ELBO weakens anomalous or missing data’s effect on reconstruction.DONUT proposed M-ELBO, and CVAE combines conditional generative models with VAE.Source: §2.2 · p. 3

3.1 Framework Overview

• The framework comprises data preprocessing, training, and testing stages.Source: §3.1 · p. 3, Figure 3 · p. 3

3.2 Data Preprocessing

• Preprocessing standardizes data and fills missing and anomalous points, then augments abnormal data with pattern mutations and random value mutations.Source: §3.2 · pp. 3–4

3.3 Network Architecture

• FCVAE’s architecture combines an encoder, decoder, and condition extractor containing global and local frequency modules.Source: §3.3 · p. 4

3.3.1 GFM.

• GFM transforms the full window with FFT, then uses a linear layer to filter useful frequency information and dropout to learn missing frequency information.Source: §3.3.1 · p. 4

3.3.2 LFM.

• LFM applies FFT to sliding small windows and uses the latest as the query, with earlier windows as keys and values for target attention.Source: §3.3.2 · pp. 4–5

3.4 Training and Testing

• Training uses CVAE-based M-ELBO, missing-data injection, and masking the last point during frequency-condition extraction; testing imputes that point with MCMC and uses reconstruction probabilities as anomaly scores.Source: §3.4 · p. 5

4.1.1 Datasets.

• Experiments cover Yahoo, KPI, WSD, and NAB, spanning open benchmarks and KPIs collected from Internet companies.WSD contains real-world KPIs from three Internet companies, while KPI covers five companies.Source: §4.1.1 · p. 5

4.1.2 Baseline Methods.

• FCVAE is compared with statistical, supervised, unsupervised reconstruction-based, prediction-based, and transformer anomaly-detection methods.The baselines include SPOT, SRCNN, TFAD, DONUT, Informer, Anomaly-Transformer, AnoTransfer, and VQRAE.Source: §4.1.2 · p. 5

4.1.3 Evaluation Metrics.

• Best F1 uses threshold search with point adjustment, while delay F1 uses delay point adjustment to evaluate detection of anomalous segments.The delay is 7 for most datasets, 3 for Yahoo, and 150 for NAB; Figure 8 illustrates the adjustment strategy.Source: §4.1.3 · p. 5, Figure 8 · p. 5, Table 1 · p. 6

4.1.4 Implementation Details.

• All experiments are unsupervised, use one model per dataset, and select dataset-specific hyperparameters through grid search.The experiments use no actual labels, and parameter sensitivity is evaluated afterward.Source: §4.1.4 · p. 6

4.2 Overall Performance

• FCVAE surpassed all baselines on four datasets by 6.45%, 0.98%, 14.14%, and 0.31% in best F1, and by 4.98%, 1.58%, 38.68%, and 0.65% in delay F1.The reported comparisons are summarized in Table 1.Source: §4.2 · p. 6
• Baseline performance varies: Informer handles many value-jump anomalies but struggles with frequency-change anomalies, while Anomaly-Transformer has low delay F1 despite commendable best F1.TFAD also achieves favorable results on various datasets but exhibits detection delay.Source: §4.2 · p. 6
• FCVAE outperforms the VAE-based reconstruction baselines DONUT and VQRAE, including VQRAE’s RNN-based temporal modeling modifications.The authors infer that UTS anomaly detection should incorporate key information without overloading the model.Source: §4.2 · p. 6

4.3 Different Types of Conditions in CVAE

• Frequency information as a CVAE condition outperforms timestamp and time-domain conditions in the reported comparison.The authors describe frequency as a complementary prior, whereas timestamps are limited and time-domain information may be redundant.Source: §4.3 · p. 6

4.4 Frequency VAE and FACVAE

• FCVAE surpasses FVAE, which appends frequency information to the VAE input, with the authors attributing the difference partly to FVAE’s inefficient frequency use in the decoder.The authors identify frequency-conditioned CVAE as the most effective structure known to date.Source: §4.4 · p. 6

4.5 GFM and LFM

• Across datasets, using both GFM and LFM together yields superior performance, while either module alone outperforms VAE except on NAB, where frequent oscillation creates inconsistency.The reported comparison is shown in Figure 9; the authors conclude both global and local frequency information play a role in anomaly detection.Source: §4.5 · pp. 6–7, Figure 9 · p. 7

4.6 Attention Mechanism

• Without attention, LFM does not attain FCVAE’s original performance; attention assigns higher weights to more informative windows.The alternatives use the latest small window or average pooling across all small windows.Source: §4.6 · p. 7
• In the illustrated case, window 8 is the query, and window 5 has both the highest spectral similarity to it and the highest heat value.Figure 10 depicts the attention mechanism in LFM.Source: §4.6 · p. 7, Figure 10 · p. 7

4.7 Key Techniques in Framework

• CM-ELBO plays the most crucial role on most datasets, consistent with its tolerance for abnormal or missing data.The comparison reports Delay F1 across settings.Source: §4.7 · p. 7, Table 2 · p. 7
• Masking the last window point improves detection when an anomaly there would affect the window’s frequency information.The evaluation examines this technique across four datasets.Source: §4.7 · p. 7, Table 2 · p. 7
• Data augmentation introduces artificial anomalies to boost CM-ELBO performance, particularly in unsupervised settings.Source: §4.7 · p. 7, Table 2 · p. 7

4.8 Parameter Sensitivity

• FCVAE achieves stable, strong results under different settings for condition dimension, window size, missing-data injection, and data augmentation on KPI and WSD.The sensitivity results are shown as Delay F1 in Figure 11.Source: §4.8 · pp. 7–8, Figure 11 · p. 8

• FCVAE was incorporated into a cloud system serving millions of users globally and generating billions of time-series points each day.The system uses it to detect potential regressions that may indicate incidents.Source: §5 · p. 8
• FCVAE substantially improved both Best F1 and Delay F1 over the legacy detector during one year of production use.Table 3 reports the online comparison; the supplied passages provide no cell values.Source: §5 · p. 8, Table 3 · p. 8
• The model processed over 1000 data points per second, faster than the system generated new temporal points.The reported experiment used a 24GB memory 3090 GPU.Source: §5 · p. 8

• Statistical methods include FFT-based detection of high-frequency abnormalities, STL-based Twitter detection, and SPOT’s use of Extreme Value Theory.Source: §6 · p. 8
• Supervised approaches learn anomaly features for classification; examples include Opprentice’s random-forest combination of detectors and SRCNN’s spectral-residual and CNN classifier [38].Source: §6 · p. 8
• Unsupervised methods include reconstruction-based models that score reconstruction error and prediction-based models that score prediction error.DONUT modifies ELBO for VAE reconstruction [50], while Informer changes self-attention for prediction [59].Source: §6 · p. 8
• Prior work has addressed practical problems through frequency-domain methods [48] [60], while other work uses augmentation for pseudo-labels [3, 53] or transfer learning [12, 27, 54, 55].Source: §6 · p. 8

• FCVAE conditions a CVAE on frequency information, using GFM and LFM to capture global and local features concurrently and target attention to extract local information.Source: §7 · p. 8
• The architecture adds CM-ELBO, data augmentation, and masking the last point.Source: §7 · p. 8
• The paper evaluates accuracy on four datasets and an online cloud system, with ablations to assess each module.Source: §7 · p. 8