Source-linked AI summary
Time-Series Anomaly Detection Service at Microsoft
Hansheng Ren, Bixiong Xu, Yujing Wang, Chao Yi, Congrui Huang, Xiaoyu Kou, Tony Xing, Mao Yang, Jie Tong, Qi Zhang
TL;DR
Industrial time-series anomaly detection must monitor changing, diverse metrics at scale despite limited manual labels and strict latency requirements. The paper presents a Microsoft service built around a three-part pipeline and an unsupervised SR-CNN algorithm that combines Spectral Residual with CNN. It reports superior results against state-of-the-art baselines on public datasets and Microsoft production data, including more than 20% F1-score improvement on production data.
Problem
Industrial anomaly detection requires accurate, efficient, and general monitoring across changing, diverse time-series without relying on extensive manual labels.
Method
The paper combines a three-component monitoring pipeline with an unsupervised method that applies CNN to Spectral Residual output using synthetic anomalies.
Results
More than 20% F1-score improvement is reported on Microsoft production data, with superior results against state-of-the-art baselines on public datasets and production data.
Takeaways & Limitations
The service supports practical online monitoring, and its SR-CNN approach is reported to remain unsupervised while achieving strong anomaly-detection performance.
Abstract
from arXiv · showhide
Large companies need to monitor various metrics (for example, Page Views and Revenue) of their applications and services in real time. At Microsoft, we develop a time-series anomaly detection service which helps customers to monitor the time-series continuously and alert for potential incidents on time. In this paper, we introduce the pipeline and algorithm of our anomaly detection service, which is designed to be accurate, efficient and general. The pipeline consists of three major modules, including data ingestion, experimentation platform and online compute. To tackle the problem of time-series anomaly detection, we propose a novel algorithm based on Spectral Residual (SR) and Convolutional Neural Network (CNN). Our work is the first attempt to borrow the SR model from visual saliency detection domain to time-series anomaly detection. Moreover, we innovatively combine SR and CNN together to improve the performance of SR model. Our approach achieves superior experimental results compared with state-of-the-art baselines on both public datasets and Microsoft production data.
1 INTRODUCTION
Industrial time-series anomaly detection must operate accurately, efficiently, and generally despite scarce labels, changing distributions, diverse patterns, and near-real-time scale. The paper introduces an unsupervised SR-CNN approach that borrows Spectral Residual from visual saliency detection and reports strong results on public and Microsoft production data.
- Challenges: Industrial services must detect anomalies across millions of changing time-series, where manual labeling is impractical and supervised models are insufficient.The data distribution changes over time, so similar anomalous patterns may not have appeared previously.
- Challenges: Existing methods generalize poorly across diverse time-series patterns, with Holt-Winters and Spot each performing poorly on different pattern classes.The paper therefore treats generality across pattern types as a central service requirement.
- Challenges: Near-real-time monitoring of millions or billions of series makes computational efficiency a prerequisite for online anomaly detection.High-complexity models may be accurate but unsuitable for online scenarios.
- Approach: The paper borrows Spectral Residual from visual saliency detection because anomalies in time-series are treated as salient parts of curves.This is presented as the first attempt to apply the idea to time-series anomaly detection.
- Approach: SR-CNN applies CNN to SR output and trains it with fully synthetic anomalies, preserving an unsupervised setting without manually labeled data.The SR output makes anomaly discrimination easier for CNN training.
- Results: More than 20% F1-score improvement is reported on Microsoft production data, while the approach is described as more accurate and general than state-of-the-art unsupervised models.The paper also reports best-ever F1-scores on the open datasets for both unsupervised and supervised approaches as of submission.
2 SYSTEM OVERVIEW
The Microsoft service combines data ingestion, experimentation, and online computation to support continuous time-series monitoring. It ingests data at configured granularities, evaluates models offline and online, and processes incoming points with sliding-window computation and alert correlation.
- System Overview: The system has three major components: data ingestion, experimentation platform, and online compute.These components organize registration, evaluation, and production processing within the service.
- Data Ingestion: Datafeeds connect to user storage through a Connect String and update at a configured granularity, with one minute as the minimum.Supported data sources include Azure storage, databases, and online streaming data.
- Data Ingestion: The ingestion module stores incoming time-series points in InfluxDB and Kafka, with throughput ranging from 10,000 to 100,000 data points per second.Ingestion tasks create new data points according to each datafeed’s configured granularity.
- Online Compute: Online compute processes each point immediately, using Flink to manage sliding-window data in memory for efficient anomaly detection.The maximum production throughput is reported as 4 million time-series per minute.
- Experimentation Platform: The experimentation platform supports offline experiments, online A/B tests, labeling, and evaluation of accuracy, efficiency, and generality before deployment.Online deployment decisions use metrics including alert click-through rate, anomaly percentage, and false anomaly rate.
3 APPLICATIONS
Microsoft’s anomaly detection service supports continuous monitoring across large-scale business applications, with Bing and Outlook teams using alerts and incident insights to investigate issues and broaden coverage.
- Service adoption: More than 200 Microsoft product teams continuously monitor over 4 million time-series through the service.The service spans Office 365, Windows, Bing, and Azure organizations.
- Bing application: Bing users ingest thousands of market-, device-, and channel-specific time-series to monitor global service usage.A Bing team member created a datafeed covering markets such as the US and UK, devices, and channels.
- Bing application: Bing incident reports select correlated anomalous time-series, helping teams connect UK usage anomalies with a relevance issue that triggered excess pagination requests.The report surfaced PC-device and PORE-channel usage anomalies for investigation.
- Outlook application: The Outlook anti-spam team integrated the service into Office DevOps to monitor spam-model effectiveness across geolocations.Compared with its original rule-based solution, the team covered more geolocations and received fewer false-positive cases.
4 METHODOLOGY
The methodology combines Spectral Residual processing with CNN-based discrimination to detect anomalies without relying on manually labeled data. SR transforms time-series into saliency maps, while forecasting and synthetic training support low-latency, adaptive detection.
- Time-series anomaly detection produces a binary output sequence indicating whether each input value is anomalous.
- SR-CNN: The proposed SR-CNN method applies CNN to SR outputs, replacing SR’s single threshold with a learned discriminative rule.Learning on saliency maps is easier than learning directly from raw input sequences.
- SR: SR computes a Fourier-domain log amplitude spectrum, subtracts its averaged spectrum to obtain the spectral residual, and applies an inverse transform to produce a saliency map.The saliency map emphasizes innovation points in the original sequence.
- SR: SR detects anomalies by comparing each saliency value with a local average using threshold τ.The local average is computed from preceding saliency-map points.
- SR: To reduce detection latency, SR uses a sliding window and appends estimated points after the latest observation so that the target lies nearer the window center.The next point is estimated from the average gradient of preceding points, with m = 5 in the implementation.
- SR-CNN: SR-CNN trains its CNN detector on production time-series containing synthetic anomalies, avoiding manual labels while adapting to changing time-series distributions.The experiments use 65 million points for training.
5 EXPERIMENTS
The experiments evaluate anomaly detection across public and Microsoft datasets using accuracy, efficiency, and generality measures. SR and SR-CNN outperform unsupervised baselines, while SR features also improve supervised detection.
- Datasets: The evaluation uses KPI, Yahoo, and Microsoft datasets spanning different intervals and time-series patterns, with labeled anomaly and normal points.Microsoft data covers revenues, active users, and pageviews; Yahoo includes synthetic and real-traffic series.
- Metrics: Accuracy is measured with precision, recall, and F1-score using segment-adjusted evaluation that tolerates limited detection delay.A contiguous anomaly segment is counted as correctly detected when at least one point is identified within the allowed delay.
- SR/SR-CNN Experiment: 36.1%, 68.8%, and 21.2% F1-score improvements are reported on KPI, Yahoo, and Microsoft, respectively, over the best cold-start baselines.These comparisons cover FFT, Twitter-AD, and Luminol in the cold-start setting.
- SR/SR-CNN Experiment: 48.0%, 92.9%, and 57.0% F1-score improvements are reported on KPI, Yahoo, and Microsoft, respectively, over the best trained unsupervised results.SR-CNN further improves accuracy over SR on all three datasets.
- Efficiency and Generality: SR is the most efficient and stable method, while SR-CNN provides better accuracy with a reasonable latency increase and good generalization across Yahoo patterns.Generality is assessed across seasonal, stable, and unstable Yahoo series.
- SR+DNN: Adding SR features improves the vanilla DNN F1-score by 1.6% and establishes a new state-of-the-art result on KPI.The SR-powered DNN also outperforms vanilla DNN across various thresholds in the precision-recall curve.
6 RELATED WORKS
Prior time-series anomaly detection methods include statistical, supervised, and unsupervised approaches. This work is inspired by Spectral Residual visual-saliency models and adapts that perspective to time-series anomalies.
- Statistical Approaches: Statistical approaches include hypothesis testing, wavelet analysis, SVD, ARIMA, and FFT-based detection.These methods represent earlier approaches to time-series processing and anomaly detection.
- Supervised Approaches: Supervised methods improve anomaly detection accuracy by combining statistical detector outputs with classifiers or collections of forecasting and detection models.Examples include Opprentice and Yahoo EGADS.
- Unsupervised Approaches: Unsupervised approaches such as DONUT and Luminol use reconstruction probabilities or time-series segmentation to identify anomalies.DONUT is based on a variational auto-encoder, while Luminol builds on prior segmentation methods.
- Visual Saliency Inspiration: Spectral Residual was originally developed for visual saliency detection by separating redundant and innovative image components.The paper uses this visual-saliency inspiration for time-series anomaly detection.
7 CONCLUSION & FUTURE WORK
The paper presents a Microsoft anomaly detection service combining operational scale with SR-based algorithms. Future work includes ensembling methods and offering the service through Azure.
- Conclusion: The service supports more than 200 Microsoft teams and detects anomalies from up to 4 million time-series per minute in production.Examples of supported teams include Bing, Office, and Azure.
- Conclusion: The paper applies Spectral Residual to time-series anomaly detection and combines SR with CNN to achieve outstanding performance.This is presented as the first application of SR to this task.
- Future Work: Future work plans to ensemble state-of-the-art methods to provide more robust anomaly detection service to customers.The service is also planned for publication on Microsoft Azure as part of Cognitive Service for external customers.