Source-linked AI summary
Argos: Agentic Time-Series Anomaly Detection with Autonomous Rule Generation via Large Language Models
Yile Gu, Yifan Xiong, Jonathan Mace, Yuting Jiang, Yigong Hu, Baris Kasikci, Peng Cheng
TL;DR
Cloud anomaly-detection systems often do not jointly provide explainability, reproducibility, and autonomy. ARGOS uses an agentic pipeline to generate and deploy anomaly rules, and it outperforms baselines across public and internal datasets.
Problem
Existing anomaly-detection approaches do not simultaneously provide explainability, reproducibility, and autonomy, despite their importance for cloud-service reliability.
Source: §1 · pp. 1–2Method
ARGOS uses collaborating agents to generate, refine, and select LLM-produced anomaly rules during training, then combines their runtime predictions with those of an existing detector.
Source: §1 · p. 2Results
ARGOS outperformed all baselines across KPI, Yahoo, and Internal; on Internal, its F1 score was 28.3% higher than the best baseline, LSTMAD.
Source: §5.5 · p. 11Takeaways & Limitations
ARGOS's training-time rule generation and runtime rule deployment target explainable and reproducible detection while retaining autonomy.
Source: §1 · p. 2, §7 · p. 13On KPI, the LLM takes an average of 113 minutes to converge to an accurate rule.
Source: §2.3 · p. 5
Abstract
from arXiv · showhide
Observability in cloud infrastructure is critical for service providers, driving the widespread adoption of anomaly detection systems for monitoring metrics. However, existing systems often struggle to simultaneously achieve explainability, reproducibility, and autonomy, which are three indispensable properties for production use. We introduce Argos, an agentic system for detecting time-series anomalies in cloud infrastructure by leveraging large language models (LLMs). Argos proposes to use explainable and reproducible anomaly rules as intermediate representation and employs LLMs to autonomously generate such rules. The system will efficiently train error-free and accuracy-guaranteed anomaly rules through multiple collaborative agents and deploy the trained rules for low-cost online anomaly detection. Through evaluation results, we demonstrate that Argos outperforms state-of-the-art methods, increasing $F_1$ scores by up to $9.5\%$ and $28.3\%$ on public anomaly detection datasets and an internal dataset collected from Microsoft, respectively.
1 Introduction
Cloud-service interruptions make timely metric anomaly detection important, yet existing approaches do not jointly provide explainability, reproducibility, and autonomy. ARGOS uses LLM-generated rules and an agent-based pipeline to address this gap, and its evaluation reports higher F1 scores than the best baselines on public and internal datasets.
1 Introduction
- Cloud-service interruptions can harm customer experience and business operations, making early detection from monitored metrics important.The introduction describes an AWS outage affecting millions of users for more than 10 hours and a GPU-training hang that manual rules detected only after human intervention.Source: §1 · p. 1
- Existing approaches fall short of jointly achieving explainability, reproducibility, and autonomy: deep-learning methods often lack explainability, runtime LLM methods reproducibility, and rule-based methods autonomy.Rule-based methods rely on manual rule generation and threshold tuning, while runtime LLM methods can produce inconsistent results for the same input.Source: §1 · p. 2
- ARGOS uses LLMs in training to generate explainable, reproducible rules that are deployed for runtime detection, rather than relying on LLMs to predict anomalies at runtime.The paper presents structured detection rules as an intermediate representation for combining LLM autonomy with rule-based explainability and reproducibility.Source: §1 · p. 2
- LLM-generated rules can contain syntax errors or misread data patterns, may not outperform mature production detectors, and can require costly repeated trials because of LLM randomness.ARGOS addresses these challenges with iterative agent feedback, aggregation with existing detectors, and parallel candidate generation followed by top-k selection.Source: §1 · p. 2
- ARGOS uses multiple collaborating agents to propose, validate, fix, and refine rules, aggregates rule predictions with an existing detector, and selects the best k of n candidates for refinement.During training, it primarily learns from samples an existing detector identified incorrectly; at runtime, an aggregation algorithm combines predictions.Source: §1 · p. 2
- 9.5% and 4.8% higher average F1 scores on KPI and Yahoo, respectively, and 28.3% higher F1 on the internal dataset than the best baselines.ARGOS also speeds inference by 3.0×, 34.3×, and 1.5× on KPI, Yahoo, and Internal, respectively.Source: §1 · pp. 2–3
2 Motivation and Challenges
Existing anomaly-detection approaches face trade-offs in explainability, reproducibility, and autonomy. ARGOS motivates LLM-generated executable rules, while identifying correctness, accuracy, and training efficiency as challenges.
2.1 Why Existing Methods Fall Short?
- Manual rules were metric-sensitive: they matched FCVAE on da403 but underperformed it on 1c35d across thresholds.Different anomaly patterns make a single rule ineffective across metrics, and adapting rules requires expertise and time.Source: §2.1 · p. 3, Table 1 · p. 3
- FCVAE F1 varied substantially across latent-dimension and window-size settings, with no clear intuition for some performance differences.Window sizes 100 and 140 performed significantly better than 120 on da403.Source: §2.1 · p. 4, Figure 4 · p. 4
- LLM-generated rules can be explainable and reproducible, but LLMs used directly at deployment can produce inconsistent predictions across trials.For da403, no two of ten SigLLM trials produced the same predictions, and majority voting detected none of the three ground-truth anomalies.Source: §2.2 · p. 4, §2.1 · p. 4
2.2 How LLMs can Help in Rule Generation?
- ARGOS proposes using LLMs offline to generate executable anomaly rules that can be deployed without involving LLMs in real-time detection.Structured rules are presented as an intermediate representation for explainability and reproducibility.Source: §2.2 · p. 4, Figure 5 · p. 4
2.3 Opportunities and Challenges in Applying LLMs
- LLM-generated rules achieved similar or better F1 than manual rules on two example metrics, including rules for spikes and level shifts on 1c35d.The da403 rule flags points with z-scores greater than 3.Source: §2.3 · p. 5
- LLM-generated rules on KPI had a 4.8% syntax-error rate and average F1 of 0.129, versus 0.819 for the best deep learning method.The comparison is for rules generated 50 times per metric; the F1 average covers correct rules across metrics.Source: §2.3 · p. 5
- LLM rule generation also faces an accuracy guarantee gap: the best generated rules regressed against the best existing model on 3 of 19 KPI metrics.Stochastic outputs can require repeated trials; convergence to an accurate KPI rule took an average of 113 minutes.Source: §2.3 · p. 5
3 System Design
ARGOS preprocesses time-series data, trains rules through an agentic feedback pipeline, and deploys them alongside an existing detector. Its design adds rule repair, validation-based review, model fusion, and top-k selection.
3.1 Overview
- ARGOS uses three stages: data preprocessing, agentic rule training, and deployment, where an Aggregator combines rules with an existing base detector.The deployed detector processes runtime data chunks and reports detected anomalies for incident response.Source: §3.1 · pp. 5–6
3.2 Autonomous Rules Generation
- The Detection Agent writes commented Python rules that map an input sample to prediction labels and describe both abnormal rules and normal behavior.The comments and code are intended to make the rules understandable and verifiable by engineers.Source: §3.2 · p. 6, Figure 7 · p. 6
3.3 Correctness and Accuracy Improvement via Feedback Loops
- Repair and Review agents iteratively correct syntax errors and check validation accuracy, with the Review Agent preventing regression below the previous iteration.Syntax checks execute rules on dummy data; accuracy review can revert changes to restore the previous level.Source: §3.3 · p. 6
3.4 Accuracy Guarantee via Model Fusion
- ARGOS trains separate rule sets on a base detector’s false negatives and false positives, then aggregates their predictions to correct the corresponding labels.The false-negative rules can change base normal labels to abnormal, while false-positive rules can change base abnormal labels to normal.Source: §3.4 · pp. 6–7
3.5 Efficiency Enhancement
- ARGOS generates n candidate rules per iteration and selects the top k by validation accuracy after review verifies no accuracy regression.The selector can be extended to consider inference time as an additional criterion.Source: §3.5 · p. 7
4 Implementation
ARGOS preprocesses time series for LLM input and selects data partitions for rule training. Its selection strategy varies with metric continuity and dataset mode.
4.1 Data Preprocessing
- ARGOS chunks continuous time series and applies scaling, indexing, and tokenizer-specific preprocessing before training.Preprocessing adapts data to LLM context and numerical-tokenization behavior.Source: §4.1 · pp. 7–8
- Scaling limits numeric precision and indexing replaces timestamps with point order, under the assumption that samples arrive at fixed intervals.The scaling and re-indexing address context-window limits and uninformative long timestamps.Source: §4.1 · p. 7
- ARGOS calibrates chunk size on a dataset metric, choosing the size with the highest training-set F1 among Detection Agent proposals.Too-small chunks may lack context; too-large chunks may exceed the LLM context window or produce overly general rules.Source: §4.1 · p. 7
4.2 Data Selection
- For metrics with at least 10 chunks, ARGOS uses one-for-one training; otherwise it uses one-for-all, which can generalize across metrics.Users may also specify the dataset mode directly.Source: §4.2 · p. 8
- For one-for-all training, ARGOS retrieves contrastive examples by Euclidean distance to the false-example mean and standard deviation; embedding-based retrieval is left for future work.One-for-one mode retrieves contrastive examples randomly because its samples share a distribution.Source: §4.2 · p. 8
5 Evaluation
The evaluation tests ARGOS on public and internal datasets, measuring rule quality, accuracy guarantees, training efficiency, end-to-end performance, cost, latency, and a case study. ARGOS outperforms the baselines across all three datasets, while its agent feedback and Aggregator address rule errors and accuracy regressions.
5.1 Experiment Setup
- ARGOS is evaluated with Event-F1 PA against deep-learning and LLM baselines on KPI, Yahoo, and Internal datasets.KPI evaluation uses 19 metrics; Yahoo contains 367 time series, and Internal contains 20 hardware metrics. The study uses one-for-one training on KPI and one-for-all training on Yahoo and Internal.Source: §5.1 · p. 9
5.2 Correctness and Accuracy Improvement
- 3.8×, 11.3×, and 1.2× average test F1 improvements with the Review Agent on KPI, Yahoo, and Internal; the Repair Agent eliminates syntax errors.With only the Detection Agent, syntax-free rule rates were 95.2%, 93.5%, and 93.3% on those datasets, respectively.Source: Figure 8 · p. 9, §5.2 · p. 10
5.3 Accuracy Guarantee
- ARGOS with the Aggregator has no accuracy regressions across any metrics in all three datasets.Without the Aggregator, ARGOS regresses on 3 KPI metrics and 2 Yahoo metrics, with accuracy drops up to 32%.Source: §5.3 · p. 10, Table 3 · p. 10
5.4 Efficiency Enhancement
- At iteration 8, top-k training F1 is 0.736 on KPI, 0.597 on Yahoo, and 0.870 on Internal, versus 0.682, 0.523, and 0.795 at iteration 40 without selection.Converged F1 scores are also consistently higher with top-k selection across all datasets.Source: §5.4 · p. 10, Figure 9 · p. 11
5.5 End-to-End Results
- 9.5%, 4.8%, and 28.3% higher F1 than the best baselines on KPI, Yahoo, and Internal, respectively.ARGOS’s F1 scores are 0.897 on KPI, 0.810 on Yahoo, and 0.936 on Internal.Source: Table 4 · p. 11, §5.5 · p. 11
5.6 Cost Analysis
- Training the rules costs under $10 total across all datasets for 50 iterations.Per iteration, input-token costs reach up to $0.177 and output-token costs up to $0.017.Source: Table 5 · p. 11, §5.6 · p. 11
- ARGOS inference is 3.0× faster on KPI, 34.3× on Yahoo, and 1.5× on Internal than the baseline with the highest average F1 on each dataset.The rules are implemented in Python and described as lightweight.Source: Table 6 · p. 12, §5.6 · p. 12
5.7 Case Study
- On KPI metric e0770, ARGOS raises recall from 0.33 to 0.67 by detecting a missed prolonged extreme-value anomaly.A false negative remains around timestamp 12,000; the authors suggest the pattern may be absent from training or the spike threshold may need tuning.Source: Figure 11 · p. 12, Figure 10 · p. 12, §5.7 · p. 12
6 Related Works
Related work spans deep-learning, rule-based, and LLM-based anomaly detection, alongside LLM applications in cloud reliability. The paper positions ARGOS as the first to use LLMs to generate explainable and reproducible time-series detection rules.
Time-Series Anomaly Detection
- Deep-learning methods predict or reconstruct time series, while rule-based methods are prevalent in industry for their explainability [49] [59] [63] [67] [68] [70] [75].Resin reports anomalies when memory usage exceeds the previous period’s mean by three standard deviations, while FBDetect uses a user-set deviation threshold.Source: §6 · pp. 12–13
LLMs for Anomaly Detection
- LLM anomaly-detection work covers log classification [20] [29] and time-series methods that fine-tune or prompt LLMs [4] [13] [35]; ARGOS instead generates explainable, reproducible rules.The authors describe their approach as the first to use LLMs to generate such time-series detection rules.Source: §6 · p. 13
LLMs for Cloud Reliability
- LLMs have also been applied to cloud reliability, including incident root-cause analysis systems RCACopilot [8] and Atlas [65].Other cited work uses historical incidents as in-context examples for root-cause analysis [73].Source: §6 · p. 13
7 Conclusion
ARGOS targets production anomaly detection’s need for explainability, reproducibility, and autonomy with LLM-generated rules and an agent-based pipeline. Its evaluation on public and internal datasets reports better performance than state-of-the-art methods.
7 Conclusion
- ARGOS autonomously generates anomaly detection rules with LLMs, and its iterative agents propose, repair, and review rules.The conclusion presents the rules as explainable and reproducible.Source: §7 · p. 13
- ARGOS incorporates model fusion for accuracy guarantees and top-k rule selection to enhance efficiency.Source: §7 · p. 13
- On public and internal datasets, ARGOS outperforms state-of-the-art time-series anomaly detection methods, with up to a 28.3% F1 improvement.Source: §7 · p. 13
A Prompts
The appendix documents prompts for ARGOS’s Detection, Repair, and Review Agents, including their task instructions and feedback inputs.
A.1 Detection Agent
- The Detection Agent prompt begins with a task summary, then specifies data format, function invariants, and empirically useful notes.Source: §A.1 · p. 18
- The Detection Agent is prompted to write rules that classify cloud-service time-series data as abnormal or normal.Source: §A.1 · p. 18
A.2 Repair Agent
- The Repair Agent receives incorrect code and error messages when syntax or runtime errors occur, then revises the rules until syntax errors are fixed.Source: §A.2 · p. 19
- Its prompt frames the task as fixing Python code using the accompanying error details.Source: §A.2 · p. 19
A.3 Review Agent
- When accuracy regresses, the Review Agent receives current code, code differences, performance metrics, and examples misclassified by the current rules.Source: §A.3 · p. 20
- The Review Agent is prompted to modify rules that describe abnormal samples and exclude normal ones, used alongside an anomaly detection model.Source: §A.3 · p. 20