PaperlayerView exact revision on arXiv

Source-linked AI summary

Have an LLM Write Your Anomaly Detector: Autonomous Discovery of Compact, Interpretable Detectors for Time Series

David Berghaus

arXiv:2610.01223v1cs.LGcs.AI

TL;DR

Time-series anomaly detection must balance accuracy, computational efficiency, and interpretability, with no single method dominating across datasets and budgets. This paper has an LLM iteratively edit a short detector program under a leakage-free objective, then evaluates the best-scoring program. Its discovered detectors lead the reported TSB-AD metrics while remaining compact, interpretable, and free of neural training or GPU use.

  • Problem

    Time-series anomaly detection has no clear winner across datasets, anomaly types, and computational budgets, while classical and learned approaches carry different practical costs.

    Source: §1 · p. 1

  • Method

    An LLM-driven loop edits a single short detector program under a fixed validation harness and returns the best-scoring attempt across the search.

    Source: §3 · p. 3

  • Results

    The two detectors take the most first-place finishes on every reported TSB-AD metric, ahead of classical, deep, and pretrained detectors including Time-RCD [14].

    Source: §1 · p. 1, §6.1 · p. 5

  • Takeaways & Limitations

    The resulting short, readable programs run on a single CPU core without GPU use or neural training, keeping their inductive biases inspectable and their cost predictable.

    Source: §8 · p. 7, §7 · p. 7

    Generality beyond two independently curated benchmarks remains unconfirmed; broader study across benchmark and dataset families, and potentially real-world deployment, is needed.

    Source: §7 · p. 7

Abstract

from arXiv · show

Time-series anomaly detection trades off predictive accuracy, computational efficiency, and interpretability. We use a large language model not as the detector but as the author of one: an autonomous research loop in which the model repeatedly edits a single short NumPy program under a leakage-free objective, keeping the best-scoring detector it finds. The loop discovers two compact detectors, one for univariate and one for multivariate series, that describe short windows by their local spectral features and compare them with the training-region distribution through a covariance-aware distance. On the TSB-AD benchmark these detectors lead the field across metrics, ahead of the strongest classical, deep, and foundation-model baselines including Time-RCD, yet they train no network and use no GPU, and the multivariate detector is faster than every similarly performing baseline. LLM-driven program search is thus a practical route to accurate, efficient, and transparent detectors.

1 INTRODUCTION

Time-series anomaly detection still lacks a clear winner across datasets, anomaly types, and computational budgets, while classical and learned methods carry different practical costs. The paper uses an LLM-driven search loop to write short detectors and reports strong benchmark performance from compact, interpretable programs.

1 INTRODUCTION

  • No time-series anomaly detector dominates across datasets, anomaly types, and computational budgets; strong classical methods remain leading baselines [26].Deep models can be more competitive in multivariate settings, especially when cross-channel structure matters.Source: §1 · p. 1
  • Classical detectors can require dataset-specific preprocessing and threshold tuning, while deep and foundation models incur training costs, GPU dependence, and opacity.Source: §1 · p. 1
  • An LLM repeatedly edits a short Python/NumPy detector program, with a fixed leakage-free harness scoring each edit on held-out validation data.The trajectory of earlier attempts, including lower-scoring edits, informs subsequent proposals.Source: §1 · p. 1
  • Candidates are validated using synthetic anomalies in a held-out training-region slice, not real labels; inference fits each series without labels.Source: §1 · p. 1
  • The discovered detectors take the most first-place finishes on every reported TSB-AD metric, ahead of classical, deep, and pretrained methods including Time-RCD [14].The two detectors emerged from one-hour sessions of roughly 100 edit-and-evaluate iterations.Source: §1 · p. 1

2 RELATED WORK

The related work spans anomaly-detection benchmarks and model families, LLM-based program discovery, and synthetic data for time-series foundation models. This paper positions the LLM as a detector-program author rather than as the model that scores time series.

Benchmarks and evaluation for time series anomaly detection

  • The TSB-AD comparison covers pretrained, deep, and classical detectors, while TAB provides a broader benchmark view [14] [38] [46] [26].Source: §2 · p. 2

Autonomous research agents and LLM-driven discovery

  • Prior work uses LLMs to propose and refine programs, from mathematical discovery to scientific coding agents and sample-efficient search frameworks [23] [15].EVIL is closest in application, while One-Fits-All and time-series foundation models use language-model backbones or learned weights to score data [3] [49] [14].Source: §2 · p. 2

Synthetic data for foundation models

  • Synthetic simulation supplies priors for time-series and event-sequence foundation models, including approaches using synthetic anomalies, causally coherent kernels, or symbolic expressions [14] [39] [43].Source: §2 · p. 2

3 DISCOVERING DETECTORS WITH AN AUTONOMOUS RESEARCH LOOP

The autonomous loop searches over a single detector program under a fixed harness, retaining the best-scoring attempt across its full trajectory. Its validation objective uses injected synthetic anomalies in held-out training data, with the generator supplying an explicit prior over anomaly types.

Detector representation

  • Candidates are short, self-contained Python/NumPy functions that map training and test segments to per-timestep anomaly scores, making validation evaluations cheap and the code inspectable.A single validation-set evaluation takes minutes rather than hours.Source: §3 · p. 3

The loop

  • The agent edits only the detector program, evaluates each proposal through a fixed harness, and returns the best-scoring attempt rather than automatically accepting or rejecting each edit.Earlier attempts, including lower-scoring ones, remain in context; the search favors short, single-principle programs and runs separately for univariate and multivariate series.Source: §3 · p. 3

Synthetic validation and leakage control

  • Validation uses synthetic anomalies injected into held-out training-region data, spanning magnitude and in-range shape/contextual families, with joint cross-channel perturbations in the multivariate mix.The generator draws on the GutenTAG taxonomy and acts as an explicit prior over possible anomalies.Source: §3 · p. 3

4 METHODS DISCOVERED BY THE SEARCHES

The strongest search produced related spectral-Gaussian novelty detectors for univariate and multivariate series. Both compare window descriptions with normal training-window patterns, while differing in how they describe spectral structure.

The Opus methods

  • The Opus 4.8 search found two spectral-Gaussian novelty methods and performed best overall among the searches.The other loops produced simpler local-feature methods, while Shinka evolutions produced ensembles of standard signals.Source: §4 · p. 3
  • Both detectors slide short windows, describe their rhythms, and score how unusual each description is relative to normal training stretches.The univariate detector captures energy spread across rhythms; the multivariate detector models the combined rhythms of all channels.Source: §4 · p. 3

The univariate detector

  • The univariate detector summarizes spectral shape and rhythm, then scores breaks in the normal relationships among features rather than judging each number alone.Its multi-scale scoring can detect wrong-period or wrong-phase cycles, out-of-context subsequences, and frozen segments even when values remain in the normal range; prior frequency-domain approaches use smoothed baselines [30], nearest neighbours, or learned reconstruction [45].Source: §4 · p. 3
  • Its 19-dimensional descriptor combines eight spectral-organization scalars with 11 log-spaced band-powers and is scored by a shrunk full-covariance Mahalanobis distance to normal windows.Scores are fused across dyadic scales with tail-emphasized weighting.Source: §4 · p. 4

The multivariate detector

  • The multivariate detector models joint per-channel band-power patterns, so it can flag broken cross-channel relationships even when each channel looks normal alone.Regular-stride sampling and a cap on windows keep cost roughly proportional to series length; its fixed-size model can fit a short training segment and apply to much longer test series.Source: §4 · p. 4

5 EXPERIMENTAL SETUP

The evaluation uses TSB-AD under the Time-RCD protocol as its primary benchmark and adds TAB for broader coverage. It compares the detectors with a broad baseline field and tests how the search model and procedure affect the discovered programs.

5 EXPERIMENTAL SETUP

  • TSB-AD evaluation follows the 16-family protocol, covering eleven univariate and five multivariate dataset families; TAB adds 14 univariate families and 25 multivariate families.The TAB field includes 816 univariate series and 214 multivariate series under one evaluation pipeline.Source: §5 · p. 4
  • TSB-AD uses affiliation F-measure, temporal F1-T, standard F1, and VUS-PR, while TAB reports VUS-PR, VUS-ROC, R-AUC-PR, R-AUC-ROC, and affiliation F-measure.The paper follows the metrics standardized for each benchmark.Source: §5 · p. 4
  • TSB-AD comparisons use published Time-RCD protocol results, while TAB comparisons run the bundled detector field from released checkpoints without tuning.The baselines span classical, deep, and foundation-model methods; the authors report verifying their evaluation wrappers against published numbers.Source: §5 · p. 4
  • Four searches hold the harness, seed, synthetic-validation objective, and single-principle prompt fixed while varying the driving model or search procedure.The three LLM loops use Claude Opus 4.8, GPT 5.6 Terra, or MiniMax-2.5 for one-hour sessions of roughly 100 evolutions per regime; ShinkaEvolve uses MiniMax-2.5 for a matched 100 generations.Source: §5 · p. 4

6 RESULTS

The Opus 4.8 loop’s detectors lead the TSB-AD field across all four metrics, with strong results also against methods closest to their design. They pair competitive accuracy with fast CPU-only scoring, while ablations show that the driving model matters more to accuracy than the search procedure.

6.1 Accuracy on TSB-AD

  • On TAB, Opus leads all four range-aware metrics on multivariate data and the ranking-oriented ROC metrics on univariate data, while KMeans leads univariate precision metrics.The detector is mid-field on TAB affiliation F-measure, which rewards broad predicted regions rather than sharp novelty peaks.Source: §6.1 · p. 5

6.2 Did the loop rediscover an existing detector?

  • The detector combines established ingredients—sliding-window spectral features, Mahalanobis distance to a Gaussian of normal windows, and multi-scale fusion—in a combination that outperforms the closest existing methods.The authors interpret this as evidence that the loop composed known concepts rather than rediscovering an existing detector.Source: §6.2 · p. 5

6.1 Accuracy on TSB-AD

  • The Opus 4.8 loop takes the most first-place finishes on all four TSB-AD metrics, with its widest advantage on standard F1 and VUS-PR.Time-RCD is its closest competitor; GPT 5.6 Terra is marginally ahead of Opus on multivariate TSB-AD, while Opus leads on univariate families and both TAB halves.Source: §6.1 · p. 5, Table 2 · p. 6

6.2 Did the loop rediscover an existing detector?

  • VUS-PR drops from 53.9 to 28.4 when catch22 features replace the loop’s descriptor under the same Mahalanobis novelty framework.The discovered detector also outperforms the canonical frequency-domain methods and catch22-based C22MP.Source: §6.2 · p. 5, Table 3 · p. 6

6.4 Ablation: the model and the search procedure

  • Opus is strongest in univariate ablations, but multivariate leadership varies by benchmark: GPT edges Opus on TSB-AD, while Opus leads on TAB.The four-search comparison holds the harness, seed, and prompt fixed while varying model and search procedure.Source: §6.4 · pp. 6–7

6.3 Efficiency

  • The discovered detector is the fastest CPU method in both regimes and the fastest method overall on multivariate TSB-AD; on univariate families, it scores about 5 μs per step.Every per-series-trained deep detector is 40–120× slower even on GPU. On multivariate families, Time-RCD costs about 1.9 s per series and deep detectors 2.5–7 s, all one to two orders of magnitude slower than the CPU-only program.Source: §6.3 · p. 6

6.4 Ablation: the model and the search procedure

  • Changing the search procedure yields no consistent accuracy winner, but agentic loops retain compact single-principle programs while ShinkaEvolve returns longer weighted ensembles.The evolutionary search takes several hours per regime and its multivariate ensemble is comparatively slow on the longest series; one run per regime cannot establish the proposed mechanism for this difference.Source: §6.4 · p. 7

7 DISCUSSION AND LIMITATIONS

The detectors lead reported TSB-AD metrics but have qualified standing on TAB; broader evaluation and the synthetic-anomaly generator constrain claims of generality. Their readable programs make design choices inspectable, while shifting human effort toward the harness and search objective.

7 DISCUSSION AND LIMITATIONS

  • The readable detector programs expose their inductive biases for inspection and make computational cost predictable, unlike opaque trained models.This shifts human effort from detector design toward the harness, synthetic-anomaly generator, and optimized objective.Source: §7 · p. 7
  • On TSB-AD the detectors lead reported metrics, while on TAB they dominate multivariate results but share the univariate top with KMeans and trail on precision metrics.They are mid-field on TAB’s region-based affiliation measure because they are sharp scorers rather than region predictors.Source: §7 · p. 7
  • Generality remains uncertain beyond two independently curated benchmarks, and anomaly types poorly captured by the synthetic generator may be missed.The authors call for broader studies across benchmarks and dataset families, potentially including real-world deployment.Source: §7 · p. 7
  • Efficiency claims concern inference, not discovery, which incurs a one-time cost in language-model queries and compute.The authors report that searches could be performed within $20 subscription plans.Source: §7 · p. 7

8 CONCLUSION

An LLM-driven research loop discovered related spectral-Gaussian novelty detectors for univariate and multivariate time series. They lead reported TSB-AD metrics, remain near the top of TAB, and run as short, unsupervised single-CPU programs without GPU or neural training.

8 CONCLUSION

  • On TSB-AD the detectors lead reported metrics and remain near the top of the broader TAB field.The univariate detector uses a multi-scale spectral-organization signature, while the multivariate detector models joint per-channel band power.Source: §8 · p. 7
  • The loop discovered related spectral-Gaussian novelty detectors: one for univariate series and one for multivariate series.Their respective signatures capture multi-scale spectral organization and joint per-channel band power.Source: §8 · p. 7
  • Both detectors are fully unsupervised, run on a single CPU core without GPU or neural training, and consist of short, readable code.The authors characterize them as cheap and auditable.Source: §8 · p. 7

GENERATIVE AI USAGE

The paper’s research idea, experimental design, and overall structure were developed by the human authors; LLM agents implemented code under human oversight. Generative AI also assisted with manuscript drafting and editing under human oversight, and produced Figure 1.

GENERATIVE AI USAGE

  • Human authors conceived and developed the research idea, experimental design, and paper structure.All claims, results, and conclusions were verified by the authors.Source: GENERATIVE AI USAGE · p. 8
  • LLM agents implemented the code with human oversight.Source: GENERATIVE AI USAGE · p. 8
  • Generative AI assisted with manuscript drafting and editing under human oversight and produced Figure 1.Source: GENERATIVE AI USAGE · p. 8

A METRIC DEFINITIONS

The section defines evaluation measures and the synthetic validation objective, then describes two discovered spectral-novelty detectors and provides their executable implementations.

A METRIC DEFINITIONS

  • Tables report TSB-AD threshold-dependent metrics at the harness-selected score-optimal point, while AUC and VUS families use raw scores directly.On TAB, label metrics are averaged over a fixed contamination-ratio grid; VUS sweeps boundary-buffer widths and is the TSB-AD leaderboard headline metric.Source: A METRIC DEFINITIONS · p. 9
  • Range-aware AUCs credit scores raised near labeled events, while Aff-F and temporal F1-T evaluate predicted regions with event-affiliation or tolerance-based matching.Standard F1 is timestamp-wise; range-aware variants address extended anomalies, and Aff-F applies to point and subsequence anomalies.Source: A METRIC DEFINITIONS · p. 9

B SEARCH CONFIGURATION

  • Synthetic anomalies are injected into the last 30% of each training region across magnitude and contextual or shape-change types, with separate univariate and multivariate generator settings.The slice is at least 128 steps; univariate anomalies include in-range contextual types, while 25% of multivariate anomalies affect all channels jointly.Source: B SEARCH CONFIGURATION · pp. 9–10

C DISCOVERED DETECTOR CONFIGURATIONS

  • Both detectors standardize from training-region statistics, extract local spectral features, and score their novelty against a Gaussian model of normal training windows.The univariate method uses robust median/MAD scaling and a square-root Mahalanobis score; the multivariate method uses per-channel mean/std scaling and a squared score.Source: C DISCOVERED DETECTOR CONFIGURATIONS · p. 10
  • The univariate detector combines spectral-organization, complexity-collapse, and band-power features with shrunk full-covariance Mahalanobis novelty across dyadic window scales.Tail-emphasized rank fusion and a gate based on the training-distance tail combine scale scores; windows adapt to series length and very long inputs are strided.Source: C DISCOVERED DETECTOR CONFIGURATIONS · p. 10
  • The multivariate detector scores length-80 windows using six log-compressed spectral bands per channel and a pooled, ridge-regularized covariance model.Windows are strided and capped to keep computation near-linear; covariance cross-channel blocks encode normal spectral coupling, so broken coupling can score highly even when channels remain individually in range.Source: C DISCOVERED DETECTOR CONFIGURATIONS · p. 10

D DISCOVERED ALGORITHMS

  • The paper reproduces both discovered detectors as complete, self-contained scoring functions that fit on a training segment without labels and return per-timestamp anomaly scores.These are the Opus 4.8 loop’s univariate and multivariate algorithms; other search outputs are available in the anonymized repository.Source: D DISCOVERED ALGORITHMS · pp. 10, 12

E COMPLETE RESULT TABLES

The section presents full TAB baseline fields and the TSB-AD comparison table, including their scopes and reporting conventions.

E COMPLETE RESULT TABLES

  • Tables 5 and 6 report full univariate and multivariate TAB baseline fields as dataset means, with higher-is-better metrics and columns marking best values and podium places.The discovered detectors appear first, and the baseline field is ordered by VUS-PR; the TAB results are presented as broader corroboration.Source: E COMPLETE RESULT TABLES · p. 12, Table 5 · p. 13, Table 6 · p. 14
  • Table 7 reports TSB-AD results under the Time-RCD setup, combining the separately discovered univariate and multivariate algorithms and reporting podium counts.Gold, silver, and bronze markers indicate first, second, and third place.Source: Table 7 · p. 15
  • Table 8 compares the closest published methods on TSB-AD multivariate families using univariate baselines applied per channel and averaged.It uses the Time-RCD protocol and dataset means, with higher values better.Source: Table 8 · p. 16

F CLOSEST-METHOD COMPARISON ON MULTIVARIATE FAMILIES

The discovered multivariate detector leads the reported comparison across four metrics, although the comparison uses univariate baselines applied per channel.

F CLOSEST-METHOD COMPARISON ON MULTIVARIATE FAMILIES

  • The discovered multivariate detector remains ahead on all four metrics, most clearly on VUS-PR, in an indicative comparison against univariate methods applied per channel.The baseline scores are averaged across channels; the authors characterize this comparison as indicative because the baselines are not native multivariate detectors.Source: F CLOSEST-METHOD COMPARISON ON… · p. 16

Related papers

Loading 2610.01223v1…