PaperlayerView exact revision on arXiv

Source-linked AI summary

See it, Think it, Sorted: Large Multimodal Models are Few-shot Time Series Anomaly Analyzers

Jiaxin Zhuang, Leon Yan, Zhenwei Zhang, Ruiqi Wang, Jiawei Zhang, Yuantao Gu

arXiv:2411.02465v1cs.LGcs.AIstat.ML

TL;DR

TSAD methods can depend on manual features or scarce training data and often provide limited interpretability. TAMA visualizes time series for few-shot LMM analysis, and the authors report state-of-the-art results with semantic anomaly insights.

  • Problem

    TSAD methods often require manual feature engineering or substantial labeled data, while offering limited interpretability and semantic classification.

    Source: §1 · p. 1

  • Method

    TAMA converts time series into charts for LMMs and uses few-shot reference learning, analysis, and multi-scaled self-reflection to detect and explain anomalies.

    Source: §1 · p. 2, §3.2 · p. 4

  • Results

    TAMA outperforms state-of-the-art methods across multiple TSAD datasets and provides semantic classifications and insights into detected anomalies.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    The authors present TAMA as an approach to time-series anomaly detection that also supplies semantic classifications and natural-language reasoning.

    Source: §1 · p. 2, §6 · p. 9

    This work uses pretrained LMMs without fine-tuning and considers only univariate time-series anomaly detection.

    Source: §6 · p. 9

Abstract

from arXiv · show

Time series anomaly detection (TSAD) is becoming increasingly vital due to the rapid growth of time series data across various sectors. Anomalies in web service data, for example, can signal critical incidents such as system failures or server malfunctions, necessitating timely detection and response. However, most existing TSAD methodologies rely heavily on manual feature engineering or require extensive labeled training data, while also offering limited interpretability. To address these challenges, we introduce a pioneering framework called the Time Series Anomaly Multimodal Analyzer (TAMA), which leverages the power of Large Multimodal Models (LMMs) to enhance both the detection and interpretation of anomalies in time series data. By converting time series into visual formats that LMMs can efficiently process, TAMA leverages few-shot in-context learning capabilities to reduce dependence on extensive labeled datasets. Our methodology is validated through rigorous experimentation on multiple real-world datasets, where TAMA consistently outperforms state-of-the-art methods in TSAD tasks. Additionally, TAMA provides rich, natural language-based semantic analysis, offering deeper insights into the nature of detected anomalies. Furthermore, we contribute one of the first open-source datasets that includes anomaly detection labels, anomaly type labels, and contextual description, facilitating broader exploration and advancement within this critical field. Ultimately, TAMA not only excels in anomaly detection but also provides a comprehensive approach for understanding the underlying causes of anomalies, pushing TSAD forward through innovative methodologies and insights.

1 Introduction

TSAD faces dataset-specific methods, training-data demands, and limited interpretability, while LLMs struggle with numerical time series. TAMA visualizes series for LMM analysis and reports stronger detection alongside semantic classifications and reasoning.

1 Introduction

  • Existing TSAD methods often specialize by dataset; machine-learning approaches require assumptions or manual features, while deep-learning methods commonly need normal training data.Such data may be rare, undesirable, or unavailable in real-world settings.Source: §1 · p. 1
  • Most TSAD methods offer limited explanations, and scarce high-quality classification labels have constrained semantic classification of anomalies.Prior taxonomies distinguish point and pattern anomalies and include finer-grained types.Source: §1 · pp. 1–2
  • LLMs can learn from few examples, but their text-based handling of numerical series struggles with subtle amplitude changes; visualizing series lets LMMs analyze them as charts.The paper motivates visualization as a way to use multimodal reasoning for time-series analysis.Source: §1 · p. 2
  • TAMA converts time series into chart inputs for LMMs, which identify anomalies, classify their types, and provide detailed reasoning through a three-stage pipeline.Its stages are Multimodal Reference Learning, Multimodal Analyzing, and Multi-scaled Self-reflection.Source: §1 · p. 2
  • TAMA outperforms state-of-the-art methods across multiple TSAD datasets and provides semantic classifications and anomaly insights.The authors also report an open-source dataset with detection labels, anomaly-type labels, and contextual descriptions.Source: §1 · p. 2

2 Relate Works

Prior TSAD work spans classical and deep-learning detectors, anomaly-analysis methods, LLM applications, and image representations. The review highlights gaps in semantic analysis, numerical time-series handling, and applying LMMs to visualized-series anomaly detection.

2.1 Time Series Anomaly Detection.

  • Classical and deep-learning detectors use varied feature, reconstruction, or forecasting strategies, but may require manual engineering or miss subtle, small-amplitude anomalies.The paper contrasts these limitations with its own few-shot image-based approach.Source: §2.1 · pp. 2–3

2.2 Time Series Anomaly Analysis.

  • Anomaly analysis remains limited; existing type classifiers rely on supervised training, and cited experiments cover only single classification datasets.The review contrasts this with LLM-based natural-language classifications and potential causes produced unsupervisedly.Source: §2.2 · p. 3

2.3 LLMs for time series.

  • LLM time-series work includes tokenizer-oriented preprocessing, forecasting prompts, and text-modality frameworks, but numerical insensitivity and scarce sequential training data hinder subtle-change detection.Most reviewed efforts focus on forecasting and textual inputs.Source: §2.3 · p. 3

2.4 Time series as images

  • Prior work encodes time series as images using methods such as GAF and MTF, but the review reports no prior LMM application to time-series anomaly detection.Other examples include line-graph images and converting time series to 2D tensors.Source: §2.4 · p. 3

3 Methodology

TAMA turns time series into visual inputs for pretrained LMMs, then uses reference learning, anomaly analysis, and self-reflection before consolidating outputs into TSAD results.

3 Methodology

  • TAMA combines visual time-series inputs with pretrained LMM perception and reasoning to target few-shot anomaly detection and robustness.Its workflow includes three LMM sections and a post-processing module for final results.Source: §3 · p. 3, §3.2 · p. 4

3.1 Preliminary

  • The framework treats multivariate data as multiple univariate series, normalizes the data, segments it into overlapping windows, and converts it into resolution-constrained graphs.The overlap allows the same sequence segment to appear in multiple windows.Source: §3.1 · pp. 3–4

3.2 Time Series Anomaly Analyzer (TAMA)

  • Multimodal Reference Learning uses normal-series images and generated descriptions to help the LMM capture normal patterns for later anomaly detection.The model receives prompts identifying the reference images as normal and generates descriptions for subsequent interactions.Source: §3.2 · p. 4
  • Multimodal Analyzing detects intervals in each window, then classifies them as point, shapelet, seasonal, or trend anomalies and supplies confidence scores and explanations.Detected intervals are represented with window-local indices before post-processing maps them to global indices.Source: §3.2 · p. 4
  • Multi-scaled Self-reflection revisits detected regions with zoomed-in images, while post-processing combines overlapping-window confidence scores and classification votes to produce final anomaly points.Self-reflection is run only on windows where anomaly intervals were detected; a threshold on the combined confidence sequence determines final points.Source: §3.2 · pp. 4–5
  • Table 1 reports results across six datasets using point-adjusted F1%, AUC-PR%, and AUC-ROC%, with mean and maxima values for each metric.The table includes TAMA and TAMA*, the framework without self-reflection.Source: Table 1 · p. 5

4 Experiments

Across diverse TSAD datasets, TAMA outperforms almost all baselines on mean metrics; classification is reliable for most anomaly types except seasonal anomalies. Ablations report benefits from reference learning and larger windows.

4 Experiments

  • Experiments span real-world datasets from web services, industry, healthcare, and transportation; classification labels were added to four real-world datasets and a synthetic dataset.Experts cross-validated labels for the four real-world datasets; the synthetic dataset has 7,200 samples labeled as point, trend, or frequency anomalies.Source: §4 · pp. 5–6, Table 2 · p. 6

4.1 Anomaly Detection

  • TAMA outperforms almost all baselines across datasets on mean metrics, while its maxima are comparable to other models and sometimes better.The authors attribute the mean-metric results to superior stability across a broader range of data.Source: §4.1 · p. 6
  • Self-reflection keeps maxima close to TAMA* but improves mean results in most datasets, especially NASA-SMAP.TAMA* denotes the method without self-reflection; the authors interpret the mean drops without it as evidence of improved stability.Source: §4.1 · p. 6
  • Performance declines for all models as the PAT threshold increases, while TAMA performs well against baselines across PATs.The authors say full point-adjustment at α=0 overestimates model performance.Source: §4.1 · p. 6

4.2 Anomaly Classification

  • TAMA reliably classifies most anomaly types, performs exceptionally well on shapelet anomalies, and struggles with seasonal anomalies.The authors interpret the seasonal difficulty as possibly stemming from limited pre-training material on seasonality or frequency.Source: §4.2 · p. 6

4.3 Ablation Study

  • All tested LMMs show substantial UCR AUC-PR improvement when integrated with TAMA, supporting the framework's generalizability.The comparison uses average AUC-PR without point-adjustment for Naive versus +TAMA.Source: §4.3 · p. 7, Table 4 · p. 7
  • With Reference Number set to 0 (no references), performance drops significantly; as reference number increases, performance fluctuates but remains relatively stable and is considerably improved compared with using no references.The ablation used UCR and NASA-SMAP without PA; reference counts from 1 to 4 showed no clear relationship with final results, while setting the count to 3 under random sampling of normal data enhanced overall robustness.Source: §4.3 · p. 7
  • Performance is positively correlated with window size, so the experiments use a window of approximately 3T.The authors explain that single-period images can hinder recognition of periodic patterns and lead to periodic or truncated features being classified as anomalies.Source: §4.3 · p. 7, Figure 5 · p. 8

5 Discussion

The discussion compares image and text inputs, tests how visual features affect detection, and examines whether reference images teach the LMM normal patterns.

5.1 Modality

  • 37.9% on NASA-MSL and 36.9% on NASA-SMAP: TAMA (Image) improves over text-modality methods in average PA F1%.The comparison omits self-reflection and uses otherwise similar prompts and procedures.Source: §5.1 · p. 8, Table 5 · p. 8

5.2 Additional Information in Images

  • Rotated images reduce AUC-PR without point adjustment, while removing auxiliary lines causes only slight declines on UCR and NASA-SMAP.The authors suggest LMMs are sensitive to orientation and locate abnormal intervals better when auxiliary lines are present.Source: §5.2 · p. 8, Table 6 · p. 8

5.3 Multimodal Reference Learning

  • Normal reference images outperform abnormal reference images on both UCR and NASA-SMAP, suggesting the LMM learns normal representations from reference data.The experiment disables Multi-scaled Self-reflection and evaluates AUC-PR without point adjustment.Source: §5.3 · p. 8, Table 7 · p. 8

6 Conclusion

The conclusion presents TAMA as an LMM framework for time-series anomaly analysis, reporting detection and semantic-analysis strengths while identifying univariate scope as a limitation.

6 Conclusion

  • Across multiple metrics, TAMA surpasses state-of-the-art methods and provides semantic classifications and insights into detected anomalies.The framework converts time-series data into visual representations; the paper describes this as its first application of LMMs to the domain.Source: §6 · p. 9
  • The authors identify practical implications for real-world applications and future research in multidimensional anomaly detection.Source: §6 · p. 9
  • The approach primarily relies on pretrained LMMs without fine-tuning and considers only univariate anomaly detection, although real-world comprehensive judgment requires multiple series.The authors plan to explore local deployment, fine-tuning, and multivariate anomaly detection.Source: §6 · p. 9

A.1 Prompts

The prompt design divides the task into three named stages and supplies context and structured output instructions.

A.1 Prompts

  • Prompts separate the task into Multimodal Reference Learning, Multimodal Analyzing, and Multi-scaled Self-reflection.Source: §A.1 · p. 12
  • Prompts also provide background on sliding windows and additional image information.Source: §A.1 · p. 12
  • GPT-4o JSON-mode prompts specify the output format, and the authors report that clear descriptions and structure help the LMM understand.Source: §A.1 · p. 12

A.2 Point Adjustment Metrics

The appendix defines point-adjusted predictions and metrics, including a threshold that controls the degree of point adjustment.

A.2 Point Adjustment Metrics

  • Point adjustment forms APA by adding points within real anomalous intervals when a predicted interval overlaps them.The predicted intervals are denoted AP; the real intervals are denoted AT.Source: §A.2 · p. 12
  • After point adjustment, recall, precision, and F1 are calculated using APA, the set of point predictions.The passage defines mT and mP as the numbers of real anomalous intervals and LLM predictions, respectively.Source: §A.2 · p. 12
  • The point-adjustment threshold α ranges from 0 for full point adjustment to 1 for the original prediction AP.Source: §A.2 · p. 12

A.3 Some Suggestions about TAMA

TAMA’s implementation suggestions cover structured output, window sizing, and plot readability.

A.3 Some Suggestions about TAMA

  • Choose an LMM with JSON or structured output, or use GPT-4o to format unstructured output as structured text.Structured output is recommended to make results easier to parse.Source: §A.3 · p. 12
  • When the series period is T, set the sliding window length to at least 3T.Source: §A.3 · p. 12
  • Use a clear axis scale, optionally add grid lines, and keep images at or below 2000x768 pixels for high-resolution mode.The authors say scale rotation does not matter.Source: §A.3 · p. 12

A.4 Full Results of Anomaly Detection across All Datasets

The appendix reports full results across datasets and evaluates metrics over individual subsequences, summarizing performance with several statistics.

A.4 Full Results of Anomaly Detection across All Datasets

  • The full results cover all datasets and metrics, with performance evaluated across every subsequence because most datasets contain multiple subsequences.Source: §A.4 · p. 12
  • For each metric, the results include the mean, variance, and maximum across subsequences.Source: §A.4 · p. 12
  • The table formats mean and variance together as “mean ± variance.”Source: §A.4 · p. 12

A.5 Full Results of the PAT Experiment

The appendix reports TAMA’s AUC-PR performance under varied point-adjustment thresholds, evaluates detection by anomaly type, and visualizes four labeled anomaly categories.

A.5 Full Results of the PAT Experiment

  • Across all datasets and varied point-adjustment thresholds, TAMA achieves outstanding AUC-PR, which the authors describe as evidence of robustness and stability.Source: §A.5 · p. 12, Figure 6 · p. 16

A.6 Performance Evaluation of Type-Specific Anomaly Detection Methods

  • On UCR-shapelet, TAMA exceeded second-best GDN by 293% in mean F1-score; interval detection can lower point-anomaly F1 through false positives.The reported strength is in detecting pattern anomalies, including shapelet, seasonal, and trend types.Source: §A.6 · p. 12, Table 9 · p. 17

A.7 Visualization of anomaly classification

  • The labeled anomaly categories are point, shapelet, seasonal, and trend; Figure 7 shows sequences from different datasets sharing an anomaly type.The dataset labels some real-world datasets and generated sequences.Source: §A.7 · p. 12, Figure 7 · p. 18
Loading 2411.02465v1…