PaperlayerView exact revision on arXiv

Source-linked AI summary

Normality Constraint Learning: Adapting Foundation Models for Time Series Anomaly Detection

Xiaohui Zhou, Yijie Wang, Hongzuo Xu, Weixuan Liang, Guansong Pang

arXiv:2610.06453v1cs.AI

TL;DR

General-purpose TSFMs can model anomalies as effectively as normal patterns, weakening reconstruction- and forecasting-error anomaly scores. NCL steers frozen TSFM features toward a few-shot normality subspace and fuses calibrated features with the original output. Across ten TSAD datasets, the paper reports consistent detection improvements across diverse TSFM backbones.

  • Problem

    Because TSFMs can model normal and anomalous patterns similarly well, anomalies may receive small reconstruction or forecasting errors, weakening error-based detection.

    Source: §1 · pp. 1–2

  • Method

    NCL adapts frozen TSFMs by constructing a few-shot normality subspace, steering patch features toward normality, and guiding compact, discriminative manifolds.

    Source: §1 · p. 2, §4.1 · p. 4

  • Results

    Across ten TSAD datasets, NCL consistently and significantly improves diverse TSFMs, with a 41.49% average improvement ratio across backbone models.

    Source: §5.2 · p. 7

  • Takeaways & Limitations

    The results support NCL as a generalizable framework for further specializing pretrained TSFMs to anomaly detection.

    Source: §6 · p. 9

Abstract

from arXiv · show

Time Series Foundation Models (TSFMs) achieve strong generalization by learning to reconstruct or forecast broad temporal patterns from large-scale time series during pre-training. Yet this strength can become a weakness for anomaly detection: TSFMs may model rare anomalous patterns as effectively as normal ones, allowing anomalies to be accurately reconstructed or forecasted and thus diminishing their reconstruction/forecasting error-based anomaly scores. This paper proposes $\underline{\textbf{N}}$$\textbf{ormality}$ $\underline{\textbf{C}}$$\textbf{onstraint}$ $\underline{\textbf{L}}$$\textbf{earning}$ ($\textbf{NCL}$), a lightweight plug-and-play framework that adapts pre-trained TSFMs for accurate anomaly detection without modifying their pre-trained parameters. Our key insight is to constrain the broad pattern space of TSFMs to the normal structure of a target time series, preventing their broad modeling capability from obscuring abnormal deviations. Specifically, NCL constructs a compact normality subspace from a few normal patch features and adaptively steers each patch feature toward normality within this subspace, guided by contrastive constraints that form compact and discriminative normality manifolds. The calibrated features are aggregated to reinforce normal components and fused with the original TSFM output, amplifying the discrepancy between normal and abnormal observations for the reconstruction/forecasting error-based anomaly scoring. Extensive experiments across diverse TSFM families and benchmarks show that NCL consistently improves anomaly detection performance, providing a generalizable framework for adapting TSFMs to anomaly detection.

1 INTRODUCTION

General-purpose TSFMs can model anomalous patterns as well as normal ones, weakening error-based anomaly scores. NCL adapts frozen TSFMs toward normal patterns, and experiments report improved detection across diverse TSFM families.

1 INTRODUCTION

  • TSFMs may reconstruct or forecast anomalies effectively, leaving them with small errors and undermining error-based anomaly detection.The paper frames this as a consequence of TSFMs' broad pattern-modeling capability.Source: §1 · p. 1
  • NCL adapts diverse TSFMs without changing pretrained parameters, steering patch features toward normality using a subspace and compact, discriminative manifolds.NormSpace constructs the subspace, PatchNT transforms features, and ConNC guides the transformation.Source: §1 · p. 2
  • NCL consistently and significantly improves anomaly detection across diverse TSFM families on popular TSAD datasets.The paper attributes sharper reconstruction- and forecasting-error anomaly scores to its normality constraints.Source: §1 · p. 2

2 RELATED WORK

The related work covers conventional anomaly detection approaches and their data demands, then turns to foundation models and TSAD-specific variants. It highlights limits in direct TSFM reuse and synthetic-anomaly-based pretraining.

2.1 TIME SERIES ANOMALY DETECTION

  • Conventional TSAD uses statistical, machine-learning, reconstruction, forecasting, and contrastive approaches to identify deviations or discrepancies.Many existing methods require separate, from-scratch training on abundant domain-specific normal data, limiting use where such data are scarce or costly.Source: §2.1 · p. 3

2.2 TIME SERIES FOUNDATION MODELS FOR ANOMALY DETECTION

  • Task-general TSFMs offer universal temporal representations, but can model normal and anomalous patterns effectively, weakening error-based anomaly detection.This challenges the common assumption that anomalies are harder to reconstruct or forecast.Source: §2.2 · p. 3
  • TSAD-specific foundation models use synthetic anomalies during pretraining, but low-quality synthetic anomalies may not transfer to real-world distributions.The cited methods require many synthetic anomalies to enrich their pretraining corpora.Source: §2.2 · p. 3

3 PRELIMINARIES

TSAD assigns anomaly scores to observations, aiming to score anomalous observations above normal ones. In this paper, TSFMs reconstruct an input or forecast future observations, and squared prediction error supplies the pointwise score.

3 PRELIMINARIES

  • TSAD calculates a score for each observation, with the goal of assigning anomalous observations higher scores than normal observations.The paper denotes the time series as having D channels and T ordered observations.Source: §3 · p. 3
  • A TSFM reconstructs its input or forecasts future observations from a subsequence, and the squared difference between target and prediction gives the pointwise anomaly score.The prediction target is the input under reconstruction and future observations under forecasting.Source: §3 · p. 3

4 METHODOLOGY

NCL steers frozen TSFM patch features toward normal structure using a few-shot normal subspace, patch-adaptive transformations, and contrastive constraints. Its training and inference combine prediction and normality signals to distinguish anomalous observations.

4.1 OVERVIEW OF NCL: TOWARD NORMALITY-CONSTRAINED TSFMS

  • NCL constrains a TSFM's broad pattern space toward normality without changing its pretrained parameters, aiming to amplify normal–abnormal discrepancy.Its three components are NormSpace, PatchNT, and ConNC.Source: §4.1 · p. 4, Figure 2 · p. 4

4.2 NORMSPACE: NORMAL SUBSPACE CONSTRUCTION VIA FEW-SHOT SVD

  • NormSpace applies SVD to patch features from a few normal training subsequences and retains top-k right singular vectors as a normal subspace.The vectors capture dominant, stable patterns shared across normal patch features; the paper theoretically supports few-shot construction and top-k selection.Source: §4.1 · p. 4, §4.2 · pp. 4–5

4.3 PATCHNT: LIGHTWEIGHT PATCH-ADAPTIVE NORMALITY TRANSFORMATION

  • PatchNT selectively adds or deletes information along each principal direction, adaptively steering each patch feature toward normal temporal structure.Its lightweight learned target and calibration strength control where and how much each feature is calibrated.Source: §4.3 · p. 5

4.4 CONNC: CONTRASTIVE NORMALITY CONSTRAINT

  • ConNC uses transformation views as self-supervised labels, encouraging within-view compactness and between-view separation into discriminative normality manifolds.Each view is encouraged to lie closer to its own normal prototype than to other view prototypes by a margin.Source: §4.4 · p. 6

4.5 TRAINING AND INFERENCE

  • The fused prediction uses a learnable channel-wise gate to balance the normality-constrained module prediction with the frozen TSFM prediction.The stated design increases reconstruction or forecasting error for abnormal observations through the normality-constrained component.Source: §4.5 · p. 6
  • The paper theoretically justifies error amplification for anomalous inputs under its stated assumptions.The result concerns fused predictions from a frozen TSFM and anomalous pairs within a neighborhood of the pretraining support.Source: §4.5 · p. 6
  • Training combines prediction squared error with ConNC, updates only NCL parameters, and uses the constraint to resist overfitting to abnormal information in training patches.During inference, the score combines amplified prediction error with view-to-prototype misalignment, taking the maximum over patches for the latter component.Source: §4.5 · pp. 6–7

5 EXPERIMENTS

Across ten TSAD datasets, NCL improves diverse TSFMs; its normality subspace grounds the ablation gains, while lightweight training supports favorable effectiveness-efficiency trade-offs.

5.1 EXPERIMENTAL SETUP

  • Experiments cover ten univariate and multivariate TSAD datasets, diverse forecasting, reconstruction, and combined TSFM backbones, and three independent runs.VUS-PR is the primary metric; several other PR and ROC metrics are reported in the appendix.Source: §5.1 · p. 7

5.2 MAIN RESULTS

  • 41.49% average improvement across backbone models; NCL consistently and significantly improves anomaly detection across ten TSAD datasets.The comparison covers eight TSFMs with and without NCL and uses VUS-PR as the primary metric.Source: §5.2 · p. 7, Table 1 · p. 8
  • NCL outperforms LoRA and CoRA across three representative TSFMs, using an explicit normality constraint absent from those adaptation methods.Figure 3 reports performance changes relative to vanilla TSFMs.Source: §5.2 · p. 8, Figure 3 · p. 8

5.3 ABLATION STUDIES

  • NormSpace-only NCLN improves consistently over all three vanilla TSFMs, indicating that a few patch features can ground a sample-efficient normality space.The ablations also examine the contribution of NormSpace in NCLP versus NCLNP and NCLPC versus full NCLNPC.Source: §5.3 · p. 8, Table 2 · p. 9

5.4 EFFICIENCY ANALYSIS

  • NCL uses an average of 14.2K trainable parameters and completes training in around 2 seconds while achieving stronger detection effectiveness and fast training.The TSFM remains frozen while lightweight projection heads and fusion weights are learned.Source: §5.4 · p. 9

6 CONCLUSION

The paper connects TSFM overgeneralization to unreliable error-based anomaly scores and presents theoretical arguments for how normality constraints address this issue.

6 CONCLUSION

  • NCL augments frozen TSFMs with NormSpace, PatchNT, and ConNC to steer patch features toward compact, discriminative normality manifolds.The conclusion presents this as a generalizable framework and perspective for adapting pre-trained TSFMs to anomaly detection.Source: §6 · p. 9

A THEORETICAL ANALYSIS OF APPLYING TSFMS TO ANOMALY DETECTION

  • Broad pre-training patterns and model capacity let TSFMs reconstruct or forecast many anomalies, undermining the assumption that anomalies yield larger errors.The paper attributes this overgeneralization to pre-training on diverse patterns and the capacity to internalize broad temporal patterns.Source: A THEORETICAL ANALYSIS OF APPLYING… · p. 15

A.1 PRE-TRAINING AND ANOMALY DETECTION ON THE TSFMS

  • When anomalous support lies near a pre-training distribution and prediction error there is small, anomaly reconstruction or forecasting error can be unreliable.The analysis bounds prediction error using approximation error, irreducible uncertainty, neighborhood radius, and the model's Lipschitz constant.Source: §A.1 · pp. 15–16

A.2 THEORETICAL JUSTIFICATION OF NORMALITY CONSTRAINT LEARNING

  • NCL's theoretical account identifies two mechanisms: output fusion amplifies anomaly error, while ConNC regularizes against anomaly overfitting.The mechanisms are described as complementary ways to mitigate TSFM overgeneralization.Source: §A.2 · p. 16

A.2.1 ERROR AMPLIFICATION VIA OUTPUT FUSION (PROOF OF THEOREM 1)

  • Under the stated assumptions, output fusion increases anomaly prediction error while maintaining low error on normal data.The theorem analysis attributes the anomaly error increase to the normality-constrained component of the fused prediction.Source: §A.2.1 · pp. 17–18

A.2.2 REGULARIZATION AGAINST ANOMALY OVERFITTING

  • ConNC clusters normal transformation views around separated centers; anomalous features outside the margin receive zero loss and do not drive parameter updates.The paper presents this as regularization that prevents adaptation to abnormal patterns.Source: §A.2.2 · p. 18

B THEORETICAL ANALYSIS FOR NORMAL SUBSPACE CONSTRUCTION

  • The normal-subspace analysis argues that few-shot normal samples can construct an informative space and top-k truncation can retain essential normal information while suppressing noise.These are the two aspects examined in the theoretical analysis of NormSpace.Source: B THEORETICAL ANALYSIS FOR NORMAL… · p. 18

B.1 FEW-SHOT SVD FOR EFFECTIVE ESTIMATION OF THE NORMALITY SPACE

The appendix supports few-shot normal-subspace estimation and analyzes why top-k truncation retains useful information while reducing noise. It also characterizes how PatchNT adjusts feature components along normal directions.

B.1 FEW-SHOT SVD FOR EFFECTIVE ESTIMATION OF THE NORMALITY SPACE

  • When N = 5 × p and only five normal samples are used, error is 0.1118 at p = 16 and 0.0791 at p = 32; few normal samples suffice for an informative normality space.Source: §B.1 · p. 20

B.2 EFFECTIVENESS OF THE TOP-k TRUNCATION

  • Top-k truncation is analyzed as retaining dominant information and reducing noise, with its effectiveness examined against arbitrary k-vector approximations.The analysis states that top-k approximation retains more effective information than arbitrary k approximation and reduces noise under a clean-data-plus-white-noise model.Source: §B.2 · pp. 20–21

B.2.2 PROOF OF THEOREM 4

  • Theorem 4 bounds the difference between each patch feature and its top-k approximation, framing the retained information in terms of dominant patterns.The proof uses decreasing singular values to establish an upper bound.Source: §B.2 · p. 20, §B.2.2 · p. 22

B.2.3 PROOF OF THEOREM 5

  • Under uncorrelated clean data and white noise, the top-k truncated approximation is the best least-squares estimate of the clean matrix among rank-k approximations.The result relies on the clean matrix having rank k.Source: §B.2 · p. 21, §B.2.3 · p. 23

B.2.1 PROOF OF THEOREM 3

  • For a well-trained decoder, top-k truncation is compared with arbitrary k-vector approximation by output error, with lower error indicating more effective information retained.The proof assumes that a singular vector's contribution to decoding output increases with its singular value.Source: §B.2 · p. 20, §B.2.1 · p. 21

C PROOF OF PROPOSITION 1 (SPECTRAL AND GEOMETRIC PROPERTY)

  • PatchNT adjusts each patch's projection along a normal direction, ranging from partial retention to replacement or over-calibration.The transformation leaves components orthogonal to that direction unchanged.Source: C PROOF OF PROPOSITION 1 (SPECTRAL AND… · p. 23

D.1 DATASETS

The evaluation uses ten TSAD datasets and ten TSFMs, with anomaly scores based on each model's forecasting, reconstruction, or combined error. NCL is trained with frozen TSFM parameters, and its complexity is approximately linear in feature dimension.

D.1 DATASETS

  • The evaluation covers ten TSAD datasets and 361 time series, using five normal training samples for NCL and testing on normal and anomalous observations.The set includes five univariate and five multivariate datasets; MSL and SMAP multivariate series are transformed into univariate ones.Source: §D.1 · p. 24

D.2 BASELINES

  • Ten TSFMs spanning forecasting, reconstruction, and combined approaches are evaluated zero-shot, with their corresponding prediction errors serving as baseline anomaly scores.NCL is applied as a plug-and-play enhancement to those baseline scores.Source: §D.2 · pp. 24–25

D.3 IMPLEMENTATION DETAILS

  • NCL uses k = 16 and trains for five epochs on five randomly selected normal samples while keeping all pre-trained TSFM parameters frozen.The specified learning rate is 1e-4, with Adam optimization.Source: §D.3 · p. 26

E ALGORITHMS DETAILS

  • Algorithm 1 combines SVD-based direction selection, patch-wise normality transformation, contrastive constraints, and gated fusion while updating only NCL parameters.The TSFM remains frozen throughout training.Source: E ALGORITHMS DETAILS · p. 27

F COMPUTATIONAL COMPLEXITY ANALYSIS

  • With k ≪ d and a frozen TSFM, NCL training is approximately O(N · k^2 · d) per iteration, linear in feature dimension d.One-time SVD subspace construction costs O(N^2 · d).Source: F COMPUTATIONAL COMPLEXITY ANALYSIS · p. 28

G DETAILED EFFICIENCY COMPARISON WITH OTHER TSFM ADAPTATION METHODS

Across TSFMs, NCL is reported to be more efficient than alternative adaptation methods, with faster training in nearly all cases. On MOMENT, its training time is lower than both LoRA's and CoRA's.

G DETAILED EFFICIENCY COMPARISON WITH OTHER TSFM ADAPTATION METHODS

H COMPARISON WITH DEEP TSAD AND ONE-LINER METHODS

Across deep TSAD methods and simple baselines, NCL improves TSFM anomaly detection; comparisons with TSAD-specific foundation models and visual analyses also show broad performance and separation gains.

H COMPARISON WITH DEEP TSAD AND ONE-LINER METHODS

  • 2.13 is UniTS+NCL’s average rank across PR metrics and datasets, versus 5.13 for CroAD; NCL-enhanced TSFMs also surpass one-liner baselines.Var-100’s average rank is 4.00, while UniTS+NCL ranks 2.13 in PR metrics and 2.73 in ROC metrics.Source: Table 4 · p. 29, H COMPARISON WITH DEEP TSAD AND… · p. 29

I COMPARISON WITH TSAD-SPECIFIC FOUNDATION MODELS

  • UniTS+NCL achieves the best performance on 4 of 10 datasets and TMM+NCL on 2, while both are competitive with TSAD-specific foundation models.The TSAD-specific models show inconsistent results across datasets, whereas NCL-augmented TSFMs have higher average performance across datasets.Source: Table 6 · p. 30, I COMPARISON WITH TSAD-SPECIFIC… · pp. 30–31

J ONE-TO-ONE COMPARISON WITH ONE-LINER BASELINES

K DISTINGUISHABLE DISTRIBUTIONS OF ANOMALY SCORES

  • NCL widens the gap between normal and abnormal anomaly-score distributions for forecasting-based Aurora and reconstruction-based UniTS.For Aurora, normal scores concentrate around 0–0.1 and anomalous scores shift toward 0.1–0.4; UniTS shows a similar separation pattern.Source: K DISTINGUISHABLE DISTRIBUTIONS OF… · pp. 32–33, Figure 8 · p. 33

L VISUALIZATION OF FEATURE SPACE

  • t-SNE visualizations show normal and abnormal patches intermingled for vanilla TMM but separated into distinct regions after NCL.The authors relate this spatial separation to more discriminative reconstruction- and forecasting-error anomaly scoring.Source: L VISUALIZATION OF FEATURE SPACE · pp. 33–34, Figure 9 · p. 34

M HORIZON AND WINDOW SIZE ANALYSIS FOR TSFMS

NCL’s gains remain stable across forecasting horizons and reconstruction window sizes, anomaly-score plots show improved alignment with labeled anomalies, and experiments report strong performance with few normal samples.

M HORIZON AND WINDOW SIZE ANALYSIS FOR TSFMS

  • NCL-enhanced TMM remains relatively stable across forecasting horizons, while NCL-enhanced UniTS improves across reconstruction windows, including PSM VUS-PR rising from around 40 to nearly 50.The paper attributes the scale robustness to constructing the normal subspace from patch features rather than raw observations.Source: M HORIZON AND WINDOW SIZE ANALYSIS FOR… · pp. 34–35, Figure 10 · p. 35

N VISUALIZATION OF ANOMALY SCORES

  • On univariate and multivariate examples, NCL suppresses normal-region scores and produces sharper peaks aligned with ground-truth anomalies.A UCR anomaly segment around timesteps 1250–1750 that was missed before NCL produces elevated scores afterward.Source: N VISUALIZATION OF ANOMALY SCORES · pp. 35, 37, Figure 11 · p. 36

O UTILITY OF FEW-SHOT NORMAL SAMPLES

  • ROSE+NCL shows substantial improvement across datasets with one normal sample; gains generally diminish beyond 5–10 samples, and five samples approach asymptotic performance.The paper notes that 1–3 samples often suffice on simpler IOPS and NAB datasets, while 5–10 provide better coverage for complex SMD and SWAT datasets.Source: O UTILITY OF FEW-SHOT NORMAL SAMPLES · p. 37

P SENSITIVITY ANALYSIS FOR SINGULAR VECTORS

The singular-vector count controls the normality subspace’s capacity and computational cost, with too few or too many vectors potentially reducing detection performance.

P SENSITIVITY ANALYSIS FOR SINGULAR VECTORS

Q FULL EXPERIMENTAL RESULTS OF VANILLA AND NCL-AUGMENTED TSFMS

Across ten TSFMs and ten benchmarks, NCL generally improves PR and ROC results, with paired tests finding significant improvements for almost all model groups. Correlation analyses also show stronger alignment among augmented models.

Q FULL EXPERIMENTAL RESULTS OF VANILLA AND NCL-AUGMENTED TSFMS

Related papers

Loading 2610.06453v1…