PaperlayerView exact revision on arXiv

Source-linked AI summary

Neural-Collapse-guided Task-Free Continual Anomaly Detection

Xiaotong Kong, Chaoyang Song, Ziai Zhou, Jinxia Zhang, Kanjian Zhang, Haikun Wei

arXiv:2609.03406v1cs.CV

TL;DR

Industrial anomaly detection lacks task boundaries and real anomaly supervision in evolving streams, limiting conventional continual-learning assumptions. NC-TFAD uses fixed ETF geometry, synthetic anomaly anchors, continual regularization, and prototype-guided localization; experiments on MVTec AD and VisA report strong image- and pixel-level performance. Its scope remains bounded by synthetic-anomaly fidelity, benchmark-only evaluation, challenging stream compositions, and calibration issues under severe imbalance.

  • Problem

    Task-free continual anomaly detection must handle evolving industrial streams without task boundaries, historical data, or real anomaly supervision.

  • Method

    NC-TFAD freezes a pretrained backbone, aligns features to fixed ETF normal–anomaly prototypes, uses synthetic anomaly anchors and FNCC-based regularization, and localizes defects with calibrated normal-patch deviations plus attention.

  • Results

    NC-TFAD achieves strong image-level detection and competitive pixel-level localization on MVTec AD and VisA under the task-free continual learning protocol.

  • Takeaways & Limitations

    Geometry-driven modeling provides a supported approach for stabilizing representations and preserving discriminative capability as industrial data distributions evolve.

  • Takeaways & Limitations

    The method depends on synthetic anomalies that cannot fully capture real defect diversity, is evaluated only on MVTec AD and VisA, and faces calibration challenges under severe imbalance.

Abstract

from arXiv · show

Recent years have witnessed growing interest in continual anomaly detection for industrial visual inspection. However, real-world manufacturing environments exhibit unpredictable shifts in data distributions, rendering task-dependent continual learning assumptions impractical. To address this limitation, we formulate industrial anomaly detection as a task-free continual learning problem and propose NC-TFAD, a neural-collapse-inspired, geometry-driven framework for learning from non-stationary data streams without task boundaries. NC-TFAD freezes a pretrained backbone and aligns streaming features to a simplex Equiangular Tight Frame (ETF) prototype space to stabilize representation geometry under non-stationary streams. To satisfy the NC-inspired geometric construction in the absence of real anomalies, we generate synthetic anomaly samples as auxiliary anchors during training. Building on this geometry, we further introduce inter- and intra-class regularization together with a Focal Neural Collapse Contrastive (FNCC) loss to suppress representation drift and improve normal-anomaly separability. Finally, a normal-patch-prototype-guided localization branch constructs calibrated patch-wise deviation maps from normal training samples and fuses them with a weak self-attention prior, producing anomaly heatmaps without pixel-level annotations. Extensive experiments on MVTec AD and VisA show that NC-TFAD consistently outperforms representative task-free continual learning methods adapted from general vision, as well as unified anomaly detection baselines, in both image-level detection and pixel-level localization under the task-free continual learning protocol. These results highlight that geometry-driven modeling offers an effective and robust solution for task-free continual anomaly detection in real-world industrial applications.

I. INTRODUCTION

Industrial anomaly detection must operate under evolving, task-free streams because conventional methods rely on joint training, task information, or anomaly supervision unavailable in deployment. NC-TFAD addresses this setting with fixed ETF geometry, synthetic anomaly anchors, continual regularization, and prototype-guided localization.

  • Industrial inspection requires high precision, low latency, and reliable automation, but anomalies vary widely and manual inspection is inefficient and inconsistent.
  • Conventional unified anomaly detection assumes jointly available training data, while continual methods often rely on task boundaries or identifiers absent from manufacturing streams.
  • Task-free continual anomaly detection remains difficult because general TFCL methods are not tailored to subtle anomalies and anomaly supervision is unavailable.
  • NC-TFAD freezes a pretrained backbone and aligns streaming features to a category-agnostic simplex ETF prototype space using two shared normal–anomaly directions.
  • Synthetic anomalies provide auxiliary geometric anchors and weak image-level supervision for the otherwise unobserved anomalous direction, without modeling the full defect distribution.
  • Extensive experiments on MVTec AD and VisA report that NC-TFAD significantly outperforms existing state-of-the-art approaches under task-free continual anomaly detection.
  • NC-TFAD combines NC-guided inter- and intra-class regularization with FNCC and fuses calibrated patch deviations with weak self-attention for localization without pixel-level supervision.

B. Task-Free Continual Learning

Task-free continual anomaly detection processes single-pass normal streams without task boundaries, historical replay, or real anomaly labels. The paper uses neural-collapse geometry as an active inductive bias, representing normal and anomalous states with maximally separated ETF prototypes.

  • B. Task-Free Continual Learning: Task-free continual learning models continuously arriving data under unknown distribution shifts without task boundaries or identifiers during training or inference.
  • B. Task-Free Continual Learning: Existing TFCL approaches include dynamically growing memory and lightweight tuning of frozen backbones, but TFCL remains underexplored for anomaly detection.
  • C. Neural Collapse: Neural collapse can function as an active geometric inductive bias that guides representation learning under non-stationary environments.
  • B. Task-Free Continual Learning: The TF-CAD protocol uses a single-pass stream of normal batches, generates synthetic anomalous counterparts, and updates only from current data without historical samples or task identifiers.
  • C. Neural Collapse: Neural collapse describes a highly symmetric feature-space structure in which class means and classifier weights converge toward an analytically characterized simplex ETF.
  • C. Neural Collapse: A simplex ETF gives prototypes equal norm and equal pairwise cosine −1/(K−1), producing maximal angular separation and minimal coherence on the unit hypersphere.
  • C. Neural Collapse: In the binary normal–anomaly setting, the simplex ETF yields two antipodal prototypes that define a fixed, maximally separated reference as streams distort representations.

B. Framework Overview

NC-TFAD uses a frozen backbone and learnable projection to align normalized [CLS] embeddings with fixed normal–anomaly ETF prototypes. A parallel patch-prototype branch produces localized anomaly maps by combining calibrated deviations with self-attention.

  • The framework trains on normal and synthetically perturbed images using a frozen backbone and learnable linear projection layer.
  • LayerNorm-normalized last-block [CLS] tokens are projected and ℓ2-normalized into an embedding space for geometric learning.The embeddings support alignment, regularization, and contrastive learning.
  • Two fixed ETF prototypes represent normal and anomalous states across all object categories, without category-specific prototypes.The prototype indices encode anomaly state rather than industrial object identity.
  • NC Loss, Reg Loss, and FNCC Loss jointly align embeddings, preserve normal–anomaly geometry, and emphasize hard sample relations.The fixed prototypes also serve as classifier weights during inference.
  • Localization compares test patches with a normal prototype bank and fuses calibrated deviation maps with pre-projection CLS-to-patch self-attention.The resulting fused response is used as the final anomaly map.

C. Synthetic Anomaly Generation

Synthetic anomalies provide auxiliary geometric anchors and weak image-level supervision when training contains only normal samples. The generation pipeline perturbs valid foreground regions, while the resulting images support global normal–anomaly separation without pixel-level supervision.

  • Synthetic Anomaly Generation: Synthetic anomalies supply an otherwise unavailable anomalous direction for NC-based geometry when real anomalies and annotations are absent.They are used exclusively as auxiliary geometric anchors during training.
  • Synthetic Anomaly Generation: Object foregrounds are estimated by thresholding and morphology, whereas entire texture images are treated as foreground because defects may occur anywhere.
  • Synthetic Anomaly Generation: Perlin noise defines irregular candidate regions that are intersected with foreground masks before local perturbations are applied.Regions outside the foreground are discarded.
  • Synthetic Anomaly Generation: Perturbed images train image-level geometric separation, while their spatial masks are discarded and never used as pixel-level supervision.Localized perturbations can still influence the global [CLS] representation through self-attention.
  • Inter-Class Alignment Regularization: NC-guided regularization uses fixed directional priors to align class means and maintain embedding geometry under streaming shifts.The inter-class term is omitted when fewer than two classes or the required prior are unavailable.

Intra-Class Compactness Regularization:

The intra-class compactness constraint strengthens within-class consistency by increasing similarity among same-class embeddings. It uses inverse-frequency weighting to address batch imbalance and does not require exact neural-collapse attainment.

  • Intra-Class Compactness Regularization: The constraint promotes within-class feature aggregation to improve consistency under distribution shift and noise.
  • Intra-Class Compactness Regularization: Pairwise similarities among distinct same-class embeddings are aggregated to define the compactness objective.
  • Intra-Class Compactness Regularization: With ζ = 1, inverse-frequency re-weighting increases sensitivity to minority anomalous classes and suppresses scattering from drift and noise.
  • Intra-Class Compactness Regularization: The overall regularizer balances inter-class alignment and intra-class compactness while using ETF directions as a fixed inductive bias.The embeddings need not reach an exact neural-collapse configuration.

E. Focal NC Contrastive Loss

FNCC improves sample-level discriminability within the NC geometry by emphasizing hard positive pairs and ambiguous prototype assignments. It combines pairwise and prototype-guided focal terms to strengthen cluster structure under continual distribution shifts.

  • E. Focal NC Contrastive Loss: FNCC targets samples near decision boundaries or far from target prototypes through focal contrastive learning.
  • E. Focal NC Contrastive Loss: The loss unifies sample–sample and sample–prototype relations in a normalized probability space using temperature-scaled similarities.
  • E. Focal NC Contrastive Loss: Positive pairs contain distinct same-class samples, and focal modulation assigns greater weight to low-confidence positive pairs.The pair term is set to zero when no positive pair exists.
  • Prototype-Assignment Term: The prototype-guided focal term strengthens sample–prototype alignment and encourages prototype-centered clusters.It is applied to samples whose labels map to available prototypes.
  • E. Focal NC Contrastive Loss: FNCC combines the pairwise and prototype-guided terms and emphasizes hard positive pairs and ambiguous prototype assignments.This design is intended to improve embedding discriminability and robustness under continual distribution shifts.

F. Anomaly Localization

The localization branch builds a normal patch prototype bank from streaming normal samples and scores test patches by deviation from their nearest prototype. Normal-distribution calibration then produces stable pixel-level anomaly maps without pixel-level annotations.

  • F. Anomaly Localization: Normal patch tokens from the frozen backbone are normalized and summarized into a compact prototype bank using only normal training samples.Online sampling followed by lightweight k-means clustering constructs the bank while excluding synthetic anomalies, test images, and pixel-level masks.
  • F. Anomaly Localization: Each test patch receives a cosine-distance deviation score from its nearest normal prototype.The method compares every normalized patch token with the prototype bank before computing the nearest-prototype distance.
  • F. Anomaly Localization: Training-derived normal-distance statistics calibrate patch deviations instead of using per-image min–max normalization.This avoids spuriously high responses from small within-image score variations and uses the normal training distribution as the reference.
  • F. Anomaly Localization: Calibrated patch responses are rearranged into a spatial grid and bilinearly upsampled to the input resolution.The resulting Sproto map is intended to improve pixel-level ranking stability under severe foreground–background imbalance.

Attention-Prior Hybrid Fusion:

The method enhances prototype-based localization with a weak self-attention prior and dual-map fusion. Connected-region filtering then preserves confident anomaly regions while extending spatial support and suppressing isolated responses.

  • Attention-Prior Hybrid Fusion:: Frozen-backbone CLS-to-patch self-attention is converted into a spatial map and fused with prototype deviations as a weak localization prior.Attention is fused across heads, rearranged spatially, and upsampled to obtain A ∈[0, 1]H×W.
  • Attention-Prior Hybrid Fusion:: Dual-map fusion preserves high-confidence anomaly responses while extending spatial support with lower-confidence regions.The support regions receive reduced confidence through ρ, limiting the influence of spatial expansion on strong anomaly responses.
  • Attention-Prior Hybrid Fusion:: Top-k connected-region filtering retains low-threshold components only when they overlap with confident high-threshold seeds.Responses outside retained support are attenuated, and this post-processing affects only pixel-level heatmaps.
  • Datasets and Evaluation: MVTec AD contains 15 categories, 4,096 normal images, and 1,258 anomalous images spanning 73 realistic defect types.The benchmark includes 10 object and 5 texture categories.
  • Datasets and Evaluation: Evaluation reports I-AUROC, I-AP, P-AP, and P-AUPRO to assess detection, localization, and region-level overlap under class imbalance.P-AUPRO complements pixel-level metrics by measuring overlap with individual ground-truth defect regions across thresholds.

C. Implementation Details

The implementation freezes a DINO ViT-S/8 backbone and evaluates task-free streams against continual-learning and industrial anomaly-detection baselines. NC-TFAD achieves the strongest reported MVTec average image-level performance and reaches 77.7% I-AUROC on VisA, while VisA I-AP remains lower than UCAD.

  • C. Implementation Details: A frozen DINO ViT-S/8 backbone produces 384-dimensional image features for a bias-free 384 →384 projection and NC-guided optimization.Only the projected normalized [CLS] representation is used for image-level detection, with 147,456 trainable parameters in the projection layer.
  • C. Implementation Details: The localization branch uses 64 normal patch prototypes from 5,000 normal patch tokens and calibrates nearest-prototype distances with the 95th-percentile normal distance.It introduces no trainable parameters and uses fixed attention-fusion settings.
  • C. Implementation Details: Category-contiguous stream segments create abrupt distribution shifts while category labels remain unavailable during training.Samples are shuffled within categories and concatenated according to a fixed random category order.
  • C. Implementation Details: Evaluation compares NC-TFAD with replay-, prompt-, geometry-, memory-, and parameter-efficient continual-learning methods alongside unified and continual industrial anomaly-detection baselines.The comparison includes ER, DualPrompt, L2P, MVP, FCA, DCM, DYSON, Online-LoRA, UniAD, UCAD, IUF, CDAD, and IB-IUMAD.
  • C. Implementation Details: Vision–language baselines use zero-shot or few-shot protocols that differ from the sequential, single-pass task-free continual-learning evaluation used here.The target stream is processed without task boundaries or category identifiers.
  • C. Implementation Details: NC-TFAD achieves the best average image-level detection performance on MVTec AD and consistently outperforms both baseline families.The result indicates that directly adapting either general continual-learning or industrial anomaly-detection paradigms is insufficient for the stated setting.
  • C. Implementation Details: FCA* and DYSON* use synthetic anomalies to satisfy their two-class training requirement.This protocol detail is explicitly noted for the NC-based continual-learning comparisons.
  • C. Implementation Details: NC-TFAD reaches 77.7% average I-AUROC on VisA, exceeding DYSON* by 5.3 percentage points, but its 45.3% average I-AP is below UCAD’s 64.06%.The reported discrepancy identifies precision–recall behavior under severe class imbalance as a limitation.

Localization Results:

NC-TFAD provides pixel-level anomaly localization through prototype-calibrated patch deviations fused with attention priors, while ablations show complementary benefits from geometric regularization and FNCC.

  • Evaluation protocol: Pixel-level evaluation compares methods that inherently support anomaly localization, avoiding architecture changes that would make comparisons depend on added localization modules.
  • Quantitative localization: 87.54% P-AUROC, 73.49% P-AUPRO, and 34.93% P-AP are NC-TFAD’s MVTec AD averages across the three reported localization metrics.
  • Quantitative localization: On VisA, NC-TFAD achieves 88.91% P-AUROC and 69.83% P-AUPRO, while sparse small defects keep P-AP comparatively low.The passage attributes the lower precision to severe pixel-level class imbalance and false-positive sensitivity.
  • Qualitative localization: Qualitative maps concentrate deviation responses around defects, while attention priors and connected-region filtering reduce isolated background responses and preserve coherent regions.Small or weakly contrasted defects remain challenging, particularly on VisA.
  • Ablation results: With NC regularization alone, MVTec AD I-AUROC rises from 75.5% to 81.2% and I-AP reaches 91.5%, whereas FNCC produces more prominent gains on VisA.
  • Ablation results: Removing both regularizers destabilizes the decision boundary, while increasing their strengths generally improves performance and the combined NC-plus-FNCC configuration performs best overall.On MVTec AD, λinter = 0.9 and λintra = 0.5 yield 81.2% I-AUROC and 91.5% I-AP; VisA peaks at 76.2% I-AUROC and 43.9% I-AP in a medium-to-high regime.

Sensitivity to the Contrastive Temperature Parameter:

Temperature sensitivity is substantial without NC-guided regularization but becomes more stable after jointly adding NC regularization and FNCC. Additional diagnostics indicate robustness across minibatch compositions and frozen backbones, alongside improved geometry and retention.

  • Temperature role: The FNCC temperature τ controls similarity-distribution smoothness and therefore the sharpness of sample decision boundaries.
  • Temperature sensitivity: Without NC regularization, increasing τ from 0.05 to 0.20 raises VisA I-AUROC from 72.0% to 77.4% and I-AP from 39.7% to 44.7%, but τ ≥0.30 degrades performance.
  • Temperature sensitivity: With NC regularization and FNCC jointly enabled, performance remains consistently high for 0.1 ≤τ ≤0.3 across both datasets.At τ = 0.1, results reach 82.1% / 91.8% on MVTec AD and 77.7% / 45.3% on VisA for I-AUROC / I-AP.
  • Stream diagnostics: NC-TFAD is generally robust to minibatch composition, although homogeneous batches are consistently weaker than random and balanced batches on VisA.
  • Temporal retention: The combined objectives improve final and old-category performance over the base model, indicating stronger retention alongside end-of-stream detection.
  • Representation geometry dynamics: NC-TFAD strengthens feature-to-ETF alignment and representation compactness during the stream, especially on VisA and for real anomalous samples.The geometry is evaluated using read-only checkpoints and offline statistics unavailable to the model during optimization.

Effect of Different Backbones:

The ablation compares synthetic-anomaly strategies under the task-free continual learning protocol, while the broader evaluation examines geometry, localization, and benchmark limitations.

  • Synthetic anomaly strategies: Local structural perturbations outperform simple appearance or blur transformations, whereas global perturbations substantially degrade image-level detection.
  • Synthetic anomaly strategies: The full foreground-constrained local mixed synthesis achieves the strongest overall performance across the evaluated synthetic-anomaly variants.All variants are trained with synthetic anomalies but evaluated exclusively on real anomalous test images.
  • Limitations and future work: The method remains bounded by synthetic anomalies, benchmark-only evaluation, challenging VisA stream compositions, and unresolved calibration under severe class imbalance.Future work targets adaptive synthesis, realistic temporal shifts, improved calibration, and broader localization mechanisms.
  • Framework design: NC-TFAD establishes a category-agnostic normal–anomaly reference with fixed ETF prototypes and geometric constraints for non-stationary streams.
  • Localization: Its localization branch combines calibrated patch deviations with a weak attention prior for pixel-level localization without pixel-level supervision.
Loading 2609.03406v1…