PaperlayerView exact revision on arXiv

Source-linked AI summary

InCLAD: A Continual Learning Benchmark for Industrial Visual Anomaly Detection

Łukasz Marcjan, Dorota Wilk-Kołodziejczyk, Roberto Corizzo

arXiv:2610.15032v1cs.LGcs.AI

TL;DR

Industrial anomaly-detection benchmarks are largely static, leaving limited evidence on how normal-only detectors retain performance as they learn new components. InCLAD defines component-level task streams and evaluates continual detection and localization across curricula and multi-dataset scenarios; its within-dataset continual variants improve both ROC-AUC and BWT over non-continual variants.

  • Problem

    Static industrial anomaly-detection evaluations leave open how normal-only models adapt to new components while retaining performance on earlier tasks.

    Source: §1 · p. 1, §2 · p. 4

  • Method

    InCLAD adds task definitions, orderings, metrics, and evaluation artifacts to public industrial datasets, evaluating normal-only component streams across curricula and multi-dataset settings.

    Source: §1 · p. 2, §3 · p. 6

  • Results

    Across within-dataset streams, continual variants raise image-level ROC-AUC from 0.588 to 0.737 and pixel-level ROC-AUC from 0.718 to 0.860, while moving BWT closer to zero.

    Source: §4.4 · p. 9

  • Takeaways & Limitations

    The benchmark provides a framework for studying adaptation and retention across component-level tasks, alternative orderings, and dataset shifts.

    Source: §1 · p. 2, §3 · pp. 6–7

    The benchmark’s public datasets only partially reflect production-line temporal, operational, and sensor variability, and its tasks do not capture every practical constraint.

    Source: §5 · p. 10

Abstract

from arXiv · show

Industrial anomaly detection benchmarks have advanced rapidly, yet they remain largely static and therefore underrepresent deployment settings in which inspection systems must be extended from one industrial component to another over time. In our setting, continual industrial anomaly detection is framed as a task-incremental problem: each component defines a task, while the learning objective remains semi-supervised. The goal extends beyond conventional anomaly detection, since models should preserve their performance on all components as they are sequentially exposed to new components. This framing is especially relevant because many existing industrial pipelines either assume fully supervised defect labels or study only isolated datasets, leaving open the question of how semi-supervised anomaly detectors behave across a sequence of heterogeneous components. We address this gap by organizing the benchmark as a sequence of related component-level tasks and by defining multiple scenarios, including random, easy-to-hard, and hard-to-easy. In addition to single-dataset continual scenarios, we propose challenging scenarios spanning across multiple datasets. We further evaluate continual performance from two complementary perspectives: image-level anomaly detection and pixel-level anomaly localization. By recasting industrial anomaly detection as a continual benchmark rather than a static train/test problem, our work establishes a concrete path toward studying knowledge retention, adaptation, and generalization in evolving industrial environments.

1 Introduction

Static industrial anomaly detection does not capture the need to extend inspection systems to new components while preserving performance on earlier ones. InCLAD frames this as normal-only, task-incremental learning and benchmarks adaptation and retention across component sequences.

1 Introduction

  • In deployment, detectors must adapt to newly introduced components without retraining from scratch on all historical data or losing competence on earlier components.Forgetting earlier component knowledge can cause missed anomalies when those products return to production.Source: §1 · pp. 1–2
  • The continual objective is to absorb component-specific normality patterns without degrading anomaly-detection performance on previously learned components.Training uses normal samples only; anomalies are detected at test time rather than learned as supervised defect classes.Source: §1 · pp. 1–2
  • InCLAD treats each component as a task and evaluates normal-only adaptation across random, easy-to-hard, hard-to-easy, and multi-dataset sequences.It assesses both image-level detection and pixel-level localization.Source: §1 · p. 2
  • The benchmark contribution includes continual adaptations of PatchCore, PaDiM, and CFA and evaluation of continual anomaly-detection methods across image-level and pixel-level perspectives.The scenarios include curriculum orderings and multi-dataset streams.Source: §1 · p. 2

2 Related Works

Industrial anomaly detection has strong static benchmarks and methods, while continual anomaly detection remains less systematically studied. InCLAD targets gaps in normal-only sequential learning, task ordering, cross-dataset streams, and joint detection-localization evaluation.

2 Related Works

  • Static industrial benchmarks typically evaluate independently within datasets, while existing broad anomaly benchmarking is not specific to industrial inspection or sequential learning.The cited industrial datasets include MVTec AD, MPDD, and VisA; ADBench systematizes evaluation across broader supervision regimes and dataset types.Source: §2 · p. 2
  • ReinAD remains a static benchmark, whereas InCLAD studies task-incremental continual learning with sequential components, normal-only training, and retention, forgetting, and adaptation under alternative task orderings.Source: §2 · p. 3
  • Non-continual detector families include patch-memory and distribution methods such as PatchCore and PaDiM, teacher–student methods such as STFPM, and flow-based FastFlow.The related-work discussion also covers reconstruction-based methods, GANomaly, and multimodal and weakly supervised extensions [Cheng et al., 2025, Wang et al., 2025, Sime et al., 2026].Source: §2 · p. 3
  • The literature rarely evaluates image-level detection and pixel-level localization together under sequential learning or measures retention after new industrial concepts arrive.These gaps leave the task-incremental, normal-only, multi-dataset setting open for study.Source: §2 · pp. 4–5
  • Prior continual anomaly-detection work spans online memory, replay, prompt-based adaptation, and incremental reconstruction, but often studies one dataset or scenario at a time.Some approaches also use synthetic anomalies or explicit defect-oriented signals, limiting their match to purely normal-only training.Source: §2 · p. 4
  • Continual industrial anomaly detection differs from closed-set classification because it is anomaly-centric, often normal-only, and may require localization as well as image-level decisions.The literature includes methods that incorporate synthetic anomalies or defect-oriented signals, and existing studies do not yet provide a cohesive analysis across diverse benchmarks and task orders.Source: §2 · p. 4

3 The InCLAD Benchmark

InCLAD adds task definitions, orderings, metrics, and evaluation artifacts to public industrial datasets. Its benchmark combines within-dataset curricula and a 27-task multi-dataset stream, evaluated for adaptation and retention at image and pixel levels.

3 The InCLAD Benchmark

  • InCLAD is a benchmark protocol layered over public industrial datasets, not a new raw-data collection.It contributes task definitions, orderings, metrics, and evaluation artifacts.Source: §3 · p. 5
  • Each task represents an inspected component or concept, with normal training samples and anomalous test samples; the task identity is not a defect label.Evaluation covers image-level anomaly detection and pixel-level localization, with shifts arising from components, datasets, acquisition conditions, or production evolution.Source: §3 · p. 5
  • Within each dataset, easy-to-hard and hard-to-easy curricula use concept difficulty estimated by average ranks across PatchCore, PaDiM, and CFA; random order provides a reference.The experts’ ROC-AUC values must exceed 0.5, and the same scenario streams are used for image-level and pixel-level evaluation.Source: §3 · p. 5, Figure 2 · p. 6
  • The multi-dataset scenario contains 27 tasks selected to retain easy, hard, and median concepts while limiting the influence of larger datasets.It has easy-to-hard and hard-to-easy orderings based on the same average-difficulty criterion.Source: §3 · p. 6
  • After each task, models are evaluated on all tasks seen so far; image- and pixel-level ROC-AUC and backward transfer measure performance and retention.Negative BWT indicates forgetting, while positive BWT indicates beneficial backward transfer.Source: §3 · pp. 6–7
  • The benchmark comprises 17 scenarios, with easy-to-hard and hard-to-easy orderings for each and an additional random ordering for single-dataset scenarios.The multi-dataset stress test evaluates the three highest-ranked methods from aggregate single-dataset performance: CFA, PaSTe, and PaDiM.Source: Table 1 · p. 7, §3 · p. 7

4 Experiments

The experiments compare continual and non-continual detectors across industrial datasets, task orderings, and image- and pixel-level evaluation. Continual variants improve detection and retention in within-dataset streams, while cross-dataset streams remain a stronger stress test.

4.1 Experimental Setup

  • Methods are evaluated on identical task streams, train/test splits, and ordering metadata, comparing naive baselines with continual variants at image and pixel levels.Continual versions use replay unless otherwise specified.Source: §4.1 · p. 7

4.2 Industrial Datasets

  • The benchmark uses public industrial datasets, including DAGM, MVTec AD, VisA, BTAD, and MPDD, which cover varied defects and inspection settings.BTAD and MPDD include pixel-level annotations; MPDD targets metallic industrial components.Source: §4.2 · pp. 7–8

4.3 Anomaly Detection Methods

  • The comparison includes nine non-continual methods spanning reconstruction, pretrained-feature and patch-distribution, teacher–student and distillation, and flow-based approaches.Six continual versions use replay; PatchCore, PaDiM, and CFA receive method-specific adaptations that preserve task-specific normality models.Source: §4.3 · p. 8

4.4 Results Discussion

  • Image-level ROC-AUC rises from 0.588 for NCL to 0.737 for CL, while pixel-level ROC-AUC rises from 0.718 to 0.860 and BWT moves closer to zero.Table 2 averages across methods and ResNet18/WideResNet backbones.Source: Table 2 · p. 9, §4.4 · p. 9
  • Easy-to-Hard has the highest image-level CL ROC-AUC (0.764), while pixel-level CL ROC-AUC is more stable across orderings, ranging from 0.849 to 0.870.Continual learning makes BWT less negative across all orderings, with especially large gains under Easy-to-Hard.Source: §4.4 · pp. 9–10
  • CL improves ROC-AUC on every dataset and ordering while moving BWT closer to zero; MPDD remains challenging for image-level detection despite strong pixel-level results.On BTAD under Easy-to-Hard, image-level BWT improves from −0.289 to −0.046 and pixel-level BWT from −0.247 to −0.010.Source: §4.4 · p. 10
  • In cross-dataset streams, CL raises image-level ROC-AUC to roughly 0.79 and pixel-level ROC-AUC above 0.90 while keeping BWT close to zero.Naive image-level detection is near chance under Easy-to-Hard and Hard-to-Easy, and the multi-dataset setting remains a stronger stress test than most within-dataset streams.Source: §4.4 · p. 10
  • Per-task heatmaps complement dataset-level averages by showing task trajectories after updates and where forgetting occurs, especially in off-diagonal cells.The reported retention improvement is more uniform for pixel-level localization.Source: §4.4 · p. 10

5 Conclusion

The paper presents a benchmark for continual industrial anomaly detection under evolving production conditions, evaluating retention, forgetting, and cross-dataset adaptation. Its scope is bounded by the limited temporal, operational, and sensor-level variability represented in public datasets.

5 Conclusion

  • The benchmark frames industrial inspection as a sequential setting in which models retain performance on seen components while adapting to new products, acquisition conditions, and dataset shifts.It evaluates multiple industrial datasets, task orderings, anomaly-detection perspectives, and continual-learning strategies.Source: §5 · p. 10
  • Public industrial datasets only partially reflect real production lines’ temporal, operational, and sensor-level variability, and sequenced tasks do not capture every deployment constraint.The cited examples include changing inspection hardware, evolving quality standards, and long-term label drift.Source: §5 · p. 10
  • Continual inspection may support adaptability and quality control, but benchmark progress is not a guarantee of deployment readiness.The paper notes risks including missed defects, false alarms, and accumulated errors, and recommends human-in-the-loop validation and periodic auditing.Source: §5 · p. 11

A Benchmark Comparison

InCLAD is presented as a continual-learning benchmark layer built on public industrial datasets, rather than as a new raw image corpus.

A Benchmark Comparison

  • InCLAD combines normal-only training, image-level detection, pixel-level localization, curriculum-style orderings, multi-dataset streams, and forgetting-aware metrics in one protocol.Table 3 compares representative anomaly-detection benchmarks; “Partial” denotes coverage limited to some datasets, tasks, or evaluation settings.Source: A Benchmark Comparison · p. 15, Table 3 · p. 15

B Benchmark Metrics

The benchmark evaluates learning, retention, and transfer across task streams, using image- and pixel-level ROC-AUC. Continual adaptations preserve task-specific normality models and select among them at inference.

B Benchmark Metrics

  • Continual ROC-AUC averages performance over the lower triangle of the results matrix, summarizing the full learning trajectory rather than only the final task.The matrix records each task's ROC-AUC after training up to each observed task; the metric is reported as ROC-AUC in experimental tables.Source: B Benchmark Metrics · p. 15
  • BWT measures how learning later tasks affects earlier-task scores; positive values indicate improvement, while negative values indicate forgetting.The comparison is against each earlier task's score immediately after it was learned.Source: B Benchmark Metrics · p. 15

C Continual Adaptations for PatchCore, PaDiM, and CFA

  • At evaluation, methods select among accumulated task-specific normality models without using task identity to choose the model.The final score or anomaly map is computed using the selected task model.Source: C Continual Adaptations for PatchCore,… · p. 16

C.1 Continual PatchCore

  • PatchCore keeps separate task banks and compresses previous banks under a history budget, then selects the bank with the closest task-level normality score.The current task retains its native coreset resolution; distance-weighted compression preferentially keeps patches far from the current distribution.Source: §C.1 · p. 16

C.2 Continual PaDiM

  • PaDiM fits and stores one location-wise Gaussian distribution per task, then selects the distribution with the lowest aggregated image-level anomaly score.Its inference produces a Mahalanobis map for each stored task distribution.Source: §C.2 · p. 17

C.3 Continual CFA

  • CFA separates its training bank from inference banks and rebuilds all task inference banks after descriptor updates to keep them in the current embedding space.The descriptor is trained on current-task and replay samples using a fixed training bank.Source: §C.3 · p. 17

D.1 Dataset Licenses and Scenario Release

The benchmark releases scenario manifests and code rather than raw dataset images or masks, respecting the source datasets' licenses. Experiments used an institutional HPC system and two local workstations.

D.1 Dataset Licenses and Scenario Release

  • Because several source-dataset licenses are non-commercial or sharealike, the release provides code, configurations, and scenario manifests instead of raw images or masks.Users obtain datasets from official sources and run scripts locally to materialize scenarios.Source: §D.1 · p. 18

D.2 Computational Resources

  • Experiments ran on an institutional Red Hat Enterprise Linux HPC system and two local high-performance workstations.The HPC resources included H100- and L40-equipped nodes and a 2 TB big-memory node.Source: §D.2 · p. 18

D.3 Empirical Time-Complexity

Runtime varies with dataset scale and model family: feature-bank methods can incur expensive evaluation, while gradient-trained methods place more cost in training. Continual variants generally add runtime through replay, accumulated models, or repeated evaluation.

D.3 Empirical Time-Complexity

  • Feature-bank methods such as PatchCore and PaDiM can have modest update costs but expensive evaluation, especially on larger streams such as VisA and DAGM.Evaluation compares test images against accumulated task-specific statistics or memory banks.Source: §D.3 · p. 18
  • Gradient-trained methods concentrate more cost in training, and continual variants usually increase runtime through replay, accumulated task models, or repeated evaluation.Evaluation-heavy methods on long streams show the effect most clearly, as separate task-specific models improve continual behavior while increasing scoring cost.Source: §D.3 · p. 19

D.4 Hyperparameter Configurations

  • The timing results are contextualized by reported hyperparameters, including multi-epoch CFA training and fixed pretrained features for PaDiM and PatchCore.Table 4 reports separate training and evaluation times as mean ± population standard deviation across available orderings; image and pixel run times are summed when run separately.Source: §D.4 · p. 19, Table 4 · p. 20

E Scenario Ordering Details

Task-difficulty agreement varies across datasets and detector families, so average-rank ordering avoids letting one detector's failure mode dominate the curriculum. Method-dependent difficulty also shapes how forgetting and backward transfer should be interpreted.

E Scenario Ordering Details

  • Average-rank ordering prevents curricula from being dominated by one detector's failure mode, especially where detector agreement is weak, as on MPDD.On MPDD, PaDiM and CFA are negatively correlated, and metal_plate is easy for CFA but difficult for PaDiM; weak agreement also appears for parts of MVTec AD and between CFA and other detectors on DAGM.Source: E Scenario Ordering Details · p. 21
  • Weak agreement makes streams useful stress tests because forgetting and backward transfer depend partly on whether a method's inductive bias matches each concept.VisA and BTAD have stronger agreement and comparatively stable difficulty across model families.Source: E Scenario Ordering Details · p. 21

F Fine-grained Heatmap Results

The heatmaps show that continual learning improves retained-task performance over non-continual learning, while revealing task-order effects and differences between image-level detection and pixel-level localization. In multi-dataset streams, this retention pattern persists across backbones despite broader domain shifts.

Fine-grained heatmap results

Ordering effects

  • NCL forgetting is task-local and ordering-sensitive: later updates darken vulnerable early-task cells, while CL reduces contrasts among sequence orderings.Random streams generally fall between the hard-to-easy and easy-to-hard directions.Source: F Fine-grained Heatmap Results · p. 22

Adaptation versus retention

  • Pixel-level heatmaps are generally warmer and more stable than image-level heatmaps, separating strong localization from less robust image-level anomaly scores.MPDD, MVTec, and VisA particularly show high, relatively uniform localization under CL, while DAGM retains class-dependent variability.Source: F Fine-grained Heatmap Results · p. 22

Multi-dataset streams

Loading 2610.15032v1…