PaperlayerView exact revision on arXiv

Source-linked AI summary

Natural Synthetic Anomalies for Self-Supervised Anomaly Detection and Localization

Hannah M. Schlüter, Jeremy Tan, Benjamin Hou, Bernhard Kainz

arXiv:2109.15222v3cs.CV

TL;DR

The paper addresses anomaly detection and localization when training data contain only normal examples and anomaly types are unknown. It introduces Natural Synthetic Anomalies, which uses Poisson-edited patches to train an end-to-end model. NSA achieves 97.2 AUROC for detection on MVTec AD without additional datasets and also performs strongly on medical images.

  • Problem

    Detecting and localizing subtle, rare, and unknown anomalies from normal-only training data is difficult, especially when annotated anomalous examples are costly to obtain.

  • Method

    NSA trains an end-to-end anomaly detector and localizer using diverse synthetic anomalies created by seamlessly blending transformed image patches with Poisson image editing.

  • Results

    97.2 AUROC detection and 96.3 AUROC localization are reported on MVTec AD among methods using no additional datasets.

  • Takeaways & Limitations

    NSA generalizes from synthetic training anomalies to real manufacturing defects and supports adaptation to medical imaging without ImageNet pre-training.

  • Takeaways & Limitations

    NSA’s patch-selection hyperparameters require assumptions because the real out-distribution is unknown during training.

Abstract

from arXiv · show

We introduce a simple and intuitive self-supervision task, Natural Synthetic Anomalies (NSA), for training an end-to-end model for anomaly detection and localization using only normal training data. NSA integrates Poisson image editing to seamlessly blend scaled patches of various sizes from separate images. This creates a wide range of synthetic anomalies which are more similar to natural sub-image irregularities than previous data-augmentation strategies for self-supervised anomaly detection. We evaluate the proposed method using natural and medical images. Our experiments with the MVTec AD dataset show that a model trained to localize NSA anomalies generalizes well to detecting real-world a priori unknown types of manufacturing defects. Our method achieves an overall detection AUROC of 97.2 outperforming all previous methods that learn without the use of additional datasets. Code available at https://github.com/hmsch/natural-synthetic-anomalies.

1 Introduction

The introduction frames unknown-anomaly detection from normal-only data as difficult because anomalies are rare, subtle, and costly to annotate. NSA addresses this setting with diverse, realistic synthetic anomalies and reports strong performance on industrial and medical imagery.

  • Motivation: Normal-only anomaly detection must identify and localize unknown anomalies without seeing anomalous examples during training.Subtle or rare irregularities may be difficult for humans to detect, while precise annotations require time and expertise.
  • Problem: A central challenge is designing training tasks that learn anomaly-relevant features without prior knowledge of expected anomaly types.Many strong MVTec AD approaches instead rely on features from ImageNet-pretrained models.
  • NSA contribution: NSA generates more natural and diverse anomalies than several prior augmentation methods by using Poisson editing, patch transformations, and background constraints.Its pixel-level labels come from differences to the normal image rather than interpolation factors.
  • Evaluation: 96.3 AUROC localization and 97.2 AUROC detection are reported on MVTec AD among methods using no additional datasets.The benchmark covers 10 object and 5 texture classes with normal training data and normal and anomalous test data.
  • Evaluation: NSA performs comparably to methods using additional data and much larger models while training only on MVTec AD data.MVTec AD provides 60–391 training images per class, far less data than ImageNet.
  • Evaluation: Without ImageNet pre-training, NSA is adapted beyond natural images and outperforms other state-of-the-art self-supervised methods for disease detection on curated chest X-rays.This evaluation supports the method’s applicability to medical imaging.

2 Related Work

Related work includes reconstruction-based scoring, proxy-task self-supervision, and synthetic sub-image anomalies. NSA builds on end-to-end self-supervised anomaly detection and uses Poisson image editing for seamless patch insertion.

  • Reconstruction-based methods: Reconstruction-based methods derive image- and pixel-level anomaly scores from reconstruction errors using generative models trained on normal data.Examples include VAEs, Bayesian autoencoders, GANs, and VQ-VAEs.
  • Self-supervision: Self-supervised proxy tasks can learn useful downstream features from unlabeled data by predicting patch positions or geometric transformations.Prior work uses these features to discover object classes or support sub-image anomaly detection.
  • Synthetic anomalies: Interpolation-based methods blend foreign patches into images and use the predicted interpolation factor as an anomaly score without further training.NSA employs this general self-supervised setup for its anomaly task.
  • Synthetic anomalies: DRAEM reconstructs images after removing foreign patches, then compares the input and reconstruction to produce an anomaly map.This differs from methods that directly predict the synthetic-anomaly signal.
  • Poisson image editing: Poisson image editing formulates seamless cloning as solving for an interpolant whose gradients follow a guidance field while matching destination boundary values.The guidance field may use source gradients or a mixture selected from source and destination gradients.
  • Poisson image editing: The Poisson formulation is solved numerically with finite differences, and seamless cloning is implemented through OpenCV in the self-supervised task.A related Poisson image-editing approach has also been used for medical-image anomaly detection.

3 NSA Self-supervised task

NSA trains anomaly localization by blending patches from normal images into other normal images, using constraints and intensity-based labels to create diverse synthetic anomalies. Poisson blending and varied patch sampling make the training examples more realistic and diverse than simpler augmentation strategies.

  • Synthetic anomaly generation: NSA creates proxy training samples by selecting a patch from one normal image, resizing it, and seamlessly blending it into another normal image.The procedure can optionally add multiple patches and then produces a pixel-wise label mask.
  • Patch sampling and constraints: Patch width and height are sampled from truncated Gamma distributions, combining mostly local anomalies with occasional larger or elongated irregularities.The bounds are selected relative to the object dimensions.
  • Patch sampling and constraints: Object-overlap constraints prevent synthetic patches from floating mainly in the background and require sufficient overlap between source and destination object regions.The procedure repeatedly resamples locations until the object and overlap thresholds are satisfied.
  • Pixel-wise labels: NSA derives pixel-wise labels from local intensity differences, using binary, continuous, or logistic forms rather than the interpolation factor used by FPI and PII.The authors state that intensity-based labels provide a more consistent training signal.
  • Training objective: Bounded labels use binary cross-entropy, while unbounded continuous labels use mean squared error for the encoder-decoder output.The objective is defined over pixel-wise predictions from a deep convolutional encoder-decoder.
  • Synthetic anomaly generation: Varying patch size, aspect ratio, location, resizing scale, shape, texture, and color creates diverse anomalies while avoiding obvious discontinuities.Blending patches containing background can also simulate missing components and preserve the overall image distribution.

4 Experiments

Experiments compare NSA with self-supervised anomaly methods on MVTec AD and evaluate its implementation and localization behavior. NSA achieves strong image- and pixel-level performance, while failure cases and dataset limitations constrain interpretation.

  • Experimental setup: The encoder-decoder models use ResNet-18 encoders, ResNet-based decoders, and task-dependent sigmoid or ReLU outputs with binary-crossentropy or mean squared error.Training uses normal-data-derived synthetic examples and evaluates AUROC across MVTec AD and a curated chest X-ray subset.
  • MVTec AD results: 97.2 image-level AUROC: NSA (logistic) outperforms CutPaste (3-way) by 2.0 on MVTec AD without additional datasets.A single NSA model also exceeds an ensemble of five CutPaste models and matches an ImageNet-pretrained comparison.
  • MVTec AD results: 96.3 localization AUROC: NSA achieves state-of-the-art MVTec AD localization among methods without additional datasets.DRAEM uses additional data and slightly outperforms NSA overall, while NSA remains comparable for many classes.
  • Synthetic anomaly design: Poisson-blended anomalies outperform simpler CutPaste and FPI augmentations, with patch shifting and resizing improving performance especially for object classes.The authors relate lower AUROC to weaker similarity between synthetic training anomalies and real test anomalies.
  • Synthetic anomaly design: Continuous labels represent anomaly degree, while bounded continuous labels outperform binary labels most for classes with high inherent variation.Unbounded continuous labels make training less stable and produce higher standard error.
  • Failure cases and limitations: Localization is imperfect: models can miss small defects, overpredict regions, produce false positives, or disagree with broad human annotations.Chest X-ray localization also lacks pixel-level metrics because fewer than 10% of test images have very rough bounding boxes.

5 Conclusion

NSA creates diverse and realistic synthetic anomalies under controlled conditions for self-supervised detection and localization. The method is reported to improve detection of real anomalies and generalize across natural and medical imaging datasets.

  • NSA creates diverse and realistic synthetic anomalies for self-supervised anomaly detection and localization.The task uses controlled training examples, an effective loss formulation, and synthetic labels.
  • NSA improves detection of real anomalies and generalizes across natural and medical imaging datasets.
  • Future extensions could quantify uncertainty or exploit known anomaly classes for critical applications.

A.1 Hyperparameters

The experiments use self-supervised-task hyperparameters that encode assumptions about unknown real anomalies rather than data-driven validation. Poisson blending choices also vary by data type to avoid artifacts near sharp object-background boundaries.

  • Patch count, relative dimensions, background brightness, and object and overlap thresholds control synthetic-anomaly generation.nmax controls the maximum number of patches, while dimension bounds are relative to image size.
  • The self-supervised-task hyperparameters encode assumptions about the unknown real out-distribution rather than validation-set tuning.The assumptions were chosen through visual inspection of input images and self-supervised examples.
  • NSA uses mixed gradients for rCXR data and MVTec AD textures, but source gradients for MVTec AD object classes.The object-class choice avoids artifacts caused by sharp contrast changes near patch boundaries.

A.2 Comparison of self-supervised tasks

The supplementary comparison organizes CutPaste, FPI, PII, and NSA by their self-supervised tasks and patch-selection procedures. The authors use their own patch-selection procedure for reimplemented baselines to support method comparisons.

  • Table 5 compares the CutPaste, FPI, PII, and NSA self-supervised tasks.
  • Table 6 compares the original patch-selection procedures for CutPaste, FPI, PII, and NSA.The authors use their patch-selection procedure for their reimplementations of CutPaste, FPI, and PII.

B.1 Additional ablation studies

Additional ablations test foreground constraints, patch count, patch-selection procedure, and patch shape. The reported results favor foreground constraints in background-heavy classes, random patch counts, and the proposed patch-selection procedure, while patch shape has limited importance.

  • The final NSA variant outperforms all three tested variants on image-level and pixel-level AUROC.Results are reported in Table 7 across five random seeds.
  • A: Foreground constraints are most important for classes with substantial background, including screw and capsule.
  • B: Using a random number of patches performs slightly better than using a single patch per training example.
  • C: The proposed patch-selection procedure performs much better overall than the procedure described in CutPaste.The CutPaste-style variant samples area ratio, aspect ratio, and contained location using the listed procedures.
  • D: Patch shape is not important beyond diverse sizes and aspect ratios in these ablations.Poisson blending can make rectangular source patches produce non-rectangular anomalies.

B.2 Per-region overlap

AU-PRO0.3 measures per-region overlap up to a 30% false-positive rate, giving equal weight to connected anomalies regardless of size. The section defines the metric and contextualizes its localization comparisons on MVTec AD.

  • Metric definition: AU-PRO0.3 is the area under the per-region overlap curve for false-positive rates up to 30%.Per-region overlap is computed from connected components of ground-truth anomaly maps.
  • Metric definition: Per-region overlap averages the fraction of each connected anomalous component covered by predicted anomalous pixels.The metric averages overlap across label maps and their connected anomaly components.
  • Motivation: Unlike pixel-level AUROC, AU-PRO0.3 weights small and large anomalies equally.This weighting is intended to value precise localization of small anomalies as much as localization of large anomalies.
  • Evaluation: PaDiM provides a reference comparison but relies on ImageNet pretraining, making the comparison unfair to the method trained from scratch.The authors note that CutPaste and DRAEM did not report AU-PRO scores.
  • Evaluation: Table 8 reports AU-PRO0.3 defect-localization scores for NSA models and PaDiM on resampled MVTec AD images and masks.Scores include standard errors across five random seeds.
Loading 2109.15222v3…