PaperlayerView exact revision on arXiv

Source-linked AI summary

RealNet: A Feature Selection Network with Realistic Synthetic Anomaly for Anomaly Detection

Ximiao Zhang, Min Xu, Xiuzhuang Zhou

arXiv:2403.05897v1cs.CV

TL;DR

RealNet targets unrealistic and insufficiently diverse synthetic anomalies, along with redundancy and pre-training bias in feature reconstruction. It combines diffusion-based anomaly synthesis with adaptive feature and residual selection, and experiments report significant Image AUROC and Pixel AUROC improvements over state-of-the-art methods.

  • Problem

    Self-supervised feature reconstruction methods need more realistic, diverse anomaly samples and better management of feature redundancy and pre-training bias.

  • Method

    RealNet combines strength-controllable diffusion anomaly synthesis with anomaly-aware feature selection and reconstruction residual selection.

  • Results

    Experiments on four benchmark datasets demonstrate significant improvements in Image AUROC and Pixel AUROC over current state-of-the-art methods.

  • Takeaways & Limitations

    RealNet provides a flexible feature-reconstruction framework that exploits large-scale pre-trained models while keeping computational overhead reasonably low.

  • Takeaways & Limitations

    RealNet avoids multi-scale feature alignment because down-sampling may reduce detection resolution, while up-sampling may increase feature redundancy and reconstruction costs.

Abstract

from arXiv · show

Self-supervised feature reconstruction methods have shown promising advances in industrial image anomaly detection and localization. Despite this progress, these methods still face challenges in synthesizing realistic and diverse anomaly samples, as well as addressing the feature redundancy and pre-training bias of pre-trained feature. In this work, we introduce RealNet, a feature reconstruction network with realistic synthetic anomaly and adaptive feature selection. It is incorporated with three key innovations: First, we propose Strength-controllable Diffusion Anomaly Synthesis (SDAS), a diffusion process-based synthesis strategy capable of generating samples with varying anomaly strengths that mimic the distribution of real anomalous samples. Second, we develop Anomaly-aware Features Selection (AFS), a method for selecting representative and discriminative pre-trained feature subsets to improve anomaly detection performance while controlling computational costs. Third, we introduce Reconstruction Residuals Selection (RRS), a strategy that adaptively selects discriminative residuals for comprehensive identification of anomalous regions across multiple levels of granularity. We assess RealNet on four benchmark datasets, and our results demonstrate significant improvements in both Image AUROC and Pixel AUROC compared to the current state-o-the-art methods. The code, data, and models are available at https://github.com/cnulab/RealNet.

1. Introduction

RealNet addresses limitations in self-supervised industrial anomaly detection by synthesizing more realistic anomalies and selecting informative pre-trained features and residuals.

  • Self-supervised anomaly detection supports industrial quality control and safety monitoring but struggles to synthesize realistic, diverse anomalies.Existing methods face particular difficulty with complex structural anomalies and unseen anomaly categories.
  • SDAS generates diverse anomalous samples from normal-image distributions while controlling anomaly strength.It uses DDPM-based perturbations in low-probability-density regions to simulate aging, structural, texture, and color changes.
  • Existing feature-reconstruction methods face high computational demands, feature redundancy, and pre-training bias.They often use small-scale CNNs or manually selected layer-specific features, while optimal feature subsets vary across categories.
  • RealNet combines AFS and RRS to exploit large-scale pre-trained CNN features while reducing redundancy and selecting informative reconstruction residuals.The framework is designed to improve anomaly detection while controlling computational demands.

2. Related work

Related work spans unsupervised anomaly detection paradigms, with RealNet positioned at the intersection of self-supervised synthesis and pre-trained feature reconstruction.

  • Unsupervised anomaly detection methods train using only normal images and include reconstruction, self-supervised, deep feature embedding, and one-class classification approaches.RealNet focuses on reconstruction-based and self-supervised learning-based methods.
  • Reconstruction-based methods detect anomalies by comparing inputs with reconstructions trained on normal images.Autoencoder, GAN, Transformer, and Diffusion models have all been explored for reconstruction.
  • Pre-trained multi-scale features improve detection across anomaly scales and patterns but introduce feature redundancy and pre-training bias.These issues motivate feature-selection strategies in feature reconstruction methods.
  • Self-supervised methods create proxy anomalies through techniques such as patch transplantation, Poisson editing, and texture transfer.CutPaste, NSA, and DRAEM differ in continuity, naturalness, and coverage of structural anomalies.
  • RealNet combines SDAS, AFS, and RRS to synthesize near-natural anomalies, reduce feature dimensionality, reconstruct features, and select informative residuals.The pipeline feeds selected residuals into a discriminator for anomaly detection and localization.
  • SDAS differs from prior image-editing approaches by operating in probability space and not requiring real anomalous images.The supplied passage introduces this contrast but is truncated before completing the comparison.

3. Method

RealNet combines diffusion-based anomaly synthesis with adaptive selection of pre-trained features and reconstruction residuals. Its modules target realistic anomaly variation, reduced feature redundancy, and multi-scale anomaly identification.

  • Framework: RealNet comprises SDAS, AFS, and RRS for synthetic anomaly generation, pre-trained feature selection, and reconstruction residual selection.These components form the proposed feature reconstruction framework.
  • Strength-controllable Diffusion Anomaly Synthesis: SDAS trains a diffusion model on normal images, then adds a perturbation controlled by anomaly strength s to sample anomalous images near the normal distribution.The scalar s satisfies s ≥ 0 and controls the added variance during reverse diffusion.
  • Strength-controllable Diffusion Anomaly Synthesis: SDAS generates normal images when s = 0 and increasingly distribution-shifted images as s grows.The method uses s to control the difference between generated images and normal-image distribution.
  • Strength-controllable Diffusion Anomaly Synthesis: During training, SDAS anomalies are combined with normal images using Perlin-noise masks, image blending, and foreground segmentation to create local anomaly regions.The normal image is I, the SDAS image is P, and the blended image is A.
  • Anomaly-aware Features Selection: AFS evaluates feature maps by comparing normalized squared feature differences with anomaly masks, then selects discriminative subsets across available layers.The selected feature maps support reconstruction while reducing dimensionality, feature redundancy, pre-training bias, and reconstruction costs.
  • Reconstruction Residuals Selection: RRS selects informative residuals and concatenates local maximum and global average pooled representations to capture anomalies across spatial scales.Emax emphasizes small local anomalies, whereas Eavg emphasizes anomalies spanning larger areas; the combined representation is passed to a discriminator.

4. Experiment

RealNet is evaluated across four industrial anomaly-detection datasets using consistent architectures and standard image- and pixel-level metrics. Results show strong performance, while ablations support SDAS, AFS, and RRS design choices.

  • Experimental setup: Experiments cover MVTec-AD, MPDD, BTAD, and VisA, including challenging variations in products, structures, backgrounds, and object multiplicity.The evaluation uses Image AUROC, Pixel AUROC, and PRO for image-level detection and pixel-level localization.
  • MVTec-AD results: RealNet with SIA achieves 99.65% Image AUROC, 99.03% Pixel AUROC, and 93.07% PRO on MVTec-AD.It also reaches 31.93 FPS on one Nvidia GeForce RTX 3090 using 4GB of GPU memory.
  • Comparisons: RealNet outperforms feature-embedding, normalizing-flow, and previous reconstruction-based alternatives when using the same pre-trained network.The compared methods include PatchCore, SimpleNet, and FastFlow.
  • MPDD results: On MPDD, RealNet achieves 96.3% Image AUROC, surpassing CFlow by 10.2% without dataset-specific tuning.SIA also improves Image AUROC by 2.88% over DTD on MPDD.
  • Other benchmarks: On VisA, RealNet reaches 97.8% Image AUROC and 98.8% Pixel AUROC, while BTAD results are 96.1% and 97.9%, respectively.These results are reported for RealNet using SIA.
  • Ablation studies: Ablations show that RRS improves performance, AFS outperforms random dimensionality reduction and random linear projections, and uniformly sampled anomaly strength supports robust SDAS training.Max&Avg residual selection is more robust across anomaly scales, while extreme anomaly strengths degrade performance.

5. Conclusion

RealNet integrates realistic anomaly synthesis with adaptive feature and residual selection in a self-supervised anomaly detection framework. The combined design targets diverse anomaly challenges while keeping computational overhead reasonably low.

  • RealNet combines SDAS, AFS, and RRS as its three core components.The framework integrates diffusion-based anomaly synthesis, anomaly-aware feature selection, and reconstruction residual selection.
  • The framework enables effective use of large-scale pre-trained models for anomaly detection.
  • RealNet keeps computational overhead within a reasonably low and acceptable range.
  • Experiments demonstrate RealNet’s proficiency on diverse real-world anomaly detection challenges.

Anomaly Detection

The supplementary material organizes additional implementation details, dataset results, ablations, efficiency analyses, multi-class evaluations, synthetic-anomaly quality assessments, and visualizations.

  • Appendix B: Appendix B provides additional implementation details for RealNet.
  • Appendix C: Appendix C reports detailed BTAD and VisA results, supplementary ablations, computational-efficiency analysis, multi-class results, and synthetic-anomaly quality assessments.
  • Appendix D: Appendix D provides additional visualization results.

B. More details

The supplementary details describe SDAS variance modeling, reverse-diffusion options and settings, anomaly-strength selection, and residual normalization for RRS. They also report implementation choices and qualitative observations about generated anomalies.

  • SDAS: SDAS models reverse-diffusion variance with a learnable interpolation between diffusion-process and conditional-posterior variances in log space.The model predicts v, which weights β_t and β̃_t in log space.
  • SDAS: SDAS can use β_t, β̃_t, or the learned Σ_θ(x_t, t) as perturbation variance during deterministic DDIM reverse diffusion.
  • SDAS: IDDPM produces slightly better anomaly images than DDIM, so SDAS uses IDDPM in the experiments.
  • SDAS: Anomaly strength is set between 0.1 and 0.2 because higher strength introduces more noise and reduces generated-image authenticity.
  • RRS: RRS applies affine-free BatchNorm to global reconstruction residuals before selection to align distributions across residual dimensions.
  • Implementation: RealNet uses no additional anomaly-synthesis augmentation, balances normal and synthetic-anomalous images, and samples blending opacity δ uniformly from 0.5 to 1.0.

C.1. Experimental results on BTAD

On BTAD, RealNet achieves state-of-the-art anomaly detection and localization against alternative methods without structural or hyperparameter tuning. The evaluation uses Image AUROC and Pixel AUROC, while the supplementary material also presents SDAS algorithm details and VisA comparisons.

  • BTAD results: RealNet demonstrates state-of-the-art anomaly detection and localization performance on BTAD without structural or hyperparameter tuning.
  • BTAD results: SIA does not significantly outperform DTD on BTAD because the dataset lacks complex structural anomalies across its three industrial products.
  • SDAS details: SDAS with DDIM is specified by an algorithm that samples from a reverse-diffusion Gaussian distribution using anomaly strength s and perturbation variance Σ.
  • BTAD results: BTAD evaluation compares methods using Image AUROC (%) and Pixel AUROC (%).
  • VisA results: The supplementary material compares RealNet with alternative methods on VisA using Image AUROC (%) and Pixel AUROC (%).

C.2. Experimental results on VisA

On VisA under the one-class protocol, RealNet achieves the best reported anomaly detection and localization performance, improving over DTD.

  • RealNet achieves the best performance in both anomaly detection and localization on VisA under the one-class protocol.
  • 1.5% improvement in Image AUROC and 0.4% improvement in Pixel AUROC over DTD are reported for RealNet trained with SIA.

C.3. Supplementary ablation studies

RealNet remains competitive across backbone and reconstruction-feature settings on MVTec-AD, with only a slight Image AUROC decrease after reducing feature dimensions.

  • 0.16% decrease in Image AUROC occurs when WideResNet50 reconstruction dimensions are reduced from {256, 512, 512, 256} to {128, 256, 256, 128}.
  • RealNet maintains competitive performance after reducing reconstruction feature dimensions despite the slight Image AUROC decrease.
  • EfficientNetB4 and ResNet34 backbones also produce competitive performance, demonstrating effectiveness across settings.

C.4. Computational efficiency analysis

The analysis compares multi-scale reconstruction architectures, showing that separate or neighboring-aligned reconstruction can balance speed, model size, and localization accuracy, while full alignment is costly and prone to overfitting.

  • Architecture comparison: Architecture A uses separate U-Net reconstruction networks for multi-scale features without interpolation or alignment.The common U-Net uses residual and down-sampling layers followed by residual and up-sampling layers with skip connections.
  • Architecture comparison: 31.93 FPS, 2.2 GB, 99.65% Image AUROC, and 99.03% Pixel AUROC are achieved by architecture A at 256 × 256 resolution.
  • Architecture comparison: 40.42 FPS and 0.74 GB are obtained after reducing reconstruction dimensions, while Image AUROC remains 99.49% and Pixel AUROC 99.07%.
  • Architecture comparison: Architecture B aligns interpolated features, reducing computational efficiency and increasing model size; its large reconstruction network is also prone to overfitting.
  • Architecture comparison: Architecture C aligns and reconstructs neighboring-resolution features to reduce reconstruction networks and balance computational efficiency with localization accuracy.Its reported PRO score reaches 94.71%.
  • Additional evaluations: Across multi-class MVTec-AD detection, RealNet achieves 97.3% Image AUROC and 98.4% Pixel AUROC with ResNet50.
  • Additional evaluations: SIA achieves the best FID and LPIPS among evaluated synthesis methods, supporting realistic and diverse anomaly generation.FID evaluates realism and diversity with lower values preferred, while cluster LPIPS measures diversity with higher values preferred.

E. Limitations

The reported limitations concern texture-heavy anomaly synthesis, additional offline training time, and reduced localization resolution when using multiple reconstruction networks.

  • SIA may slightly underperform DTD on categories dominated by texture anomalies in MVTec-AD.DTD contains diverse real-world texture images that simulate common texture anomalies such as color, oil, and glue.
  • SDAS adds offline training time, requiring 6 hours on one RTX 3090 to generate 10,000 256×256 anomaly images per category.SDAS is omitted during inference and real-world applications.
  • Not upsampling multi-scale features improves computational efficiency but may reduce PRO by introducing deviations when localizing small anomalous areas.Architecture C achieves a reported PRO score of 94.71% after reducing the number of reconstruction networks.
Loading 2403.05897v1…