PaperlayerView exact revision on arXiv

Source-linked AI summary

EfficientAD: Accurate Visual Anomaly Detection at Millisecond-Level Latencies

Kilian Batzner, Lars Heckler, Rebecca König

arXiv:2303.14535v3cs.CV

TL;DR

Industrial anomaly detection needs both strong accuracy and fast inference, particularly where runtime limits affect safety and production economics. EfficientAD combines efficient patch features, student–teacher and autoencoder-based detection, and specialized training losses; it reports strong detection and localization with 2 ms latency and more than 600 images per second. Fine-grained logical anomalies remain challenging, and the autoencoder requires training.

  • Problem

    Industrial anomaly detection must balance detection quality with strict runtime requirements, while logical anomalies involving invalid combinations of normal features are challenging.

  • Method

    EfficientAD combines a lightweight patch description network, student–teacher feature prediction with a loss-induced asymmetry, and an autoencoder for global logical-anomaly detection.

  • Results

    EfficientAD improves anomaly detection and localization on industrial benchmarks, achieving 2 ms latency and more than 600 images per second.

  • Takeaways & Limitations

    EfficientAD is presented as a suitable real-world solution and a baseline or foundation for future anomaly-detection research.

  • Takeaways & Limitations

    Fine-grained logical anomalies remain challenging, and detecting them would require traditional metrology methods.

Abstract

from arXiv · show

Detecting anomalies in images is an important task, especially in real-time computer vision applications. In this work, we focus on computational efficiency and propose a lightweight feature extractor that processes an image in less than a millisecond on a modern GPU. We then use a student-teacher approach to detect anomalous features. We train a student network to predict the extracted features of normal, i.e., anomaly-free training images. The detection of anomalies at test time is enabled by the student failing to predict their features. We propose a training loss that hinders the student from imitating the teacher feature extractor beyond the normal images. It allows us to drastically reduce the computational cost of the student-teacher model, while improving the detection of anomalous features. We furthermore address the detection of challenging logical anomalies that involve invalid combinations of normal local features, for example, a wrong ordering of objects. We detect these anomalies by efficiently incorporating an autoencoder that analyzes images globally. We evaluate our method, called EfficientAD, on 32 datasets from three industrial anomaly detection dataset collections. EfficientAD sets new standards for both the detection and the localization of anomalies. At a latency of two milliseconds and a throughput of six hundred images per second, it enables a fast handling of anomalies. Together with its low error rate, this makes it an economical solution for real-world applications and a fruitful basis for future research.

1. Introduction

EfficientAD targets industrial anomaly detection under strict runtime constraints by combining efficient feature extraction, a student–teacher model, and autoencoder-based logical-anomaly detection.

  • Motivation: State-of-the-art methods often trade computational efficiency for detection performance through ensembles, large backbones, or high-resolution inputs.Recent methods may process images at resolutions up to 768×768 pixels.
  • Motivation: Real-world anomaly detection requires low computational and economic cost because late detection can threaten safety, production rates, and viability.Industrial settings may impose strict runtime limits due to high production rates.
  • Method: EfficientAD uses a sub-millisecond feature extractor and trains a student to predict pretrained teacher features on normal images, exposing anomalies through prediction failures.A training loss induces asymmetry without increasing test-time computational cost or restricting architecture design.
  • Method: The method combines student–teacher detection of anomalous local features with an autoencoder that learns logical constraints and detects invalid arrangements of normal objects.The paper proposes calibrated combination of both detection results.
  • Results: 2 ms latency and more than 600 images per second accompany improved anomaly detection and localization on industrial benchmarks.The paper reports substantial state-of-the-art improvements while maintaining low computational requirements.
  • Contributions: EfficientAD introduces an efficient architecture, a training loss that improves detection without affecting inference runtime, and autoencoder-based logical-anomaly detection.The feature-extraction architecture is intended to accelerate extraction by an order of magnitude relative to recent methods.

2. Related Work

Related work spans industrial anomaly-detection benchmarks, feature-space outlier detection, student–teacher models, and generative reconstruction methods, while logical anomalies remain a distinct challenge.

  • Anomaly Detection Tasks: Visual anomaly detection serves applications including medical imaging, autonomous driving, and industrial inspection, but this work benchmarks RGB or gray-scale images without sequence conditioning.The evaluation uses industrial anomaly-detection datasets.
  • Anomaly Detection Tasks: MVTec AD uses normal-only training images and anomalous test images, reflecting settings where defect types and locations are unknown during development.Its 15 inspection scenarios each contain training and test sets.
  • Anomaly Detection Tasks: VisA and MVTec LOCO extend industrial benchmarks with empirically more challenging anomalies, including logical violations such as wrong object ordering or combinations.All three collections provide pixel-precise masks for evaluating anomaly localization.
  • Anomaly Detection Methods: Traditional algorithms can process images within a few milliseconds but may fail when requirements such as well-aligned objects are not met.Deep-learning methods have been shown to handle such cases more robustly.
  • Anomaly Detection Methods: Feature-space methods apply outlier detection or density estimation to pretrained CNN features, with approaches including Gaussians, normalizing flows, and k-nearest neighbors.PatchCore reduces kNN search cost using a clustered feature-vector database.
  • Anomaly Detection Methods: Student–teacher methods train students to mimic a frozen teacher on training images, using failures on unseen anomalies for detection.The paper compares EfficientAD with Asymmetric Student Teacher and the original student–teacher method.
  • Anomaly Detection Methods: Autoencoder-based methods detect anomalies through differences between input images and reconstructions, but inaccurate normal reconstructions can cause false positives.Generative adversarial networks are also used for anomaly detection.

3. Method

EfficientAD combines efficient patch descriptors with a lightweight student–teacher model and an autoencoder to detect local and logical anomalies. Its training losses and calibrated map combination target accurate detection while keeping inference efficient.

  • 3.1. Efficient Patch Descriptors: The patch description network uses four convolutional layers, 33×33 receptive fields, and fully convolutional processing to generate patch features for variable-size images.The PDN is distilled from a WideResNet-101 feature extractor while reducing feature-extraction cost.
  • 3.2. Lightweight Student–Teacher: EfficientAD uses the distilled PDN as both teacher and student, allowing the student–teacher pair to operate with low latency.The student is trained to predict teacher features on normal images, where mismatches on anomalous images support detection.
  • 3.2. Lightweight Student–Teacher: A pretraining-image penalty discourages the student from generalizing teacher imitation beyond the application’s normal training images.Random ImageNet images are sampled during student training to impose this penalty.
  • 3.3. Logical Anomaly Detection: EfficientAD combines local student–teacher and autoencoder-based anomaly maps after estimating their normal-image noise scales with validation quantiles.The combined map is useful when an anomaly appears in only one of the two maps.
  • 3.3. Logical Anomaly Detection: An autoencoder models global logical constraints, while the student predicts both teacher and autoencoder outputs to account for systematic reconstruction errors on normal images.The autoencoder processes the complete image through a 64-dimensional bottleneck; the student learns normal reconstruction errors without learning anomaly-specific errors.

4. Experiments

EfficientAD is evaluated across 32 scenarios from three industrial anomaly-detection collections using detection, localization, runtime, and ablation analyses. The experiments report strong anomaly performance alongside low computational requirements and assess contributions from normalization and training losses.

  • 4. Experiments: EfficientAD is compared with AST, DSR, FastFlow, GCAD, PatchCore, SimpleNet, and S–T across MVTec AD, VisA, and MVTec LOCO.The evaluation includes EfficientAD-S and EfficientAD-M variants and uses official implementations where available.
  • 4. Experiments: The evaluation measures image-level anomaly detection with AU-ROC and pixel-level localization with AU-PRO, using AU-sPRO for MVTec LOCO logical anomalies.Dataset-collection results average scenario means, while MVTec LOCO gives logical and structural anomalies equal weight.
  • 4. Experiments: EfficientAD’s MVTec LOCO lead is attributed equally to performance on logical and structural anomalies.Table 2 also reports mean anomaly-detection AU-ROC by dataset collection and anomaly type.
  • 4. Experiments: Latency comparisons account for parameters, FLOPs, and memory because parameter count and FLOPs can misrepresent convolutional runtime.The paper reports computational measurements to provide a more transparent comparison.
  • 4. Experiments: Quantile-based map normalization performs substantially better than the Gaussian-based baseline and is independent of the score distributions between qa and qb.The baseline is sensitive to the validation anomaly-score distribution.
  • 4. Experiments: The hard feature loss raises anomaly-detection AU-ROC by 1.0%, while the pretraining penalty further improves performance without increasing computational requirements.The ablation study evaluates cumulative and isolated contributions of the proposed techniques.

5. Conclusion

EfficientAD combines strong anomaly detection with high computational efficiency, addressing both structural and logical anomalies. Its main limitation is that fine-grained logical anomalies remain challenging and require traditional metrology methods.

  • Conclusion: EfficientAD-S and EfficientAD-M outperform other methods by a large margin on anomaly detection and localization.The comparison covers both detection and localization tasks.
  • Conclusion: EfficientAD-S reduces latency by a factor of 24 and increases throughput by a factor of 15 compared to AST.AST is identified as the second-best method in the comparison.
  • Conclusion: The method is suitable for real-world applications because of its low latency, high throughput, and high detection rate.The efficient patch description network can also serve as a feature extractor for other anomaly detection methods.
  • Limitations: Fine-grained logical anomalies remain a challenge, such as a screw that is two millimeters too long.Practitioners would have to use traditional metrology methods to detect these anomalies.
  • Limitations: Unlike kNN-based methods, EfficientAD requires training, including twenty minutes for the autoencoder to learn logical constraints in the reported setup.The stated training time is specific to the experimental setup.

Appendices

The supplementary material provides implementation details for EfficientAD and other evaluated methods, along with additional backbone and metric evaluations.

  • Supplementary material: The supplementary material includes EfficientAD training and inference procedures and patch description network distillation details.These materials are organized under sections A.1 and A.2.
  • Supplementary material: It provides implementation and configuration details for the other evaluated anomaly detection methods.These details are grouped separately from the EfficientAD implementation.
  • Supplementary material: Additional material evaluates EfficientAD across different distillation backbones and reports extra anomaly detection metrics.The listed examples include area under the precision-recall curve.

A.1. Training and Inference

EfficientAD training jointly optimizes a student–teacher model and an autoencoder, while inference combines local and global anomaly maps into image-level scores. The appendix specifies the architectures, normalization, augmentation, and preprocessing used by the pipeline.

  • Training: EfficientAD-S training uses a pretrained teacher, a randomly initialized student, and an autoencoder operating on 256 × 256 images.The teacher outputs 384 × 64 × 64 features; the student outputs 768 × 64 × 64 features.
  • Training: The student is trained against normalized teacher features, with hard-feature loss and autoencoder-related losses contributing to the total objective.The total loss is Ltotal = LST + LAE + LSTAE.
  • Training: Training augments images using random brightness, contrast, or saturation adjustments with coefficients sampled from U(0.8, 1.2).The autoencoder receives augmented images, while the student–teacher model is applied to original and augmented training images.
  • Inference: Inference computes student–teacher and autoencoder squared-difference maps, normalizes them using validation quantiles, and resizes them to the input resolution.The anomaly maps are resized to 256 × 256 before final processing.
  • Inference: The combined anomaly map averages normalized local and global maps, and the image-level score is the maximum map value.The combination is M = 0.5 ˆMST + 0.5 ˆMAE and mimage = maxi,j Mi,j.
  • Architectures: The patch description network architecture differs between EfficientAD-S and EfficientAD-M, while the autoencoder architecture is specified for both variants.The student uses additional kernels in designated convolutional layers compared with the teacher.
  • Distillation: The teacher distills WideResNet-101 features used by PatchCore into a compact teacher network with compatible output dimensions.PatchCore features have shape 384 × 64 × 64 for 512 × 512 inputs; other extractors can be adapted to the required spatial size.

Appendix B. Implementation Details for Other Evaluated Methods

The appendix documents implementations and reproduction settings for the evaluated comparison methods, including deviations from reported or default configurations.

  • Compared methods: The appendix covers AST, DSR, FastFlow, GCAD, PatchCore, SimpleNet, and Student–Teacher implementations.These methods are described using official implementations or paper-based reimplementations.
  • Reproduction results: AST achieves 98.9 % mean image-level detection AU-ROC on MVTec AD across five runs, compared with 99.2 % reported in its paper.The authors use the official implementation with its default configuration but could not fully reproduce the reported result.
  • Reproduction results: DSR achieves 98.1 % mean image-level detection AU-ROC on MVTec AD, close to the 98.2 % reported by its authors.The number of epochs is changed to 50 on VisA to keep total training iterations similar.
  • Configuration choices: FastFlow uses a WideResNet-50-2 feature extractor, disables early stopping, and trains for a constant 200 steps.The fixed duration is chosen to work well on average across evaluated datasets.
  • Configuration choices: GCAD is implemented as an ensemble of two models using 17 × 17 and 33 × 33 patch sizes, with a configuration adapted to perform better in these experiments.The appendix states that the reported results could be reproduced before adaptation.
  • Configuration choices: PatchCore disables center cropping for fair comparison and uses a 1 % coreset subsampling ratio with 224 × 224 images in its single-model variant.This is the configuration associated with the authors’ lowest reported latency.
  • Configuration choices: SimpleNet disables scenario-specific test-image tuning of training duration for fair comparison.The original multi-scale Student–Teacher implementation uses default hyperparameters without modification.

Appendix C. Robustness to the Distillation Backbone Architecture

EfficientAD maintains comparable robustness across evaluated distillation backbones, while PatchCore and EfficientAD show a structural-anomaly performance gap on VisA.

  • EfficientAD’s anomaly detection performance is similarly robust to the choice of backbone as PatchCore’s across three dataset collections.The comparison evaluates WideResNet-101, ResNeXt-101, and DenseNet-201 backbones.

Appendix D. Additional Anomaly Detection Metrics

The appendix reports additional image-level and pixel-level anomaly metrics, including alternative precision-recall and localization measures, with MVTec LOCO scores averaged across structural and logical anomalies.

  • Evaluation procedure: MVTec LOCO overall scenario scores average separate structural- and logical-anomaly performance scores.The official evaluation script computes each metric separately before averaging the two scores.
  • Image-level anomaly detection: Additional evaluations cover image-level AU-PRC alongside the main paper’s AU-ROC metric.AU-PRC is reported as an additional image-level anomaly detection metric.
  • Pixel-level anomaly localization: Localization is evaluated with AU-PRO up to false positive rates of 30% and 5%, plus pixel-wise AU-ROC and pixel-wise AU-PRC.The main-paper localization metric is AU-PRO up to a 30% FPR; the appendix adds the other measures and limits.

Appendix E. Timing Methodology and Additional Computational Efficiency Metrics

The appendix defines latency and throughput measurements and supplements them with computational metrics, emphasizing that runtime efficiency depends on hardware parallelization rather than FLOPs or parameter counts alone.

  • Reported efficiency metrics: EfficientAD’s efficiency metrics are reported on an NVIDIA RTX A6000 GPU, including latency, throughput, parameters, FLOPs, and GPU memory.Table 16 reports additional computational metrics, while Table 17 reports latency per GPU.
  • Timing methodology: Latency is single-image inference runtime, while throughput is images per second under batched processing.Throughput is measured with batched images when latency constraints are fulfilled or absent.
  • Timing methodology: Measurements run on GPUs with CPU-to-GPU transfer included for latency, and throughput is computed from 1,000 forward passes at batch size 16.The timing protocol uses the mean runtime of repeated forward passes.
  • Interpretation of efficiency metrics: Parameter counts can also mislead: FastFlow has roughly 2.5 times more parameters than S–T but substantially lower latency and 6.5 times higher throughput.
  • Interpretation of efficiency metrics: FLOPs can misrepresent runtime because highly parallelizable convolutions may execute efficiently on GPUs.S–T has over 2,000% more FLOPs than AST but only 42% higher latency.
  • Interpretation of efficiency metrics: GPU memory should be analyzed jointly with latency and throughput because freeing memory during inference can worsen runtime.The reported memory measure is peak reserved GPU memory during a forward pass.
  • Performance and throughput: Across evaluated methods, EfficientAD is presented with anomaly detection performance and throughput jointly, while method rankings differ between image-level and pixel-level tasks.Figure 8 averages AU-ROC and AU-PRO across MVTec AD, VisA, and MVTec LOCO.

Appendix F. Qualitative Results

Qualitative results show that methods differ in their handling of logical, structural, and small anomalies, while EfficientAD responds to both logical and structural anomalies. These visualizations are informative about method tendencies but should not replace objective evaluation metrics.

  • EfficientAD responds to both logical and structural anomalies, although its response strength leaves room for improvement on some logical cases.The breakfast box and box of pushpins are cited as examples.
  • AST detects some logical anomalies, but lacks a mechanism designed specifically for logical anomalies.Its response varies across examples, detecting an additional cable but missing a pushpin anomaly.
  • DSR produces precise segmentations but also generates false positives and can sometimes show no response to defects.
  • FastFlow’s anomaly maps contain substantial noise from false positive detections, hindering interpretation.
  • PatchCoreEns and SimpleNet struggle with challenging small defects, while GCAD has difficulty with some structural anomalies.The cited examples include printed-circuit-board defects, defective capsules, PCBs, scratches, and green capsules.
  • S–T struggles with logical anomalies because its patch-based receptive field cannot place locally normal patches in global image context.Its receptive field is limited to 65×65 pixels, unlike AST’s image-spanning receptive field.
  • Qualitative anomaly maps reveal method tendencies, but AU-ROC and AU-PRO provide objective comparisons across thousands of test images.
Loading 2303.14535v3…