PaperlayerView exact revision on arXiv

Source-linked AI summary

Spatial Attention Supervision for Defect Localization: Exploiting Ground-Truth Masks as Training Signal in Diffusion-Augmented Defect Detection

Sajjad Rezvani Boroujeni, Muskan Saraf, Gnana Tulasi Makineni, Tom Bush, Hossein Abedi

arXiv:2609.06232v1cs.CVcs.LGeess.IV

TL;DR

Prior diffusion-augmented defect classifiers reported image-level metrics, leaving open whether they attend to defects, while benchmark masks were largely reserved for evaluation. This paper uses masks to guide classifier activations in mixed-supervision training alongside maskless DDPM-generated images. Attention supervision improves held-out activation-based localization in several CNN settings without a significant classification change.

  • Problem

    Prior diffusion-augmented defect detection reported image-level metrics, and benchmark masks were used for localization evaluation rather than as auxiliary attention guidance for classifiers with both masked and maskless samples.

    Source: §1 · p. 2

  • Method

    An activation-alignment loss uses available defect masks to guide classifier feature maps, while maskless samples receive classification-only supervision.

    Source: §1 · p. 2, §3.4 · p. 7

  • Results

    Activation-based Pixel-AUROC rose by up to +18.7% for ResNet50 and +18.0% for EfficientNetB0 with augmentation; gains were significant in four of eight CNN settings, with no significant classification change.

    Source: §7 · p. 18

  • Takeaways & Limitations

    The approach combines synthetic-data quantity with mask-based spatial precision, and only 19% of training images need masks.

    Source: §1 · p. 3, §7 · p. 19

    Results are limited to the MVTec-AD bottle dataset, and generalization to other categories requires validation.

    Source: §6.5 · p. 17

Abstract

from arXiv · show

Ground-truth defect masks in industrial inspection datasets are typically reserved for evaluation. This paper repurposes them as spatial supervision signals during training of classification networks, teaching a model not just what to predict but where to look. The method adds an activation-based attention alignment loss that steers convolutional feature maps toward defect regions, in a mixed-supervision formulation that also accommodates samples without masks, such as diffusion-generated images. Combined with DDPM augmentation, synthetic images contribute quantity while masks contribute spatial precision. We evaluate 85 models (four CNN backbones under a 2x2 data/training factorial over five seeds, plus a Swin-V2-T transformer baseline) on the MVTec-AD bottle benchmark, with localization measured on held-out defect images excluded from classifier gradient updates. Main findings: (1) attention-guided training improves activation-based localization (Pixel-AUROC) by +18.0% for EfficientNetB0 with augmentation (p=0.005, Cohen's d=2.6) and +18.7% for ResNet50 (p=0.008), significant in four of eight CNN settings (uncorrected for multiple comparisons) with no significant change in classification; (2) for EfficientNetB0 a data x training-mode interaction is significant (p=0.002), consistent with a super-additive effect (+13.6% combined vs +1.6% summed individual effects); (3) architectures with weaker spatial representations benefit most, whereas ConvNeXt-T shows no effect, apparently because its depthwise-convolution activations yield spatially uninformative channel-mean maps; (4) unsupervised PatchCore remains the strongest localizer (Pixel-AUROC=0.983), contextualizing the supervised gains. These results show that existing evaluation masks can act as practical training signals that measurably and reproducibly improve where defect classifiers attend.

1 Introduction

Industrial inspection needs to locate defects, not merely classify images, but prior diffusion-augmented work reported image-level metrics without showing whether models attend to defects. This paper uses existing defect masks as auxiliary spatial guidance for classifiers, including in training that mixes masked and maskless DDPM-generated images.

1 Introduction

  • Ground-truth masks guide convolutional feature maps toward defect regions through an activation-based alignment loss, while the classifier still predicts a binary label.The mixed-supervision framework uses classification-only loss for samples without masks.Source: §1 · p. 2
  • The study combines mask supervision with DDPM augmentation so synthetic images provide quantity while masks provide spatial precision.Maskless generated samples remain usable in the mixed-supervision setting.Source: §1 · pp. 2–3
  • The paper evaluates attention guidance and DDPM augmentation across four CNN architectures in a 2×2 design, with Swin-V2-T as a transformer baseline.The benchmark comprises 85 models across five seeds and includes activation-based and Grad-CAM localization assessments.Source: §1 · p. 3
  • Localization is measured on held-out validation images excluded from classifier gradient updates, with results aggregated across five seeds and tested using paired significance tests and effect sizes.This contrasts with prior single-run localization on training images.Source: §1 · p. 3

2 Related Work

Related work covers diffusion-based defect synthesis, attention and explainability methods, supervised attention, and unsupervised localization. The paper positions its approach as using masks to guide classifier attention during training rather than as segmentation targets or post-hoc visualizations.

2.1 Diffusion Models for Defect Synthesis

  • Prior diffusion defect-synthesis work uses masks to guide generation when present, but the reviewed approaches do not use them as downstream classifier training supervision.DDPM-generated defects have improved MVTec-AD bottle classification, while reported studies focus on image-level performance.Source: §2.1 · p. 3

2.2 Explainability and Attention in Defect Detection

  • Reviewed defect-detection work uses attention modules or Grad-CAM visualizations, rather than attention as a classifier training loss.Examples include CBAM modules and Grad-CAM applications to glass-bottle print defects.Source: §2.2 · p. 3

2.3 Attention Supervision in Deep Learning

  • Prior work supervises attention maps, including with partial masks, but the cited applications concern natural-image benchmarks rather than industrial defect detection with maskless diffusion samples.The paper identifies the industrial setting, maskless synthetic samples, and modern-backbone panel as its focus.Source: §2.3 · p. 4

2.4 Unsupervised Anomaly Localization

  • PatchCore is included as an unsupervised localization baseline, while MVTec AD 2 was introduced because existing benchmarks had saturated.PatchCore scores test patches by nearest-neighbor distance to a coreset of normal patch features.Source: §2.4 · p. 4

2.5 Positioning of This Work

  • The proposed approach uses ground-truth masks as an auxiliary attention-training signal for a classifier whose output remains a class label.This differs from using masks as segmentation output targets or using Grad-CAM for post-hoc visualization.Source: §2.5 · p. 4

3 Methodology

The methodology combines optional DDPM augmentation with mask-guided spatial attention training, then evaluates classification and localization on the MVTec-AD bottle benchmark. Its mixed-supervision design uses masks where available while retaining classification-only training for maskless samples.

3.1 Dataset

  • The dataset pairs 209 non-defective training images with 63 real defective images and their masks; AugmentedData adds 60 maskless DDPM-generated defectives.The generated images were created from the real defective set before the per-seed held-out protocol was defined.Source: §3.1 · pp. 4–5

3.2 Backbone Architectures

  • The study evaluates five architectures spanning lightweight to modern CNNs and a vision transformer, with Swin-V2-T serving as the transformer baseline under standard training only.All models use ImageNet-1K pretrained weights; attention supervision via convolutional activation maps is not applicable to transformers.Source: §3.2 · p. 6

3.3 Training Protocol

  • All models use a linear-probe-then-fine-tune protocol, with the backbone frozen for five probe epochs and unfrozen for up to 25 fine-tuning epochs.The fine-tuning learning rate is 0.1× the probe learning rate; training also uses AdamW, early stopping, and five seeds.Source: §3.3 · p. 6

3.4 Spatial Attention Supervision

  • The method uses ground-truth masks as auxiliary spatial supervision for a classifier whose output remains a binary class label, rather than as segmentation targets.Classification loss applies to all samples, while spatial attention loss is applied selectively.Source: §3.4 · p. 7, §3.4.2 · p. 7

3.4.1 Activation-Based Spatial Attention

  • The spatial attention map is formed from last-convolutional-layer activations by taking a channel-wise mean, normalizing it, and resizing it to the mask resolution.EfficientNetB0, MobileNetV2, and ResNet50 use a 1 × 1 pointwise last convolution, whereas ConvNeXt-T uses a 7 × 7 depthwise convolution.Source: §3.4.1 · p. 7

3.4.2 Mixed-Supervision Loss

  • For real defective images, the attention loss aligns the normalized map with the resized ground-truth mask using mean-squared error and a cosine-similarity term.Maskless DDPM-generated images receive classification loss only; defect-free images are penalized for strong activations.Source: §3.4.2 · pp. 7–8

3.5 Localization Evaluation

  • Localization is assessed with activation-based maps and Grad-CAM, while supervised-model scores use held-out defective images excluded from classifier gradient updates.The held-out validation split is also used for early stopping and checkpoint selection, but localization metrics do not select checkpoints.Source: §3.5 · p. 8

4 Experimental Setup

The experiment crosses DDPM augmentation with attention-guided training across four CNN backbones, using Swin-V2-T as a reference baseline. Classification and localization are evaluated across five seeds, with within-setting comparisons designed to isolate the effect of attention guidance.

4.1 Factorial Design

  • The 2 × 2 design crosses RealData (272 images) versus AugmentedData (+60 DDPM, 332 images) with standard versus attention-guided training to test whether the strategies are complementary or redundant.Source: §4.1 · p. 9
  • The study trains 85 models across four CNN backbones and Swin-V2-T, with five seeds per configuration; the transformer is a reference baseline, not a factorial cell.Swin-V2-T receives only the RealData/Standard configuration because the attention loss uses convolutional feature maps and its RealData F1 is already 1.000.Source: §4.1 · p. 9

4.2 Statistical Protocol

  • Metrics are means across five seeds; paired t-tests and Cohen’s d compare standard with attention-guided training, while uncorrected per-setting significance is treated as exploratory.Source: §4.2 · p. 9
  • Within each setting, the two modes share data, class weights, held-out pool, and checkpoint-selection criterion, differing only in training loss; Table 3 reports held-out F1, AUROC, and Recall.For ConvNeXt-T, the modes’ classification metrics coincide to three decimal places and are reported jointly as “Std/AG.”Source: §4.2 · p. 10, Table 3 · p. 10

5 Results

Attention-guided training improved held-out activation-based localization in several CNN settings without significantly changing classification, while PatchCore remained the strongest localizer. The results also show an EfficientNetB0 interaction between DDPM augmentation and attention guidance, and differing effects across localization methods and architectures.

5.1 Classification Performance

  • Attention guidance did not significantly change classification for any architecture, while Swin-V2-T reached F1=1.000 and ConvNeXt-T reached F1=0.988.Among lighter CNNs, reported F1 ranged from 0.84 to 0.90; ResNet50 performed poorly under this study's LP-FT protocol.Source: §5.1 · p. 10, Figure 2 · p. 11

5.2 Localization: Attention Guidance Effect

  • Attention guidance significantly improved held-out activation-based localization in four of eight CNN settings, including +18.0% for EfficientNetB0 with DDPM and +18.7% for ResNet50 on real data; PatchCore remained strongest at Pixel-AUROC 0.983.The reported significance threshold is uncorrected for multiple comparisons.Source: §5.2 · p. 11, Table 4 · p. 12, §5.4 · p. 13

5.3 DDPM and Attention Guidance Appear Complementary

  • For EfficientNetB0, combined DDPM and attention guidance raised Pixel-AUROC from 0.552 to 0.627 (+13.6%), exceeding the +1.6% summed individual effects; the interaction was significant (p=0.002).DDPM alone was slightly harmful, while attention alone yielded +5.4%; the interaction was not significant for MobileNetV2 or ResNet50.Source: §5.3 · pp. 12–13

5.4 Classification and Localization Trade-off

  • Among backbones responding to attention supervision, EfficientNetB0 with DDPM and attention guidance offered the best held-out balance: F1=0.886 and activation-based Pixel-AUROC=0.627.No supervised classifier matched PatchCore's localization performance; results were not broken down by defect type because per-type samples were too small for reliable estimates.Source: §5.4 · p. 13

5.5 Grad-CAM Comparison

  • Grad-CAM outperformed raw activation maps, and attention guidance significantly improved Grad-CAM localization only for ResNet50: +6.6% on real data and +10.7% with DDPM.ConvNeXt-T had the strongest Grad-CAM localization among the CNNs, but its activation map did not improve with guidance.Source: §5.5 · pp. 13–14, Table 5 · p. 14, Figure 4 · p. 14

5.6 Qualitative Heatmap Comparison

  • Across all three defect types, attention-guided heatmaps were more concentrated on defect regions than standard-training heatmaps.The qualitative comparison is for EfficientNetB0 with augmented data.Source: Figure 5 · p. 15, §5.6 · p. 15

6 Discussion

Attention supervision benefits architectures with weaker spatial representations, while ConvNeXt-T appears insensitive to the channel-mean alignment signal. The discussion situates mixed-mask training alongside DDPM augmentation and PatchCore, and identifies limits in dataset scope and evaluation.

6.1 Why Attention Supervision Works for Some Architectures

  • ResNet50 gained +18.7% (p = 0.008) and EfficientNetB0 with augmentation +18.0% (p = 0.005); weaker spatial representations benefited most.MobileNetV2 gained +4.0% (p = 0.020) on real data; the authors suggest the method is most valuable for lightweight models in resource-constrained manufacturing.Source: §6.1 · pp. 15–16
  • ConvNeXt-T showed no reliable response: its hooked 7 × 7 depthwise layer yields largely flat channel means despite strong Grad-CAM localization.The authors propose that an architecture-specific attention signal may be needed; they did not verify the proposed mechanism with gradient-flow diagnostics.Source: §6.1 · p. 16, §6.5 · p. 18

6.2 Mixed Supervision and Practical Deployment

  • Only 63 of 332 augmented-setting training images (19%) have masks; maskless samples receive classification loss only.The available masks provide spatial guidance without additional labeling beyond masks already used for evaluation.Source: §6.2 · p. 16

6.3 Relationship to Prior Work

  • The study extends prior DDPM classification work by evaluating localization, reporting held-out activation-based Pixel-AUROC gains up to +18.7%.The authors describe data quantity and spatial precision as complementary contributions for lightweight architectures.Source: §6.3 · p. 16

6.4 PatchCore as a Localization Reference

  • PatchCore reaches Pixel-AUROC 0.983, exceeding supervised methods; it models patch-level normality but requires a feature memory bank and lacks class-discriminative explanations.The paper frames PatchCore and supervised models as serving complementary roles: precise localization versus efficient deployment with optional attention guidance.Source: §6.4 · p. 17

6.5 Limitations

  • Results cover only MVTec-AD bottle, and the held-out validation split also guides early stopping and checkpoint selection, so it is not a fully untouched test set.Augmented-setting images also involve indirect exposure to later-held-out defectives through the prior unconditional DDPM; the authors treat within-setting comparisons as primary evidence.Source: §6.5 · pp. 17–18
  • Per-setting significance counts are exploratory because five-seed tests lack multiple-comparison correction, and only 9–12 defective images are held out per seed.The authors note that larger benchmarks would yield tighter estimates and that per-defect-type analysis remains too sparse.Source: §6.5 · p. 18

7 Conclusions

The paper concludes that mask-guided training improves held-out localization in several CNN settings without significant classification changes, while maskless DDPM samples remain usable through classification loss alone. It also identifies mask-conditioned diffusion and architecture-specific signals as future directions.

7 Conclusions

  • The approach uses defect masks as auxiliary spatial supervision with maskless DDPM-generated defective samples handled in a mixed-supervision framework.The study trained 85 models across four CNN backbones and a Swin-V2-T reference baseline.Source: §7 · p. 18
  • Held-out activation-based Pixel-AUROC improved by up to +18.7% for ResNet50 and +18.0% for augmented EfficientNetB0, with significant gains in four of eight CNN settings and no significant classification change.The EfficientNetB0 data × training-mode interaction was significant (p = 0.002), consistent with a super-additive combined effect.Source: §7 · p. 18
  • Only 19% of training images need masks; maskless DDPM-generated images receive classification loss only.The paper states that architectures with weaker spatial representations benefit most, particularly for lightweight models in resource-constrained manufacturing environments.Source: §7 · p. 19
  • Future work includes mask-conditioned diffusion to generate image/mask pairs, architecture-specific attention extraction, and extension to multi-class defect classification.Source: §7 · p. 19

Funding.; Data availability.; Competing interests.

Loading 2609.06232v1…