PaperlayerView exact revision on arXiv

Source-linked AI summary

NFAD: Nuisance-Filtered Anomaly Detection Under Distribution Shift

Dat Cao, Son Nghiem, Phan Nguyen, Jun Rekimoto, Jhih-Ciang Wu

arXiv:2608.29112v1cs.CV

TL;DR

Industrial anomaly detectors can mistake acquisition changes for defects because standard benchmarks largely use controlled imaging conditions. NFAD estimates and suppresses nuisance variation without anomaly labels or target-domain data, using full projection for detection and selective suppression for localization. It achieves state-of-the-art shifted-domain performance while remaining competitive on standard benchmarks.

  • Problem

    Controlled acquisition benchmarks provide limited evidence about robustness when illumination, background, viewpoint, or camera configuration changes shift normal features.

  • Method

    NFAD estimates an acquisition-nuisance subspace label-free from content-preserving perturbations and applies full or selective suppression through task-specific read-outs.

  • Results

    91.0% I-AUROC and 90.8% AUPRO on AeBAD-S establish state-of-the-art performance, while NFAD remains competitive on VisA, Real-IAD, and MVTec-AD.

  • Takeaways & Limitations

    Nuisance suppression adds shift robustness to a frozen normal-only detector without labels, target-domain data, or retraining.

  • Takeaways & Limitations

    Existing nuisance modeling is constrained by its intervention family; the authors identify geometric viewpoint interventions and online subspace re-estimation as extensions.

Abstract

from arXiv · show

Recent advances in anomaly detection (AD) for industrial inspection have pushed performance on standard benchmarks toward saturation. However, strong benchmark performance does not necessarily translate to real-world deployment, as these benchmarks are primarily collected under controlled acquisition conditions. Changes in illumination, background, viewpoint, and other environmental factors can shift normal samples away from the learned normal distribution and cause false anomaly responses. We address AD under such distribution shifts by explicitly modeling nuisance variation from changing imaging conditions in feature space. Without anomaly labels or target-domain data, our Nuisance-Filtered Anomaly Detection (NFAD) framework estimates a nuisance subspace from matched feature displacements induced by content-preserving perturbations and suppresses its contribution to anomaly residuals at inference. The same subspace supports two complementary branches: full projection for image-level detection and selective suppression for pixel-level localization, preserving evidence of localized defects. On AeBAD-S, a benchmark specifically designed for AD under acquisition shifts, NFAD achieves 91.0\% image-level AUROC, establishing a new state of the art. Notably, this robustness does not come at the expense of conventional AD performance: NFAD remains competitive on standard benchmarks that do not explicitly evaluate distribution shift, including VisA, Real-IAD, and MVTec AD. These results show that explicitly suppressing such nuisance variation improves AD under distribution shift while preserving strong performance in standard settings.

1. Introduction

Industrial anomaly detectors can mistake acquisition changes for defects because standard benchmarks use controlled imaging conditions. NFAD addresses this problem by estimating nuisance variation label-free and using task-specific read-outs, improving shifted-domain robustness while remaining competitive on conventional benchmarks.

  • Controlled benchmark performance provides limited evidence about detector behavior when illumination, background, viewpoint, or camera configuration changes.
  • AeBAD, ADShift, and FiCo show that conventional anomaly detectors can degrade under real or synthetic acquisition shifts.
  • NFAD estimates an acquisition-nuisance subspace label-free from content-preserving perturbations of normal images.
  • Full nuisance projection supports image-level detection, while selective spatial suppression preserves localized defect evidence for localization.
  • 91.0% I-AUROC and 90.8% AUPRO on AeBAD-S establish state-of-the-art performance while NFAD remains competitive on VisA, Real-IAD, and MVTec-AD.

2. Related Work

Related work spans feature-reconstruction anomaly detectors, distribution-shift methods, and representation-invariance techniques. NFAD is positioned around post-hoc, controllable nuisance removal rather than retraining-based invariance.

  • Feature-reconstruction methods use discrepancies between learned normal representations and test features as anomaly evidence.
  • AeBAD, ADShift, and FiCo improve robustness through shifted benchmarks, augmented out-of-distribution views, or distribution-specific filtering and compensation.
  • The operator’s 32-dimensional nuisance subspace is evaluated through defect, normal, and discriminative residual alignment against a random-subspace chance floor.
  • Prior approaches that filter nuisance information in feature space require retraining and cannot control suppression strength or apply it selectively across spatial regions.
  • Concept erasure, domain-adversarial, self-supervised, and augmentation-factorization methods provide broader mechanisms for controlling unwanted representation information.

3. A Closer Look at Nuisance Entanglement

Feature residuals combine intrinsic defect deviations with acquisition nuisance, and nuisance and defect evidence can occupy the same directions. NFAD therefore uses full projection for image-level robustness and spatial incidence to preserve localized evidence.

  • Under domain shift, feature residuals contain intrinsic image deviations and nuisance arising from acquisition changes such as illumination or background.
  • NFAD estimates a nuisance subspace from content-preserving interventions and removes its component from residuals.
  • Defect residuals place 37–48% of their energy in the 32-dimensional nuisance subspace on ViT backbones, or nine to eleven times chance.
  • The defect-versus-normal discriminative direction lies even deeper in the nuisance subspace, reaching about 13× chance on ViTs and 7.6× on a supervised ConvNet.
  • Projection removes 52% and 44% of normal-patch residual energy in modeled background and illumination domains, versus 40% in the shift-free domain.
  • Image-level scores remain strongly rank-correlated after projection, with Spearman 0.923 on AeBAD-S, while spatial incidence limits removal of localized defect evidence.

4. Method

NFAD estimates acquisition-nuisance directions from matched content-preserving feature displacements, then uses task-specific read-outs to suppress nuisance variation while preserving defect evidence. Its detection branch projects out the full nuisance subspace, whereas localization selectively attenuates globally acting directions.

  • Feature reconstruction: The framework uses a frozen DINOv3 teacher and a normal-only MAE student to produce hierarchical feature-reconstruction residuals for anomaly scoring.Four intermediate teacher blocks provide multiscale patch features, while the student reconstructs teacher features from masked normal-image views.
  • Nuisance subspace estimation: NFAD estimates a nuisance-sensitive subspace from matched intervention responses between normal images and content-preserving perturbations.Differencing paired features cancels shared content and isolates the intervention-induced nuisance response; leading eigenvectors form the basis.
  • Detection read-out: Detection applies full projection onto the complement of the nuisance subspace, suppressing nuisance contributions in the teacher–student residual.The projection is applied separately to teacher and student features and is equivalent by linearity to projecting the residual.
  • Localization read-out: Localization retains localized nuisance directions but removes globally acting directions using spatial incidence and participation-ratio gating.Incidence scores near 1 indicate image-wide variation and scores near 0 indicate localized variation; the localization basis leaves the most localized quartile untouched.
  • Localization read-out: Responses at least as diffuse as the training-normal median receive full attenuation, while more concentrated responses are attenuated less to preserve local defect evidence.The resulting operator is soft rather than an idempotent projection, and both branches are calibrated on their own normal-training maps.

5. Experiments

NFAD is evaluated on AeBAD-S, standard benchmarks, acquisition domains, corruption shifts, and component ablations. It achieves strong domain-shift robustness while remaining competitive on conventional benchmarks, with projection and MIR-based readouts providing the main gains.

  • AeBAD-S comparison: NFAD ranks first on five of seven AeBAD-S metrics and second on a sixth, reaching 91.0 image AUROC and 90.8 AUPRO.It exceeds FiCo by 15.3 image-AUROC points and leads image-level performance over the best competing method by 6.4, 3.1, and 2.5 points across reported metrics.
  • Per-domain performance: View is the hardest AeBAD-S acquisition domain, with 86.0 image AUROC and 87.5 AUPRO, while background and illumination achieve stronger results.NFAD remains robust under view shift even though its nuisance families explicitly model photometric and background variation.
  • Standard benchmarks: NFAD remains competitive on standard benchmarks, reaching 98.9/98.6/95.8 on VisA and 91.5/99.1/96.1 on Real-IAD for image AUROC, pixel AUROC, and AUPRO.On MVTec-AD, its 99.0/97.7/92.7 is marginally below leading methods in a saturated 98.4–99.7 image-AUROC range.
  • Detection-head ablation: Projection increases AeBAD-S image AUROC by 3.8 points, from 86.7 to 90.5, compared with 0.7 on MVTec-AD and 0.3 on VisA.The larger AeBAD-S gain indicates that projection primarily suppresses acquisition nuisance rather than adding generic capacity.
  • Ablation study: MIR-based nuisance estimation adds 0.5 image-AUROC points on AeBAD-S and 0.4 on VisA over variance-based estimation.For localization, projection improves AU-PRO by 1.2, 0.5, and 0.3 points on AeBAD-S, MVTec-AD, and VisA, while gated selective suppression adds 0.5 pixel-AUROC points on AeBAD-S.
  • Robustness and backbone analysis: Under synthetic corruptions, NFAD gains up to 5.9 points and improves across modeled photometric shifts and held-out blur and Gaussian-noise corruptions.The operator also improves every tested foundation backbone, by 10.04 image-AUROC points on average.

6. Conclusion

NFAD addresses distribution-shift failures by modeling acquisition changes as nuisance variation and suppressing their contribution to anomaly residuals. Its practical scope includes frozen, normal-only detectors, while geometric and online extensions remain future directions.

  • NFAD identifies acquisition nuisance as a feature-space component that can make normal shifts appear anomalous and designs suppression around this entanglement.
  • The framework operates on a frozen normal-only detector without anomaly labels, target-domain data, or retraining.
  • Full nuisance removal supports image-level detection, while selective suppression preserves localized defect evidence for pixel-level localization.
  • Geometric transformations for viewpoint changes and online nuisance-subspace re-estimation are identified as natural extensions.

Supplementary Material

The supplement expands the main paper with materials for reproducibility, analytical validation, and broader experimental assessment.

  • The supplement provides complete implementation details, configurations, dataset statistics, and evaluation splits.
  • It validates the analysis, reports exhaustive per-category and per-domain results, includes remaining ablations, and presents qualitative visualizations.

A. Full Implementation Details

NFAD uses frozen teacher–student features, content-preserving interventions, and a nuisance basis to define complementary detection and localization read-outs. The implementation uses shared settings across datasets and avoids test-time tuning.

  • Backbones and features: The frozen DINOv3 teacher provides normalized multi-block features, while an MAE student predicts corresponding token features through block-specific 1×1 convolutional heads.
  • Backbones and features: The foreground prior is thresholded at 0.5 to define the region used by the background intervention.
  • Training: Training uses normal images only, with fixed optimization schedules and resolution settings that differ between AeBAD-S/MVTec-AD and VisA/Real-IAD.
  • Intervention families: The nuisance basis is estimated from photometric and background content-preserving interventions applied in image space.
  • Nuisance basis: For each feature block, the method retains the top 16 eigenvectors per intervention family and combines the orthogonalized bases into V_l ∈ R^768×32.
  • Incidence subset and gate: Incidence ratios separate globally acting nuisance directions from local ones to form the gated localization sub-basis.
  • Incidence subset and gate: The per-image localization gate uses participation ratio relative to the median normal-training participation ratio.
  • Scoring and read-out: Both read-outs project features away from the nuisance subspace and compare teacher–student features with an ε-stabilized cosine score.

B. Dataset Statistic

The evaluation covers four industrial anomaly-detection benchmarks under class-separated training, with AeBAD-S serving as the primary domain-shift benchmark. Its source-domain training uses one object category, while test images span multiple capture conditions.

  • Experiments use four industrial anomaly-detection benchmarks with one detector trained per object category.
  • AeBAD-S is the primary domain-shift benchmark and contains 521 source-domain normal training images for a single aero-engine turbine-blade object.
  • AeBAD-S test images span four capture conditions, making it the benchmark used to assess acquisition-shift robustness.
  • The supplement provides dataset statistics verified from the released splits, including test totals and normal/anomalous counts.

C.1. Entanglement across subspace rank

NFAD’s nuisance subspace overlaps defect and normal residuals across ranks and residual constructions, showing that nuisance and defect information are directionally entangled. Projection preferentially removes shift-induced normal residual energy, while rank selection balances detection against localization.

  • Entanglement across subspace rank: Defect and normal residuals remain near-equally aligned with the nuisance subspace at ranks 32 and 64, far above chance.The overlap is about 10× chance at rank 32 and 7× at rank 64.
  • Entanglement across subspace rank: The defect-discriminative direction lies inside the nuisance subspace across datasets and backbones, confirming that nuisance and defect signals share feature directions.For DINOv3, DINOv2, and WideResNet-50, discriminative overlap is 0.644, 0.657, and 0.365, respectively.
  • Projection mechanism: Projection removes 52% and 44% of normal-patch residual energy under modeled background and illumination shifts, versus 40% on the shift-free domain.Defect-to-normal contrast changes by under 5% after projection on AeBAD-S.
  • Seed variance: Across five seeds, image-AUROC standard deviation is 0.15 and AUPRO variation is within 0.12.These measurements quantify run-to-run stability on AeBAD-S.
  • Rank sensitivity: Increasing the per-family rank raises detection I-AUROC from 89.8 at d=8 to 91.4 at d=32 but lowers AUPRO from 91.1 to 90.0.The default d=16 balances the two metrics and remains within 0.4 of the best value on either.

D.7. Operator as a model-agnostic plug-in

The operator is evaluated beyond its primary within-framework evidence as a plug-in for established detectors. Transfer is strongest for detectors whose residuals reflect per-object clean-normal statistics, while structural mismatches can degrade performance.

  • Scope: The plug-in study swaps the entire base detector, extending the primary backbone-based generality evidence as a secondary transfer analysis.The operator is defined for the paper’s teacher-student residual, so within-framework evaluations remain the central evidence.
  • Compatible reconstruction detectors: ReContrast improves all seven reported metrics at d=64, including image AUROC by 1.2 and AUPRO by 3.5.RD4AD improves six of seven metrics, with image AUROC increasing by 1.8 while image F1-max remains flat.
  • Partial transfer: GLASS shows non-monotone behavior, although its pixel AUROC improves by 4.3 and pixel AP by 2.6 at d=32.Its synthetic-outlier training produces features that only partially match clean-normal interventions.
  • Structural mismatch: UniAD’s image AUROC degrades from 56.0 to 51.9 because its jointly trained, all-class design lacks a per-object nuisance subspace.This is a structural mismatch with the operator’s per-object estimation.
  • Conclusion: The results indicate transfer to detectors whose residuals reflect per-object clean-normal statistics, rather than to every detector architecture.The conclusion combines the plug-in detector study with the backbone study’s within-framework evidence.

E. Full Qualitative Result and Visualization

Qualitative results show NFAD suppressing diffuse acquisition-shift responses while retaining localized defect evidence across real and synthetic shifts. Per-category results indicate near-saturated detection on MVTec-AD, harder multi-view detection on Real-IAD, and persistent small-defect difficulty on VisA.

  • MVTec-AD: MVTec-AD detection is near-saturated, with eight of fifteen categories reaching perfect image AUROC; remaining variation appears mainly in pixel metrics.Thin or low-contrast defects in screw, grid, and toothbrush are particularly variable.
  • Real-IAD: Real-IAD reaches mean image AUROC of 91.5 under multi-view acquisition, while high pixel AUROC indicates defects remain localizable once detected.Its multi-view setting is substantially harder than the other reported benchmark patterns.
  • VisA: VisA retains small-defect difficulty, especially in the macaroni categories, despite qualitative recovery under synthetic corruptions.The per-category tables report the underlying image- and pixel-level metrics.
  • AeBAD-S visualization: On AeBAD-S, full projection suppresses responses caused by real illumination and background shifts in both defective and normal examples.Figure 5 uses the same subspace estimated from both intervention families.
  • VisA-C visualization: Under brightness, contrast, defocus blur, and Gaussian noise on VisA-C, NFAD recovers localized defects after corruption spreads control responses across objects.On normal images, the operator reduces false alarms toward the annotated image-level z-score.
Loading 2608.29112v1…