PaperlayerView exact revision on arXiv

Source-linked AI summary

What Remains Normal? Clean Images Miss Useful Near-Defect Normal Patches for Anomaly Detection

Joongwon Chae, Runming Wang, Peiwu Qin

arXiv:2608.23299v1cs.CV

TL;DR

The paper asks whether clean-only training provides all normal features needed for defect localization, given that contextual encoders represent normal patches according to their surrounding context. It compares matched fixed-budget memories and introduces BOUNDARYSUPPORT, which alters context around preserved normal patches and learns from neighboring features. The diagnostic raises MVTec P-AP from 73.34 to 76.95, while BOUNDARYSUPPORT consistently improves pixel localization across the evaluated detector and dataset settings.

  • Problem

    Clean-only training can omit useful normal patches near defects because contextual representations depend on surrounding image content.

  • Method

    BOUNDARYSUPPORT alters context around pixel-preserved normal patches, excludes modified tokens, and uses preserved neighboring features as memory references or reconstruction targets.

  • Results

    73.34 to 76.95 MVTec P-AP: a fixed-budget GT-normal oracle outperforms the clean-only memory, while BOUNDARYSUPPORT improves P-AP in all six detector–dataset settings.

  • Takeaways & Limitations

    Normal-only detection depends on which contextual normal features are available, not only on how references or targets are scored.

  • Takeaways & Limitations

    Localization gains can trade off against defect evidence and image-level metrics, and deployment-relevant missing patches remain an open problem.

Abstract

from arXiv · show

Memory-based anomaly detectors store nominal training patches and score test patches against this memory. A patch selected for coverage therefore becomes a nor- mal reference without a separate check that geometric rarity makes it safe to trust. We probe this coupling with sparse training contamination. Under fixed representa- tions and memory budgets, we compare random, medoid, local, and global coverage selectors. We then use CLEANCON, an out-of-bag cross-image support gate that changes candidate-image eligibility while fixing the representation, absolute mem- ory size, builder, and inference rule. Global coverage strongly over-represents sparse contamination. CLEANCON reduces final-memory contamination to approx- imately zero and increases category-macro P-AP in all 12 matched comparisons. Yet along a retention sweep, the lowest-contamination memory does not attain the highest P-AP; performance continues to improve while contamination rises. Mem- ory contamination therefore does not order the resulting memories by P-AP.Code is publicly available at https://github.com/jw-chae/cleancon.

1 Introduction

The paper argues that clean-only training can omit useful normal patches near defects because contextual representations depend on surrounding image content. A fixed-budget diagnostic and BOUNDARYSUPPORT show that altered-context features improve anomaly localization across detector paradigms.

  • Motivation: Clean-only training can omit normal test patches whose contextual representations are absent from clean candidate sets.These patches may appear immediately beside defects while remaining locally normal.
  • Fixed-budget diagnostic: 73.34 to 76.95 MVTec P-AP: allowing ground-truth-normal patches from defective images into a matched fixed-budget memory raises localization performance.The representation, scoring rule, reference count, and held-out evaluation are otherwise fixed.
  • Near-defect evidence: 94.70% of the fixed-budget gain is recovered by ground-truth-normal candidates within two patch cells of defects.The oracle uses ground-truth masks only diagnostically.
  • Method: BOUNDARYSUPPORT alters context around pixel-preserved normal patches, excludes modified tokens, and uses preserved neighboring features as references or reconstruction targets.The memory branch preserves the original bank budget, while the reconstruction branch adds auxiliary normal targets without changing the clean branch.
  • Results: Both BOUNDARYSUPPORT branches improve P-AP in all six detector–dataset settings, while matched controls show altered-context features outperform clean-view features as references or targets.The learned score change is strongest on normal patches adjacent to real defects.

2 Related Work

Related work draws normal evidence from clean training data, contaminated pools, synthetic anomalies, or defect-containing images. This paper instead studies useful normal features absent from an otherwise clean training pool.

  • Normal references and targets: Memory and reconstruction methods primarily use clean patch features as normal references or targets.Examples include PatchCore, ProCon, reverse distillation, and Dinomaly.
  • Learning with contaminated training data: Contaminated-training methods identify or suppress unreliable patches already present in the training pool.The cited methods use outlier filtering, recurrence, reconstruction, or multiple discriminators.
  • Synthetic anomaly learning: Synthetic anomaly methods create corrupted or difficult examples for training anomaly detectors.The cited approaches include pasted transformations, reconstruction, blending, and prototype-boundary synthesis.
  • Normal evidence inside defect-containing images: Methods using defect-containing images show that useful normal evidence can remain inside images containing anomalies.This paper asks whether that evidence is already represented by the clean training pool.

3 Clean Images Miss Useful Normal Patches Near Defects

A matched fixed-budget diagnostic isolates candidate identity as the source of missing normal evidence: ground-truth-normal patches near defects substantially improve pixel localization. Approximate k-center selection can further prioritize this small near-defect cohort.

  • Matched diagnostic: The diagnostic changes the candidate set while fixing the encoder, representation, reference count, and test-time score.Clean-only and oracle memories use identical held-out evaluations, with donor images excluded from both arms.
  • Oracle comparison: 73.34 to 76.95 P-AP: the fixed-budget GT-normal oracle outperforms the clean-only memory on MVTec AD.The expanded oracle with 5.3% more references reaches 76.96 P-AP, only 0.01 point higher than the fixed-budget oracle.
  • Near-defect candidates: 94.70% of the fixed-budget gain is recovered when additional candidates are restricted to ground-truth-normal patches within two patch cells of defects.The full near–mid–far ordering is reported as a two-category diagnostic, not an all-category result.
  • Controls: Inpainting and cross-defect-type controls retain positive gains, narrowing explanations based on visible-defect leakage or exact defect memorization.Inpainting improves P-AP by 11.72 and 6.11 points in leather and tile; three source-to-target directions remain positive.
  • Selection geometry: 0.23% to 2.68% in leather and 0.70% to 3.44% in tile: approximate k-center selection enriches near-defect patches among selected anchors.The enrichment is 11.73× in leather and 4.95× in tile, while downstream P-AP comparisons establish usefulness.

4 BoundarySupport: Learning Preserved Patches under Altered Context

BOUNDARYSUPPORT creates altered-context views from clean images, excludes modified token footprints, and learns only from pixel-preserved neighboring positions. It adds altered-context features or targets without changing the memory budget or test-time scoring rule.

  • Exclude modified tokens; learn from preserved neighbors: The excluded set combines nominal insertion tokens with tokens affected by directly measured RGB changes.This accounts for blending and interpolation modifying pixels outside the nominal insertion mask.
  • Design: BOUNDARYSUPPORT changes surrounding context while using only preserved neighboring features as additional normal references or reconstruction targets.The synthetic insertion remains visible as context, but excluded tokens cannot become memory candidates or direct reconstruction targets.
  • Memory branch: The memory branch selects altered-context features alongside clean candidates under the original fixed bank budget and readout.The target size B equals the clean-baseline bank size.
  • Reconstruction branch: The reconstruction branch preserves the original clean objective while adding altered-context encoder targets only at ring positions.Positions in the excluded set receive no direct loss but remain visible to contextual attention.
  • Feature identity: Matched controls show that altered-context feature identity matters: replacing it with the clean-view feature lowers P-AP in both memory and reconstruction branches.The controls keep the selected positions, inputs, rings, budgets, and other training or inference choices fixed while changing feature identity.

5 Experiments

Across MVTec AD, VisA, and Real-IAD, BOUNDARYSUPPORT improves pixel localization broadly and changes normal responses most strongly near defects. The results also show that P-AP depends on the complete defect-versus-normal ranking rather than any single score summary or metric.

  • 5.2 P-AP improves in all six detector–dataset settings: 329/342 category–seed pairs show higher P-AP across the six detector–dataset settings.Memory improves all 171 pairs, while reconstruction improves 158/171.
  • 5.5 Failure cases expose the same ranking trade-off: Real-IAD reconstruction gains 4.20 ± 0.12 P-AP while image AUROC falls 0.52 ± 0.04.This demonstrates that the measured effect is a pixel-localization ranking improvement rather than monotone improvement across metrics.
  • 5.3 Altered-context features are the effective normal evidence: Altered-context features outperform clean-view features by 2.23 P-AP points on MVTec and 1.84 on VisA at identical selected locations.Positive category-level differences occur in 26 of 27 categories, while AUPRO and image AUROC remain mixed or nearly unchanged.
  • 5.4 Where does the final score change?: Near-defect normal scores decrease relative to farther normal scores in all 15 MVTec categories, with median near-minus-far and near-minus-mid contrasts of −0.51 and −0.43 on the 100∆s scale.Far-normal and clean-image normal scores increase slightly rather than decrease.
  • 5.1 Useful normal patches are missing from clean images: Near-defect normal patches recover 94.70% of the fixed-budget oracle gain, with oracle gains decreasing from near to mid to far candidate bands.The near pool contains ground-truth-normal patches within two patch cells of the annotated defect.

6 Discussion

The discussion argues that normal-reference composition and contextual representation coverage are first-order detector design factors. It also identifies localization trade-offs and representation-family limits that remain open.

  • Reference composition: A 3.62-point P-AP gain shows that changing reference identity alone makes normal-evidence composition a first-order detector factor.The fixed-budget oracle holds representation, test-time score, and reference count fixed.
  • Representation coverage: Image-level cleanliness does not guarantee feature-level coverage because contextual encoders represent patches differently when surrounding context changes.Normal evidence depends on represented contextual states, not only nominal source images.
  • Clean-only intervention: BoundarySupport excludes tokens whose pixel footprints change under controlled context modification before adding normal references or reconstruction targets.Unlike the oracle, this clean-only intervention neither reuses real-defect pixels nor relies on defect annotations.
  • Cross-paradigm evidence: Matched controls show altered-context representations provide more useful localization evidence for both reconstruction targets and memory references.The two computations use the feature differently, but exhibit the same ordering in the tested settings.
  • Limits and open problems: Localization gains can trade off against defect evidence, while the current agreement is limited to DINOv2 ViT-B-family representations.Other encoders remain an empirical question, and future work should preserve defect ordering while identifying deployment-relevant missing patches directly.

7 Conclusion

The paper finds that clean-only training can omit useful normal features, especially near defects, and transfers this insight to clean-only training through BOUNDARYSUPPORT. Across tested datasets and detector paradigms, the method consistently improves pixel localization.

  • At a fixed memory budget, adding ground-truth-normal patches from defective images raises MVTec P-AP from 73.34 to 76.95.Patches within two cells of defects recover 94.70% of the fixed-budget gain.
  • BOUNDARYSUPPORT transfers the diagnostic insight to clean-only training by altering context, excluding modified tokens, and learning from preserved neighbors.
  • Across memory and reconstruction on three datasets and three paired seeds, BOUNDARYSUPPORT consistently improves pixel localization.Matched controls show altered-context features outperform clean-view features as both targets and references.

A.1 Datasets and evaluation

The evaluation uses normal-only training and labeled test splits from MVTec AD, VisA, and Real-IAD, with method configuration separated from Real-IAD outcome inspection. Memory and reconstruction share standardized crop and scoring conventions while retaining branch-specific inference details.

  • Datasets: MVTec AD, VisA, and Real-IAD provide normal-only training splits and labeled test splits for evaluation.MVTec and VisA configure the method, while Real-IAD settings are fixed before inspecting evaluation outcomes.
  • Evaluation protocol: All six detector–dataset settings are evaluated with seeds 0, 1, and 2, with Real-IAD donors restricted to the same category and camera view.
  • Preprocessing: The memory and reconstruction branches use standardized RGB preprocessing with a 392 × 392 crop and a 28 × 28 token grid from patch size 14.Memory uses aspect-preserving resize for MVTec, while VisA and Real-IAD use square resize; reconstruction uses square resize on all datasets.
  • Scoring: Memory and reconstruction apply distinct smoothing, resizing, and top-score image aggregation rules, while BoundarySupport leaves the test-time input and scoring path unchanged.
  • Metrics: Metrics include image-level I-AUROC, I-AP, and maximum I-F1, plus pixel-level P-AUROC, P-AP, maximum P-F1, and AUPRO.All metrics use a 0–100 scale, with dataset results averaged arithmetically over categories.
  • Diagnostic design: The diagnostic freezes DINOv2-B/14, the five-bank memory readout, a 5% memory ratio, and evaluation geometry while comparing clean and oracle memories on matched held-out images.Ground-truth masks identify normal-labeled patches inside real defective images.

B.1 Candidate identity and memory budget

The candidate-identity diagnostic isolates the effect of adding normal patches from defective images under a fixed memory budget, then analyzes their spatial origin and the clean-only BoundarySupport implementation. Near-defect patches provide most of the demonstrated gain, while altered-context features are integrated into fixed-size memory construction.

  • Candidate identity and memory budget: The fixed-budget GT-normal oracle restores the clean-only absolute bank count after enlarging the candidate pool.This isolates candidate identity from a change in final memory size.
  • Candidate identity and memory budget: The fixed-budget candidate change raises P-AP by 3.62 points, while expanded and fixed-budget banks differ by only 0.01 point.
  • Candidate identity and memory budget: The fixed-budget oracle improves 14 of 15 categories, while the near-defect variant improves 13 of 15.
  • Robustness checks: Inpainting and cross-defect transfer test whether the gain depends on visible defect pixels or exact defect memorization.The procedures remove visible annotated defects or collect candidates beside one defect type while evaluating other types.
  • Reference selection: Near-defect patches are a small candidate-pool fraction but a larger fraction of selected references, with selected shares rising from 4.95% to 15.98% in leather and 4.20% to 8.65% in tile.
  • BoundarySupport implementation: BoundarySupport generates clean/synthetic pairs, measures context-related changes, excludes affected tokens, and selects preserved altered-context features alongside clean candidates under the original bank budget.The memory branch uses fixed DINOv2 features, approximate greedy k-center selection, and exactly B final references per bank.

G Full Dataset-Level Metrics

BoundarySupport improves localization by changing preserved normal context and stored features rather than merely changing candidate positions or architecture. Across matched controls, altered-context targets and features consistently outperform clean-view alternatives, with gains concentrated near defects.

  • 329/342 paired comparisons show increased P-AP overall across memory and reconstruction branches.Memory is positive in 171/171 category–seed pairs, while reconstruction is positive in 158/171.
  • Control 3: Reconstruction target identity: 7.13 points below baseline for clean targets contrasts with 4.94 points above baseline for altered-context targets on MVTec.The clean target decreases P-AP in 15/15 categories, whereas the altered-context target improves all 15.
  • Control 5: Distance-resolved final memory scores: -0.51 and -0.43 category-median contrasts show that near-defect normal responses change more than mid- or far-distance responses.Near-minus-far and near-minus-mid contrasts are negative in 15/15 categories on the 100∆s scale.
  • Reconstruction controls: The controls keep the reconstruction architecture and auxiliary weight fixed while changing the supervised patch set or objective.The clean-view patch applies the auxiliary objective to matched clean-view positions, while other controls alter ring exclusion or apply a margin objective to the synthetic core.

J Generator and Hyperparameter Analysis

The generator and hyperparameter analyses test whether BoundarySupport’s gains persist across intervention settings, score behavior, and benchmark branches. Positive P-AP changes are broadly consistent, although one reconstruction trade-off lowers I-AUROC and policy differences are small.

  • Hyperparameter robustness: Positive P-AP persists for r ∈{1, 2, 3} and λ ∈{0.025, 0.05, 0.1} across all three intervention families.These results come from the ten-category development subset.
  • Score behavior: At seed 0, high normal-region anomaly scores and mean defect scores decrease together on all three datasets while P-AP increases.The same macro direction repeats over all completed MVTec and VisA seeds.
  • Category-level consistency: 57/57 memory categories and 52/57 reconstruction categories improve P-AP at seed 0.Across all three seeds, the corresponding sign counts are 171/171 and 158/171.
  • Metric trade-offs: -0.52 ± 0.04 I-AUROC accompanies +4.20±0.12 P-AP in Real-IAD reconstruction.P-F1 changes by +2.56±0.09 and AUPRO by +0.16±0.03 in the same comparison.
  • Policy selection: 54.12 versus 54.05 P-AP shows only a 0.07-point difference between mixed and uniform VisA policies at seed 0.The uniform policy is used in the main paper because the difference is small relative to the gain over the 52.23 clean baseline.

N Compute and Reproducibility Details

The implementation fixes numerical precision, training settings, and reporting procedures to support reproducibility. BoundarySupport adds offline or training computation while leaving the deployed architecture, bank size, and scoring path unchanged.

  • Numerical and training settings: FP16 stores memory caches and banks, while candidate, coreset, and readout distances use FP32.Reconstruction uses FP32 training, 5,000 iterations per category, and batch size 16.
  • Compute boundary: BoundarySupport reconstruction increases training cost while leaving the deployed architecture and scoring path unchanged.The memory branch adds work during offline synthesis, candidate scoring, and bank reconstruction while keeping final bank size and readout fixed.
  • Reporting and release: Reported tables are regenerated from category-level results using ROUND HALF UP to two decimals.Algorithm-defining constants retain their stated precision, and the release includes scripts for result aggregation and table/figure regeneration.
  • Qualitative evaluation: Qualitative figures use a fixed column order across MVTec AD, VisA, and Real-IAD, while quantitative claims rely on category-level results.The figures cover all 57 benchmark categories.
Loading 2608.23299v1…