Source-linked AI summary
When More References Hurt: Contamination-Aware DINOv2 Memory Banks for Few-Shot Steel Defect Detection
Hannaneh Kalantary, Javad Khoramdel
TL;DR
Patch-memory anomaly detectors can be harmed by anomalous patches in unverified reference images, motivating contamination-aware expansion from a small trusted seed. The paper filters candidate patches by seed compatibility before fixed-budget memory construction and reports consistent held-out gains over naive expansion, while noting incomplete coverage and method-specific scope limits.
Problem
Unverified industrial references may contaminate patch-memory detectors that assume stored patches are normal, so the paper asks whether trusted normal images can safely recover useful patches without defect masks.
Method
DT20 scores unverified candidate patches by distance to a clean seed, rejects the most suspicious 20%, merges retained patches with the seed, and applies greedy coreset selection at a fixed budget.
Results
On all five completed held-out pairs, DT20 improves over naive expansion, with a mean gain of +0.0142 AUPRC.
Takeaways & Limitations
Reference purity is a first-order variable, and unverified images are useful only when their contribution is filtered explicitly.
Takeaways & Limitations
Held-out evidence covers only folds 1 and 2 with an incomplete result matrix, and the trim rate, backbone, and scoring rule are not established across other settings.
Abstract
from arXiv · showhide
Patch-memory anomaly detectors assume that their reference bank is normal, an assumption that is difficult to guarantee when additional industrial images are unverified. We study whether a few trusted normal images can safely recover useful normal patches from such references without defect masks. Starting from the DINOv2 patch-memory formulation used by AnomalyDINO, we score candidate patches by distance to a clean seed bank, discard the most suspicious 20%, merge the retained patches with the seed, and enforce a fixed budget by greedy coreset selection. On Severstal, naive additional references contain 9.46% anomalous patches; the proposed trim rejects 78.1\% of them and reduces residual contamination to 2.59%. At an equal 51,200-patch development budget, the proposed bank reaches 0.1084 AUPRC versus 0.0950 for naive expansion, 0.0952 for random removal, and 0.1030 for eight clean images. Injecting only 0.5\% anomalous patches into a clean bank reduces AUPRC from 0.1030 to 0.0759. On all five completed held-out pairs, the proposed bank improves over naive expansion, with a mean gain of 0.0142 AUPRC. Reference purity is therefore a first-order design variable, and unverified images are useful only when their contribution is filtered explicitly.
1 Introduction
Steel inspection often relies on labeled classification, detection, or segmentation, but industrial deployments may have few verified examples. This paper asks whether trusted normal images can safely recover useful normal patches from unverified references without defect masks.
- Supervised steel inspection methods model defects effectively when labeled examples are available.
- DINOv2 enables generic visual features without task-specific fine-tuning, motivating lower-supervision industrial anomaly detection.
- Every stored patch helps define normality, so an anomalous region in an otherwise useful reference can mislead a patch-memory detector.
- The study quantifies contamination sensitivity and tests mask-free filtering against oracle, random, clean-shot, and exact-budget controls.
2 Related Work
Prior steel inspection work commonly uses supervised labels, while patch-based anomaly detectors reduce supervision by modeling nominal visual features. This paper focuses on contamination in reference memories, a failure mode not addressed by compression alone.
- Supervised steel methods span multi-label recognition, pixel-level segmentation, and combined detection-segmentation when labels are available.
- ViT and DINOv2 representations transfer across domains without task-specific fine-tuning, supporting low-supervision inspection.
- PaDiM models patch-feature distributions, whereas PatchCore and AnomalyDINO use representative or nearest-neighbor patch memories of nominal data.
- Coreset selection reduces redundancy but does not guarantee removal of anomalous patches already present in reference images.
3 Method
The method expands a trusted clean patch memory with unverified candidates, filters candidates by compatibility with the clean seed, and then enforces an exact memory budget. Controls separate purification effects from memory size and random removal.
- 3.1 Base Detector and Reference Expansion: Frozen DINOv2 patch features are stored in a normal memory bank, and query anomalies are scored by nearest-neighbor distance.
- 3.1 Base Detector and Reference Expansion: Naive expansion inserts all candidate patches, risking that local defects in unverified images become part of normality.
- 3.2 Distance-Trimmed Purification: DT20 ranks candidates by distance to the trusted seed and rejects the largest fixed fraction, retaining patches most compatible with that seed.
- 3.2 Distance-Trimmed Purification: The frozen DT20 configuration uses a 20% trim, merges retained candidates with the clean seed, and greedily selects an exact 51,200-patch final memory.
- 3.3 Controls: Clean, naive, random20, and oracle controls distinguish targeted purification from effects of memory capacity, reference count, or coreset selection.
4 Experimental Setup
Experiments use Severstal steel-strip images with DINOv2 patch features, mask-based patch evaluation, stratified fold controls, and AUPRC as the primary metric. Development settings freeze the trim rate and memory budget before held-out evaluation.
- Severstal provides 256 ×1600 steel-strip images and run-length-encoded masks for four defect classes.
- The primary configuration uses DINOv2 ViT-S/14 at 448-pixel short-edge resolution, producing 6,400 patch tokens per reference image.
- Ground-truth masks label a patch anomalous when at least 50% of its area overlaps a defect, while DT20 never uses these masks.
- Fold 0 freezes the 20% trim and 51,200-patch budget before held-out evaluation, which covers folds 1 and 2 over completed fold-seed cells.
- 4.3 Metrics: Patch-level AUPRC is primary because anomalous patches are strongly imbalanced, with AUROC and F1-max as supporting views.
5 Results
Results show that targeted distance trimming improves patch-memory quality under contamination, while matched-budget controls rule out memory size or coreset selection as sufficient explanations. The method also remains better than naive expansion across all five completed held-out pairs, although clean verified references perform best when available.
- Purification quality: 9.46% of candidate patches were anomalous; DT20 rejected 78.1% of anomalous candidates and reduced residual contamination to 2.59%.It retained 86.1% of normal candidates, indicating a deliberate purity–diversity trade-off.
- Trim selection: 0.1084 AUPRC was the best result among 5%, 10%, and 20% fixed trims, which reached 0.0987, 0.1022, and 0.1084, respectively.The 20% trim also reached 0.8132 AUROC and 0.2151 F1-max and was frozen before held-out evaluation.
- Exact memory-budget controls: 0.1084 AUPRC was achieved by DT20 at the fixed 51,200-patch budget, versus 0.0950 for naive expansion and 0.0952 for Random20.DT20 also exceeded clean-8 at 0.1030 on fold 0.
- Exact memory-budget controls: Matched-budget results indicate that targeted patch filtering, rather than greedy coreset selection or indiscriminate removal, drives the improvement.Naive and Random20 were nearly identical at 0.0950 and 0.0952.
- Controlled contamination: 0.5% anomalous replacement reduced AUPRC from 0.1030 to 0.0759 at constant memory size, showing strong sensitivity to memory contamination.Performance declined to 0.0587 at 20% contamination.
- Held-out results: Clean-8 achieved the highest completed-cell held-out mean AUPRC at 0.1406, making verified clean references the safest strategy when available.DT20 targets settings where additional images cannot be exhaustively certified.
- Held-out results: +0.0142 AUPRC was the mean DT20 gain over naive expansion across all five completed held-out pairs.The individual paired gains were all positive, ranging from +0.0055 to +0.0276.
6 Discussion
The experiments show that reference purity is central to patch-memory anomaly detection, while DT20 trades reference certainty for appearance coverage. Held-out evidence supports contamination-aware expansion when trusted references are scarce, but not as a replacement for sufficient verified references.
- 6 Discussion: A nearest-neighbor detector cannot distinguish anomalous memory patches from true normal patches once both are stored.Controlled contamination therefore makes reference purity a first-order design variable.
- 6 Discussion: DT20 rejects candidate patches least compatible with a clean seed rather than segmenting defects or predicting defect classes.Matched-budget controls indicate that its gain is not explained by larger memory capacity or coreset selection alone.
- 6 Discussion: 78.1% of anomalous candidates are rejected while 86.1% of normal candidates are retained, but reject precision is only 0.369.The method deliberately sacrifices some unusual normal patches for safety.
- 6 Discussion: 0.1140 AUPRC is achieved by combining four clean and eight additional images at the same final budget on fold 0.This supports improving appearance coverage through filtered additional imagery without increasing memory capacity.
- 6 Discussion: Clean-8 remains stronger than DT20 2+8 on currently available held-out folds, so verified clean references remain preferable when enough are available.Contamination-aware expansion is instead positioned for small trusted seeds and plentiful unverified imagery.
7 Limitations
The evaluation has incomplete held-out coverage and several scope boundaries that constrain how broadly the results should be interpreted. The study also requires operating-point calibration and excludes incomplete experimental branches.
- 7 Limitations: The held-out matrix covers only folds 1 and 2, with one DT20/oracle cell missing for each method.These results should not be interpreted as completed five-fold cross-validation.
- 7 Limitations: The 20% trim rate is selected on a single development fold and may not be optimal across different steel lines, illumination, or acquisition conditions.
- 7 Limitations: Quantitative contamination sensitivity is evaluated with one DINOv2 backbone and one nearest-neighbor scoring rule.Results may differ for other feature representations or anomaly scores.
- 7 Limitations: Severstal pixel masks enable oracle contamination analysis, but residual contamination cannot be measured directly in deployment.
- 7 Limitations: Fixed-threshold F1 is unstable across reference compositions, and incomplete SAM2, efficiency, anomaly-memory, and attention branches are excluded.A production system would require explicit operating-point calibration.
8 Conclusion
The paper finds that unverified reference images can harm patch-memory anomaly detection, but a trusted seed can filter them to recover useful normal appearance. Verified clean references remain safest when available.
- 8 Conclusion: 0.5% controlled anomaly contamination sharply reduces AUPRC, showing that more references can hurt when their patches are not guaranteed normal.
- 8 Conclusion: DT20 filters candidate patches using a small trusted seed, substantially reduces contamination, and improves over naive expansion on every completed held-out pair.
- 8 Conclusion: Verified clean references remain the safest option when available, while contamination-aware expansion is suited to scarce trusted seeds and plentiful risky imagery.