PaperlayerView exact revision on arXiv

Source-linked AI summary

What Memory Composition Does Not Tell Us About Anomaly Detection

Joongwon Chae, Runming Wang, Peiwu Qin

arXiv:2608.23295v1cs.CV

TL;DR

Memory-based anomaly detectors can turn geometrically rare contaminated patches into deployed normal references. The paper compares selectors under fixed detector scaffolds and tests CLEANCON, finding that it reduces contamination and improves P-AP, while purity alone does not identify the best-performing memory.

  • Problem

    Sparse contamination may be preferentially selected by global coverage and then retained as normal inference references, but the strength of this bias and its detection impact require measurement.

  • Method

    The paper compares random, medoid, Local FF, and Global FF selectors, then applies CLEANCON to change candidate-image eligibility while fixing representation, memory size, builder, and inference rule.

  • Results

    CLEANCON reduces final-memory contamination to approximately zero and increases category-macro P-AP in all 12 matched comparisons, while P-AP can continue rising as contamination reappears.

  • Takeaways & Limitations

    Memory contamination does not order memories by P-AP; useful memories depend on structure beyond purity alone.

  • Takeaways & Limitations

    CLEANCON changes candidate-image eligibility, while selected rare valid patches may remain anomalous anchors; the achieved-radius certificate covers only 0.58–6.32% of selected contamination.

Abstract

from arXiv · show

Memory-based anomaly detectors store nominal training patches and score test patches against this memory. A patch selected for coverage therefore becomes a nor- mal reference without a separate check that geometric rarity makes it safe to trust. We probe this coupling with sparse training contamination. Under fixed representa- tions and memory budgets, we compare random, medoid, local, and global coverage selectors. We then use CLEANCON, an out-of-bag cross-image support gate that changes candidate-image eligibility while fixing the representation, absolute mem- ory size, builder, and inference rule. Global coverage strongly over-represents sparse contamination. CLEANCON reduces final-memory contamination to approx- imately zero and increases category-macro P-AP in all 12 matched comparisons. Yet along a retention sweep, the lowest-contamination memory does not attain the highest P-AP; performance continues to improve while contamination rises. Mem- ory contamination therefore does not order the resulting memories by P-AP

1 INTRODUCTION

Memory-based detectors reuse selected patches both for feature-space coverage and as direct normal references at inference. The paper tests whether sparse contamination is preferentially selected and whether reducing contaminated references improves detection.

  • 1 INTRODUCTION: Selected patches remain direct normal references at inference, so memory is more than a compressed training set.This couples coverage-based selection with the detector’s deployed normal evidence.
  • 1 INTRODUCTION: Sparse contamination tests whether rare anomalous patches become attractive to global coverage selection and whether contaminated references affect detection.The study asks both how strong the selection bias is and whether reducing contaminated references improves the detector.
  • 1 INTRODUCTION: Random, medoid, Local FF, and Global FF are compared under identical representations and memory budgets.CLEANCON changes candidate-image eligibility while fixing the representation, absolute memory size, builder, and inference rule.
  • 1 INTRODUCTION: Global FF draws substantially more contaminated patches into memory, while CLEANCON reduces final-memory contamination to approximately zero and increases category-macro P-AP in all 12 matched comparisons.The intervention changes candidate-image eligibility without changing the downstream detector scaffold.
  • 1 INTRODUCTION: The lowest-contamination memory does not attain the highest P-AP; performance continues to improve as contamination rises.Thus, memory contamination does not order the resulting memories by P-AP.

2 RELATED WORK

Prior work develops memory-based anomaly detectors, outlier-robust selection, and alternative memory-construction strategies. This paper distinguishes its matched intervention by changing candidate-image eligibility while holding the deployed detector scaffold fixed.

  • 2 RELATED WORK: SPADE and PaDiM model nominal patterns with pretrained local features, while PatchCore uses coreset selection and nearest-memory distances.Later methods strengthen representations, adapt features, or introduce projection- and consistency-based scoring.
  • 2 RELATED WORK: One-class learning and anomaly detection have long addressed latent outliers and noisy nominal data through methods such as SoftPatch, InReaCh, FUN-AD, and MeDS.These methods modify memory construction, inference, reconstruction, or score distillation in different ways.
  • 2 RELATED WORK: The paper’s matched intervention fixes the deployed detector while changing candidate-image eligibility, measuring selection amplification and detector response within the same scaffold.This separates the effect of memory filtering from changes in detector, backbone, or contamination construction.
  • 2 RELATED WORK: Coverage methods seek limited-center representations of a space, whereas downstream performance may depend on objectives beyond geometric coverage.This motivates examining memory composition as an inference-time reference set rather than only as a compression device.

3 PROBLEM SETUP AND EXPERIMENTAL PROTOCOL

The study uses frozen representations, injected anomalous images, labeled analysis partitions, and fixed detector scaffolds to measure memory contamination under controlled selection and filtering protocols.

  • 3 PROBLEM SETUP AND EXPERIMENTAL PROTOCOL: A frozen encoder maps training images to patch descriptors, and a builder selects K references from candidate set X to form memory M_K.For analysis, candidates are partitioned into normal N and contaminated Q, but these labels are withheld from selector and detector.
  • 3 PROBLEM SETUP AND EXPERIMENTAL PROTOCOL: Contaminated grid cells correspond to image regions containing at least one anomalous pixel, with no anomaly-area threshold applied.Labels are aligned to the 28×28 feature grid; WRN50 labels refer to grid positions rather than entire receptive fields.
  • 3 PROBLEM SETUP AND EXPERIMENTAL PROTOCOL: Amplification compares contamination prevalence in the candidate pool with occupancy in final memory, where A_K > 1 indicates over-representation and memory purity is 1 − p_mem.The protocol also records pooled category and depth statistics for the DINOv2/ProCon analyses.
  • 3 PROBLEM SETUP AND EXPERIMENTAL PROTOCOL: All 15 MVTec AD and 12 VisA categories receive injected anomalous images, while anomalous image paths are excluded from evaluation.The main analyses use nominal 5% image-level contamination across three folds and a default memory of 1% of the unfiltered candidate pool.
  • 3 PROBLEM SETUP AND EXPERIMENTAL PROTOCOL: The experiments evaluate PatchCore with WRN50 and ProCon with frozen DINOv2, comparing random, medoid, Local FF, and Global FF selection.Reported metrics include category-macro I-AUROC, P-AUROC, P-AP, and AUPRO; matched CLEANCON comparisons fix representation, preprocessing, builder, inference, and memory size.

4 GLOBAL COVERAGE OVER-REPRESENTS SPARSE CONTAMINATION

Global farthest-first coverage strongly over-represents sparse contamination, especially at small memory budgets. The analysis shows that some contaminated centers may be needed for the achieved covering radius, but most amplification reflects selector behavior rather than established necessity.

  • 4 GLOBAL COVERAGE OVER-REPRESENTS SPARSE CONTAMINATION: At a 1% memory budget, Global FF’s three-fold mean amplification ranges from 16.04× to 40.61×, with one fold reaching 43.57×.These values span four dataset–representation settings.
  • 4 GLOBAL COVERAGE OVER-REPRESENTS SPARSE CONTAMINATION: At 1%, random remains within 0.99–1.02×, medoid within 1.05–1.44×, Local FF within 2.55–3.42×, and Global FF reaches 16.41–43.57×.Restricting farthest-first competition to individual images removes most of the amplification.
  • 4 GLOBAL COVERAGE OVER-REPRESENTS SPARSE CONTAMINATION: Coverage favors rare candidates because they can yield large uncovered-space gains, without distinguishing rare normal variation from contamination.This provides the geometric explanation for global coverage’s selection bias.
  • 4 GLOBAL COVERAGE OVER-REPRESENTS SPARSE CONTAMINATION: A_K = 43.57 means approximately 43.57% of contaminated candidate descriptors enter the final 1% memory.Amplification can therefore be interpreted as the fraction of contaminated candidates selected.
  • 4 GLOBAL COVERAGE OVER-REPRESENTS SPARSE CONTAMINATION: Amplification decreases as memory budgets increase from 0.1% to 10%, with the largest effects in the sparse-memory regime.The curves are descriptive because A_K has a K-dependent ceiling.
  • 4 GLOBAL COVERAGE OVER-REPRESENTS SPARSE CONTAMINATION: A certified packing lower bound establishes that 0.58–6.32% of selected contaminated centers are required to maintain the achieved covering radius in four pooled fold-0 settings.The certificate does not establish necessity for the remaining contaminated centers.

5 CLEANCON: A CONTROLLED INTERVENTION THAT REDUCES CONTAMINATED REFERENCES

CLEANCON filters training images using out-of-bag cross-image support scores before unchanged memory construction, while preserving representation, memory size, builder, and inference. The procedure retains low-scoring images and builds the fixed-size memory from all their descriptors.

  • CLEANCON changes candidate-image eligibility while keeping the representation, absolute memory size, builder, and test-time inference unchanged.
  • Each training image is scored by how well its features are explained by features from other training images, excluding banks containing that image.The out-of-bag restriction prevents an image from using its own descriptors as support.
  • The method uses 20 support banks, each containing ⌈0.20n⌉ images, with shared image indices across feature depths but separate descriptor banks per layer.
  • Only images at or below the within-category median score are retained, and all descriptors from retained images enter memory construction.Filtering operates at the image level rather than deleting individual patches.
  • The final memory size remains fixed after filtering by selecting the same absolute target K as the unfiltered pipeline.The target K is set to 1% of the unfiltered candidate pool.

6 THE CLEANEST MEMORY IS NOT ALWAYS THE BEST MEMORY

Matched experiments show that CLEANCON reduces final-memory contamination to approximately zero while increasing category-macro P-AP in all 12 comparisons. However, retention results show that higher P-AP can coincide with higher contamination, so memory purity alone does not rank performance.

  • 6.1 CLEANCON REDUCES CONTAMINATION AND INCREASES MACRO P-AP: CLEANCON increases category-macro P-AP in all 12 dataset–representation–fold comparisons while reducing final-memory contamination to approximately zero.Three-fold mean gains are +3.438 for MVTec–WRN50, +3.231 for MVTec–DINOv2, +0.380 for VisA–WRN50, and +4.319 for VisA–DINOv2.
  • 6.2 CONTAMINATION AND P-AP CAN RISE TOGETHER AS RETENTION INCREASES: R80 exceeds R50 in all 12 comparisons, while contamination is higher in 11 of them.The setting-level mean differences are +0.929, +0.521, +0.875, and +2.293 points.
  • 6.2 CONTAMINATION AND P-AP CAN RISE TOGETHER AS RETENTION INCREASES: Across the full sweep, P-AP can continue rising after contamination reappears, although performance eventually falls at R90 on VisA.On MVTec, P-AP rises from R50 through later retention after contamination returns; on VisA, both rise through R80 before P-AP falls at R90.
  • 6.3 INCREASING THE NUMBER OF NORMAL SOURCES DOES NOT REPRODUCE THE GAIN: Increasing normal-source coverage alone does not explain the retention gain.A fixed-budget control increased Rsrc by 20.88 percentage points but changed P-AP by −0.137 ± 0.024 points.
  • CLEANCON has higher macro P-AP in five of six matched comparisons against the SoftPatch-style control.On VisA–WRN50, the scores are 40.061 for CLEANCON and 41.495 for the SP-style control.

7 WHAT IS STORED AND WHAT IS USEFUL ARE DIFFERENT

Memory contamination and detector usefulness are related but distinct: contamination measures what entered memory, whereas P-AP also depends on retained normal states, densities, and local geometries.

  • Memory contamination and P-AP can rank memories differently because inference depends on how retained references relate to test patches.The detector scores retained references against test patches, so contamination alone does not determine usefulness.
  • Broader candidate pools outperform the cleanest tested memory over part of the retention sweep, while adding more normal source images alone does not reproduce the gain.Increasing retention admits normal and contaminated candidates together, so the sweep does not isolate purity alone.
  • Memory construction, candidate filtering, and inference impose different orderings over memories.Coverage selects representatives, filtering controls retained training evidence, and inference evaluates how well that memory explains test samples.

8 LIMITATIONS AND CONCLUSION

CLEANCON sharply reduces contamination and improves matched P-AP, but the cleanest memory is not consistently the most useful, and the method’s scope remains limited.

  • CLEANCON changes candidate-image eligibility, while selecting rare valid patches without restoring anomalous anchors remains open.The achieved-radius certificate covers 0.58–6.32% of selected contamination, and retention is evaluated at five fixed-hyperparameter operating points.
  • CLEANCON drives final-memory contamination to approximately zero and raises category-macro P-AP in all 12 matched comparisons.P-AP can continue rising as contamination reappears along the retention sweep.
  • Useful memories have structure beyond purity alone because increasing normal-source count does not reproduce the gain.The cleanest memory and highest P-AP occur at different operating points.

A.1 CONTAMINATION PROTOCOLS AND NO-OVERLAP EVALUATION

The study injects anomalous images into nominal training data under two contamination protocols, removes injected images from evaluation, and compares selectors under fixed representations and memory budgets.

  • Contaminated training sets are formed by injecting anomalous images into clean training images under Exact-Final and Addition protocols.Exact-Final 5% targets approximately 5% anomalous images in the mixed set, while Addition-0.10 corresponds to approximately 9.09% final contamination.
  • Injected anomalous images are removed from downstream evaluation, with each fold using different injected images and its corresponding evaluation split.Results report means and sample standard deviations across three contamination folds.
  • Contamination labels align ground-truth masks to a 28 × 28 feature grid without an anomaly-area threshold.A descriptor is contaminated when its corresponding image region contains at least one anomalous pixel.
  • The default final memory retains an absolute 1% of the unfiltered candidate pool, and P-AP is the primary localization metric.The same memory size is retained after CLEANCON filtering; metrics are computed per category and macro-averaged.
  • Selectors are compared under identical representations and absolute memory budgets, including random, medoid, Local FF, and Global FF procedures.Random controls use 20 repetitions, while Local FF allocates capacity across images before within-image farthest-first selection.

B CLEANCON IMPLEMENTATION

CLEANCON ranks images using out-of-bag cross-image residuals, retains low-scoring images before the original builder, and evaluates amplification, selector behavior, and achieved-radius necessity.

  • B CLEANCON IMPLEMENTATION: CLEANCON computes out-of-bag residuals from banks excluding each target image, aggregates them across layers and depths, and ranks images by their highest-residual patches.The image score averages the highest 0.5% patch residuals.
  • B CLEANCON IMPLEMENTATION: The method retains the lowest-scoring 50%, 60%, 70%, 80%, or 90% of images and passes all retained descriptors to the original memory builder.The final memory banks keep fixed target sizes after filtering.
  • C.1 BUDGET-DEPENDENT AMPLIFICATION: Among 162 category-level trajectories, 156 are non-increasing on the coarse budget grid.All six increases occur between 0.1% and 0.5%.
  • C.3 ACHIEVED-RADIUS CERTIFICATE: The achieved-radius certificate directly certifies 0.58%–6.32% of selected contaminated centers across all four settings.The certificate establishes necessity for some contaminated centers, but not for the remainder.
  • D.1 DISTRIBUTION OF MATCHED INTERVENTION GAINS: The matched intervention improves P-AP in all 12 dataset–representation–fold macro comparisons and in 89 of 162 category–fold comparisons.The VisA–WRN50 win count is 13/36.

D.2 CATEGORY-LEVEL RETENTION RESULTS

Category-level retention results show broad P-AP gains for CLEANCON, but higher retention can improve detection while also increasing contamination. Control experiments indicate that these gains are not explained by simply expanding normal-source representation or changing the builder.

  • Retention sweep: R80 exceeds R50 in all 12 runs, while contamination is higher at R80 in 11, showing that stronger retention can improve detection alongside increased contamination.Removing the largest positive MVTec–WRN50 category outlier still leaves a mean gain of +0.292; removing pcb4 on VisA–WRN50 changes the mean difference to −0.156.
  • Fold variation: The matched P-AP changes are distributed across contamination folds, with ticks representing three-fold means and labels reporting category–fold wins.The figure summarizes fold-level variation rather than a single aggregate comparison.
  • Control experiments: Normal-source expansion raises Rsrc by 20.88 percentage points in the 15-category MVTec experiment but changes P-AP by −0.137 ± 0.024 points.On clean training data, the same 50% image-retention rule reduces P-AP by 0.939 points.
  • Control experiments: The risk-aware builder reduces pmem at both tested ratios, changing P-AP from 72.848 to 72.814 at 5% and from 70.415 to 70.573 at 10%.This intervention modifies candidate traversal and replaces a subset of high-risk centers while retaining the coverage-building framework.
Loading 2608.23295v1…