PaperlayerView exact revision on arXiv

Source-linked AI summary

LUMIN: Lightweight Universal Manufacturing Inspection Network for Anomaly Detection

Pengfei Yang

arXiv:2609.04775v1cs.CV

TL;DR

Industrial anomaly detection is constrained by slow memory-bank construction and inefficient inference under production latency and resource requirements. The paper introduces PSP, inference optimizations, and the lightweight LUMIN architecture to address these bottlenecks. Across five benchmarks, PSP preserves sampling accuracy at near-random construction cost, while stratified sampling reduces evaluation time 20× with stable metrics.

  • Problem

    Industrial anomaly detection must meet strict real-time and resource constraints, but memory-bank construction remains slow and inference pipelines contain substantial redundancy.

  • Method

    The paper combines PSP’s metadata-driven four-stage sampling pipeline with parallel similarity computation, stratified pixel sampling, and progressive segmentation-head lightweighting in LUMIN.

  • Results

    PSP achieves P-AUPR 57.92, matching KM MF and GSS MF, while sampling in 14.98 s at k=20 and N=3,629 with zero backbone forward passes.

  • Takeaways & Limitations

    The results support PSP as an efficient memory-bank sampling approach and LUMIN as a lightweight architecture for resource-constrained industrial anomaly detection.

Abstract

from arXiv · show

Industrial anomaly detection faces two engineering bottlenecks: memory bank construction latency and inference efficiency. Traditional sampling algorithms (Farthest Point Sampling, K-Means, etc.) rely on numerous backbone forward passes and iterative distance computations, with construction times ranging from minutes to hours; heavy computation components such as multi-scale feature extraction struggle to meet the millisecond-level real-time requirements of production lines. This paper focuses on sampling efficiency and inference optimization for industrial deployment with two core contributions: (1) PSP (Plugin Sampler Pipeline)---a four-stage adaptive memory bank sampling pipeline based on 18-dimensional pixel metadata and five complementary visual plugins. PSP completes all sampling with zero backbone forward passes; coarse filtering is sub-second numerical sorting, and metadata extraction is a one-time offline cost. PSP supports progressive deployment and incremental updates. (2) Two engineering optimization strategies---parallel memory bank similarity computation (reducing inference memory and latency by over 95\%) and stratified pixel sampling for large-scale evaluation (reducing computation time by 20$\times$ while keeping metrics stable). As a vehicle for validation, we introduce LUMIN (Lightweight Universal Manufacturing Inspection Network) with extreme segmentation-head compression, systematically exploring the accuracy-efficiency frontier against strong baselines. Experiments on five benchmarks demonstrate that PSP matches state-of-the-art sampling accuracy at near-random construction cost (341$\times$ faster than FPS), while inference optimizations reduce evaluation time by 20$\times$ with negligible accuracy loss.

I. INTRODUCTION

Industrial anomaly detection must localize defects under strict real-time and resource constraints, but existing systems remain bottlenecked by slow memory-bank construction and redundant inference. LUMIN addresses these constraints through PSP sampling, inference optimizations, and progressive lightweight architecture design.

  • Industrial deployment targets less than 100 ms per image and low resource use, while scarce defect samples increase overfitting risk.
  • Existing memory-bank samplers require repeated backbone passes and iterative distance computations, taking minutes to hours and hindering rapid product switching.
  • Inference pipelines using multi-scale features, cross-attention, or serial similarity computation suffer low GPU utilization, high memory consumption, and poor batch scalability.
  • PSP combines 18-dimensional pixel metadata with five visual plugins in a four-stage pipeline that samples without backbone forward passes.Metadata extraction is performed once offline, while coarse filtering uses sub-second numerical sorting and plugins activate on demand.
  • Parallel similarity computation and stratified pixel sampling reduce inference memory and accelerate large-scale evaluation while preserving stable metrics.Stratified sampling reduces evaluation time from 20.1 s to 1.0 s, a 20× speedup.
  • LUMIN progressively lightweightens UniADet architectures, exploring accuracy-efficiency trade-offs through segmentation-head compression across five benchmark datasets.The study compares three architectures and 11 sampling algorithms on MVTec-AD, VisA, BTAD, KSDD, and Real-IAD.

B. Memory Bank Sampling and Building Strategies

The paper develops lightweight architecture variants and engineering methods for efficient anomaly detection. It combines branch removal, cross-layer feature fusion, and shared segmentation-head parameters to reduce computational and parameter overhead.

  • PatchCore-style coreset sampling is effective but slow because it computes a full patch distance matrix, motivating faster image-level sampling alternatives.
  • UniADet seg removes UniADet’s classification branch, halves parameters from 4LD to 2LD, and reduces training memory, latency, and inference cost.
  • Removing classification makes image-level anomaly scores depend entirely on Top-K pooling of pixel-level anomaly maps, so missed fine-grained defects degrade image-level performance.
  • UniADet seg may still overfit small industrial datasets because its segmentation head retains independent mapping weights at every layer.
  • LUMIN fuses features across backbone layers before detection using cross-layer self-attention, producing a unified representation for its compressed segmentation head.The fused representation is obtained by weighting layer features with the attention matrix A.
  • LUMIN replaces L per-layer segmentation-weight sets with one shared matrix, reducing parameters to 1/L of the original and shrinking model size from approximately 37KB to 11KB when L = 4.

C. Adaptive Memory Bank Sampling Pipeline Based on Metadata and Multiple Plugins (PSP)

PSP replaces iterative deep-feature sampling with a four-stage adaptive pipeline built on cached 18-dimensional pixel metadata and lightweight visual plugins. It performs coarse numerical filtering, dynamically adapts plugin scoring, and selects memory-bank samples using typicality, diversity, and feature richness.

  • Motivation and design: PSP uses lightweight pixel-level metadata and adaptive plugin fusion instead of iterative high-dimensional feature-distance comparisons for memory-bank sampling.The design targets joint accuracy, computation, and storage constraints in industrial deployment.
  • Four-stage pipeline: Sampling proceeds through offline metadata extraction, coarse filtering, adaptive decision, and multi-plugin fusion with fine-grained filtering.The pipeline completes filtering through lightweight CPU/GPU numerical operations without real-time backbone inference.
  • Offline metadata extraction: 18-dimensional metadata captures global attributes and distributional properties, is extracted once per dataset, and is cached for subsequent sampling iterations.For 512px inputs, extraction takes approximately 48 ms per image; MVTec’s 3,629-image cache is approximately 200 KB.
  • Coarse filtering: Coarse filtering retains K_coarse = keep_ratio × K candidates through numerical sorting, with default keep_ratio=5 and execution under 1 s.At keep_ratio=1.0, this stage can independently complete sampling as a minimal deployment scheme.
  • Adaptive decision: Adaptive rules determine five-plugin fusion weights and fine-grained thresholds from candidate distributions, calibrating typicality and diversity across scenes.The resulting scoring strategy supports scene-specific filtering rather than fixed plugin weights.
  • Plugin fusion and selection: Five plugins analyze color, texture, frequency, contrast, and spatial layout before normalized fusion and re-ranking by typicality, diversity, and feature richness.Top-K candidates are selected to build a lightweight memory bank, with default hyperparameters documented for reproduction and tuning.

D. Optimization Strategies for Model Inference and Evaluation

The paper introduces two engineering optimizations that target memory-bank matching and pixel-level evaluation without changing core training or prediction logic. They are designed to reduce deployment overhead while preserving detection accuracy.

  • Optimization overview: Two optimizations target model matching inference and pixel-level metric evaluation while preserving the model’s core training and prediction logic.The stated goals are lower GPU memory overhead and faster experiment iteration without detection-accuracy loss.

1) Parallel Computation Optimization for Memory Bank Retrieval:

Parallel memory-bank retrieval replaces serial cosine-similarity computation with normalized matrix multiplication. This removes the large intermediate tensor, reduces memory complexity, and preserves similarity values and P-AUPR.

  • Bottleneck: The naive per-image cosine-similarity implementation materializes a [C, M, D] tensor, causing high memory use and frequent OOM errors during large-scale inference.The bottleneck becomes especially severe for high-resolution inputs and serial per-image computation.
  • Parallel retrieval: Pre-normalized memory prototypes and normalized query features enable full similarity computation through matrix multiplication.The equivalence between cosine similarity and normalized dot products supports the implementation change.
  • Memory reduction: Memory complexity decreases from O(C · M · D) to O(C · M) by eliminating the 3D intermediate tensor.The optimization is applied during inference after offline prototype normalization and caching.
  • Validation: Experiments report numerically identical similarity values and unchanged P-AUPR after parallelization.GPU parallelism keeps latency nearly flat as batch size increases without additional overhead.

2) Stratified Sampling for Pixel-Level Evaluation:

Stratified pixel sampling reduces the cost of pixel-level evaluation by sampling anomaly and normal pixels separately while preserving their original distribution. The method operates only during evaluation and is intended to accelerate large-scale iteration without changing predictions.

  • Evaluation bottleneck: Full-pixel evaluation on large datasets such as Real-IAD creates heavy floating-point workloads that slow iteration and increase hardware cost.The affected metrics are P-AUROC, P-AUPR, and P-F1max.
  • Evaluation-only design: Stratified sampling is applied exclusively during evaluation and does not alter forward propagation, predictions, or feature distributions.Its sole stated effect is reducing computational overhead.
  • Sample-size rule: The effective sample size is Nsamp = max(⌈N · ratio⌉, 100000), ensuring a minimum sample count for large pixel sets.When N ≤ Nsamp, the full pixel set is evaluated directly.
  • Stratified allocation: Anomaly and normal pixels are sampled separately and merged for metric computation, preserving the original positive/negative distribution.This design suppresses statistical error from random sampling while reducing evaluation cost.
  • Reported effect: The strategy substantially reduces evaluation cost with essentially lossless accuracy, accelerating iteration and ablation validation.The evaluation protocol reports image- and pixel-level AUROC, AUPR, and F1-max across five industrial benchmarks.

B. Main Results

Across five-benchmark evaluations, LUMIN combines lightweight segmentation with strong pixel-level performance and efficient inference. Its DINOv3 variant matches UniADet pixel-level metrics while reducing memory and maintaining low, batch-stable latency.

  • LUMIN exhibits the most compelling performance across combined zero-shot and few-shot evaluations.
  • 1/8 trainable parameters enables LUMIN (DINOv3) to match UniADet in pixel-level metrics, including 97.5 P-AUROC on VisA.
  • P-AUPR reaches 44.0 at k=4 on KSDD for LUMIN (CLIP), outperforming all competitors by over 2× after zero-shot collapse.
  • LUMIN (DINOv3) achieves the best segmentation visualization, eliminating spurious predictions on defect-free regions across four categories.
  • 75 ms latency at batch size 1 and flat 75→75 ms latency through batch size 8 demonstrate efficient, batch-stable inference for LUMIN (DINOv3).DINOv3 uses 1.14 GB versus CLIP’s 1.60 GB, a 29% memory reduction.

E. Pixel-Level Metric Sampling Acceleration

The paper accelerates pixel-level evaluation and PSP sampling through stratified pixel selection, metadata-based filtering, and controlled plugin use. These strategies substantially reduce computation while preserving stable metrics and competitive sampling accuracy.

  • 20× speedup reduces full MVTec-bottle evaluation from 20.1 s to 1.0 s with small, decreasing metric fluctuations at larger sampling ratios.
  • PSP achieves P-AUPR 57.92, matching GSS MF and SGSS MF while surpassing FPS MF and FPS FF.
  • 14.98 s sampling time at k=20 and N=3,629 requires zero backbone forward passes, using only filtering logic after metadata preparation.
  • P-AUPR varies by less than 0.2% across work_size values from 16 to 512, with work_size 64 offering the best accuracy-efficiency trade-off.
  • At k=4, coarse filtering alone reaches the best P-AUPR of 58.24, whereas at k=20 plugins slightly improve P-AUPR to 58.75.
  • Color and texture capture most effective plugin information, reducing Layer 3 time by 30% from 92.02 s to 64.82 s.

D. Effect of Coarse-Filter Granularity on Each Category (k=20)

At k=20, increasing the coarse-filter ratio generally helps modestly, but plugin benefits depend on candidate-pool size and category. The broader lightweighting comparison also shows that LUMIN preserves DINOv3 performance better than CLIP, whose inter-layer heterogeneity limits single-head fusion.

  • Coarse-filter granularity: Mean ∆ rises by +0.109 from ratio=1 to 5, with 13/15 categories improving, although screw declines by −0.73.The decline is attributed to ContrastPlugin misjudging metallic reflections.
  • Coarse-filter granularity: Metadata ranking already nears convergence at ratio=1, while plugins help weakly at k=20 but hurt at k=4 by adding noise.The reported mean plugin effect is +0.11 at k=20 and −0.155 at k=4.
  • Lightweighting paths: Removing UniADet’s classification branch halves trainable parameters and reduces inference latency by 10% while preserving pixel-level performance on three benchmarks.
  • Lightweighting paths: LUMIN’s single-head fusion works better with DINOv3 than CLIP because DINOv3 has more consistent inter-layer features.Increasing sampled layers improves DINOv3 P-F1max from 38.5 to 39.0, but CLIP P-AUPR changes from 4.4 to 4.0.
  • Lightweighting paths: Local feature enhancement is proposed as future work to preserve independent layer processing, but its storage overhead grows linearly with layer count.

B. Analysis of PSP’s Mechanism Advantages, Synergistic Characteristics and Applicability Boundaries

PSP combines lightweight metadata ranking with optional plugin refinement to reduce memory-bank construction cost and support progressive deployment. Its coarse mode is effective for standardized settings, while richer plugins target more complex or shifted scenarios within stated applicability boundaries.

  • Mechanism advantages: PSP uses high-accuracy adaptive sampling at minimal computational cost, with hierarchical stages that allow coarse or full-pipeline deployment.
  • Synergistic characteristics: PSP’s image-level selection complements PatchCore’s patch-level coreset, jointly addressing image coverage, feature storage, and inference latency.
  • Mechanism advantages: All four PSP stages avoid backbone forward passes, while metadata extraction requires one pass at 48 ms per image.
  • Applicability boundaries: At ratio=1, coarse filtering alone reaches P-AUPR 58.24 versus 58.09 for full PSP, with each rebuild requiring only sub-second sorting.
  • Mechanism advantages: PSP supports customizable plugins with adaptive or manual weights, selective activation, and additions through base_plugin.py.
  • Applicability boundaries: Progressive deployment begins with broad cold-start coverage and later tightens filtering; unchanged Top-K rankings avoid memory-bank rebuilds.
  • Applicability boundaries: PSP is best adapted to standardized scenes, while uncontrolled noise, clutter, and natural backgrounds can degrade metadata-based accuracy.The full pipeline is recommended for chaotic scenes.
  • Overall findings: Across five benchmarks, PSP matches K-Means/GSS sampling accuracy at a cost close to random sampling, while LUMIN compresses DINOv3 segmentation with minimal degradation.

APPENDIX A HORIZONTAL COMPARISON OF SAMPLING ALGORITHMS

The sampling comparison identifies backbone forward passes as the dominant efficiency bottleneck. PSP addresses this by filtering with precomputed lightweight metadata, avoiding the storage-computation redundancies of non-streaming methods and the inference overhead of streaming variants.

  • Experimental setup: Table XIV reports P-AUPR and memory-bank build time across k=0, 1, 4, 10, and 20, with metadata extracted once across experiments.
  • Experimental setup: The comparison includes theoretical complexity and characteristic properties for the evaluated sampling algorithms.
  • Efficiency comparison: Backbone forward passes are the key efficiency bottleneck in conventional sampling methods.
  • Efficiency comparison: Non-streaming samplers incur full-set feature-extraction storage and computation, whereas streaming samplers reduce memory but multiply inference overhead.
  • PSP comparison: PSP uses precomputed lightweight metadata for filtering and completes sampling without backbone passes.
Loading 2609.04775v1…