PaperlayerView exact revision on arXiv

Source-linked AI summary

CORE: Contrast-and-Reflect for Self-Supervised Prompt Optimization in Anomaly Synthesis

Haoxiang Rao, Fang Zhao, Zhao Wang, Weijia Li, Chenyang Si, Caifeng Shan

arXiv:2610.19610v1cs.CV

TL;DR

Industrial anomaly synthesis is constrained by scarce real defects and limitations in existing zero-shot and one-shot methods. CORE uses training-free, self-supervised prompt optimization with LMMs to unify both settings. Experiments on MVTec-AD and VisA report improved generation quality and downstream anomaly-detection performance.

  • Problem

    Anomaly scarcity and the limitations of existing zero-shot and one-shot methods leave a need for realistic synthesis that supports downstream anomaly detection.

    Source: Introduction · p. 1

  • Method

    CORE grounds sparse context into structured prompts and iteratively refines them through self-supervised contrast-and-reflect optimization using LMMs.

    Source: Introduction · p. 1, Conclusion · p. 7

  • Results

    CORE improves generation quality and downstream anomaly-detection performance on MVTec-AD and VisA, and achieves state-of-the-art anomaly-detection performance in both zero-shot and one-shot settings.

    Source: Conclusion · p. 7, Anomaly Detection Results · p. 7

  • Takeaways & Limitations

    CORE's model-agnostic, plug-and-play design supports integration with diverse LMM backends and generalization to unseen open-world scenarios.

    Source: Introduction · p. 2, Conclusion · p. 7

Abstract

from arXiv · show

Industrial anomaly detection (AD) is bottlenecked by the scarcity of anomalous samples, making high-fidelity anomaly synthesis crucial for improving downstream performance. However, existing methods either rely on scarce real anomaly references or auxiliary data, while lacking complex reasoning and iterative optimization capabilities for generating realistic defects in open-set scenarios. To address these challenges, we propose CORE, a cost-efficient and training-free prompt optimization framework based on Large Multimodal Models (LMMs), which unifies zero-shot and one-shot anomaly synthesis. CORE first introduces an Anomaly Context Grounding module to transform sparse user context into defect-aware generation context, which is subsequently instantiated as structured anomaly prompts that disentangle normal-region preservation from anomaly-specific variation, enabling controllable and faithful defect generation. Furthermore, CORE autonomously refines prompts through a self-supervised contrast-and-reflect mechanism. This design empowers general-purpose LMMs to synthesize realistic anomalies under both zero-shot and one-shot settings without task-specific training. Extensive experiments on MVTec-AD and VisA demonstrate that CORE synthesizes realistic and diverse anomalies, yielding superior improvements in downstream AD performance. Furthermore, its model-agnostic and plug-and-play paradigm enables effortless integration with diverse LMMs, facilitating adaptation to unseen open-set scenarios.

Introduction

Industrial anomaly detection faces abundant normal samples but scarce anomalies, while existing synthesis approaches retain dependencies that constrain realism and open-set use. CORE proposes a training-free framework that unifies zero-shot and one-shot synthesis through structured prompts and self-supervised refinement.

  • Scarce anomalous samples limit modeling real defect patterns, motivating realistic and diverse synthesis to enrich defect distributions for supervised anomaly detection.Source: Introduction · p. 1
  • Zero-shot synthesis can depend on auxiliary data and retrieval quality, adding computational and preparation overhead that hinders open-set adaptation.Source: Introduction · p. 1
  • One-shot methods use a real anomaly reference but can depend on masks, spatially plausible placement, and struggle to represent logical or abstract defects.Source: Introduction · p. 1
  • CORE grounds available visual and semantic context into defect-aware generation context, then structures prompts to preserve normal content while specifying anomaly variations.Source: Introduction · pp. 1–2
  • CORE uses generated results and limited references as self-supervised feedback for prompt refinement, and is presented as cost-efficient, training-free, and unified across both settings.Source: Introduction · p. 2

Related Work

Prior anomaly-generation work spans few-shot and zero-shot methods, but faces overfitting, spatial-plausibility, or anomaly-guidance limitations. General-purpose LMMs offer open-world capabilities, yet direct use can produce inaccurate defect semantics or alter normal regions.

Anomaly Generation

  • Few-shot generation progressed from GANs to diffusion models, including learned anomaly embeddings and backbone finetuning, but extreme data scarcity makes training-based methods prone to overfitting.Source: Anomaly Generation · p. 2
  • Training-free methods such as TF-IDG and O2MAG remain sensitive to anomaly-mask plausibility and provide limited support for logical anomalies.Source: Anomaly Generation · p. 2
  • Zero-shot approaches include cut-and-paste, attention-based guidance, auxiliary retrieval, and test-time stylization, yet limited anomaly-specific guidance remains a challenge.Source: Anomaly Generation · pp. 2–3

Large Multimodal Models (LMMs)

  • LMMs have open-world visual understanding, reasoning, and generation capabilities, but broad training data lack domain-specific defect priors and can lead to inaccurate semantics or unintended normal-region changes.Source: Large Multimodal Models (LMMs) · p. 3

Method

CORE formulates anomaly synthesis as constrained image editing and autonomously optimizes prompts through self-supervised feedback. It grounds sparse context into structured prompts, then refines those prompts using generated outputs or a single anomaly reference.

Framework Overview

  • CORE’s Execute-Evaluate-Optimize loop generates anomaly image-mask pairs, assesses them, and feeds diagnostic feedback into prompt updates for iterative refinement.The loop operates with black-box LMMs without manual intervention.Source: Framework Overview · p. 3

Structured Prompt Formulation

  • Anomaly Context Grounding derives anomaly design dimensions and generation constraints from normal images, optionally enriched by one anomaly reference.The dimensions define admissible variations; constraints enforce semantic, structural, and material plausibility while preserving normal regions.Source: Structured Prompt Formulation · p. 3
  • CORE structures prompts as BASE, VARIANT, and NEGATIVE components to preserve normal content, specify anomaly attributes, and suppress artifacts.Component-aligned feedback supports targeted, interpretable refinement.Source: Structured Prompt Formulation · p. 3

Self-Supervised Prompt Optimization

  • In zero-shot synthesis, CORE derives self-supervised feedback by comparing generated candidates and separately evaluating normal-region preservation and anomaly quality.Preservation checks identify BASE-related failures such as background changes or object-identity distortion.Source: Self-Supervised Prompt Optimization · p. 4
  • In one-shot synthesis, a real anomaly reference provides weak visual guidance, and Output vs. Ground Truth evaluation scores preservation, prompt alignment, and anomaly quality.Feedback determines whether BASE, VARIANT, or both are refined.Source: Self-Supervised Prompt Optimization · p. 5

Experiments

Across MVTec-AD and VisA, CORE is evaluated for zero- and one-shot synthesis and reports gains in downstream detection, generation quality, and transfer across LMM backends.

Experiment Settings

  • Experiments use MVTec-AD and VisA under zero-shot synthesis and one-shot synthesis with one reference per defect type, excluded from downstream evaluation.Training-based few-shot methods are excluded because the stricter one-shot protocol makes their training unstable and prone to overfitting.Source: Experiment Settings · p. 5

Anomaly Generation Results

  • CORE achieves the best performance across all one-shot generation metrics, while Anomagic has the highest class-wise IS on VisA in zero-shot evaluation.Because IS uses full-image statistics, salient changes at implausible locations may raise its score without improving localized defect fidelity.Source: Anomaly Generation Results · p. 6, Table 2 · p. 6

Anomaly Detection Results

  • CORE achieves state-of-the-art anomaly detection on MVTec-AD and VisA under both zero-shot and one-shot settings.Results are averaged over categories within each benchmark; Table 1 compares methods across the two settings.Source: Anomaly Detection Results · pp. 6–7, Table 1 · p. 6

Ablation Study

  • Anomaly Context Grounding and prompt optimization improve downstream detection and localization, while GPT backends also yield improvements under both settings.The backend comparison keeps the shared ACG initialization fixed.Source: Ablation Study · p. 7, Table 3 · p. 7

Conclusion

CORE unifies zero-shot and one-shot industrial anomaly synthesis through grounded, structured prompts and self-supervised prompt optimization. Experiments on MVTec-AD and VisA show improved generation quality and downstream anomaly detection, with transfer across LMM backends and to unseen objects.

  • CORE is a cost-efficient, training-free prompt optimization framework that unifies zero-shot and one-shot industrial anomaly synthesis.Source: Conclusion · p. 7
  • Anomaly Context Grounding converts sparse user context into defect-aware context and structured prompts that separate normal-content preservation from anomaly-specific variation.Source: Conclusion · p. 7
  • A self-supervised contrast-and-reflect mechanism optimizes prompts without task-specific training or manual prompt tuning.Source: Conclusion · p. 7
  • Experiments on MVTec-AD and VisA show improved generation quality and downstream anomaly detection, with transfer across LMM backends and generalization to unseen objects.Source: Conclusion · p. 7

Related papers

Loading 2610.19610v1…