Source-linked AI summary
GRC-Net: Global Representation Consistency Network for Unsupervised Multimodal Anomaly Detection
Seyoung Jeong, Jong Pil Yun, Sang Jun Lee
TL;DR
RGB-based inspection has limited ability to detect geometric anomalies, motivating multimodal methods that incorporate 3D information. GRC-Net uses global representation consistency and stable reconstruction, and outperforms existing methods on MVTec 3D-AD and Eyecandies at image and pixel levels.
Problem
RGB-based methods rely on surface color and texture, limiting detection of geometric anomalies, while environmental variations can cause false detections.
Source: §1 · p. 1Method
GRC-Net combines a global-token GA-MLP that enforces consistency across patch embeddings with an SRM that stabilizes reconstruction.
Source: §1 · p. 1, §2.2 · p. 2Results
GRC-Net outperforms existing methods on MVTec 3D-AD and Eyecandies at image and pixel levels, with particularly strong image-level AUROC.
Source: §1 · p. 1, §3.1 · pp. 3–4, Conclusion · p. 4Takeaways & Limitations
Qualitative results report reduced reconstruction noise in normal regions and clearer distinction and localization of anomalous regions.
Source: §3.1 · p. 4, Conclusion · p. 4
Abstract
from arXiv · showhide
Automated quality inspection is essential for ensuring product reliability in manufacturing.While image-based methods effectively capture appearance-related defects, these methods are limited in detecting structural and geometric anomalies, motivating multimodal approaches incorporating 3D information. However, existing methods mainly rely on local patch-level representations, which often lead to unstable reconstruction errors even in normal regions. To address this limitation, we propose GRC-Net, which integrates a global-attention MLP to enforce global representation consistency across patch embeddings with a stable reconstruction module to improve reconstruction stability. The proposed method captures holistic contextual information through a global token and suppresses reconstruction noise by minimizing discrepancies between original and predicted embeddings. Experiments on MVTec 3D-AD and Eyecandies demonstrate that GRC-Net consistently outperforms existing methods at both image and pixel levels. Qualitative results further demonstrate reduced reconstruction errors in normal regions and more distinct reconstruction differences between normal and anomalous regions.
1. INTRODUCTION
RGB-based inspection is limited for geometric anomalies, motivating multimodal methods that incorporate 3D information. GRC-Net combines global representation consistency with stable reconstruction and reports strong results on two benchmarks.
1. INTRODUCTION
- RGB methods capture surface color and texture but are limited for geometric anomalies, while lighting, shadows, and reflections can cause false detections.Multimodal approaches incorporating 3D information have gained attention for detecting geometric anomalies and improving robustness to environmental variations.Source: §1 · p. 1
- Prior multimodal methods face memory and feature-alignment issues or difficulty reconstructing manufacturing variations such as curvature and material diversity.M3DM uses modality-specific memory banks, while CFM learns cross-modal feature reconstruction from normal data.Source: §1 · p. 1
- GRC-Net outperforms existing multimodal methods on MVTec 3D-AD and Eyecandies, particularly in image-level AUROC.The introduction reports strong performance on both datasets.Source: §1 · p. 1
- GRC-Net uses RGB images and 3D point clouds, combining a global attention MLP with a stable reconstruction module.The architecture uses the global attention MLP to capture global contextual relationships and the stable reconstruction module to suppress unstable reconstruction errors.Source: §1 · p. 1, Figure 1 · p. 2
- The GA-MLP generates a global token from normal-data feature embeddings to enforce global representation consistency and predict counterpart-modality features.Source: §1 · p. 1
- The SRM suppresses noise in normal-data anomaly maps and enhances the reconstruction gap between normal and anomalous regions.Source: §1 · p. 1
2. METHODOLOGY
GRC-Net uses globally informed RGB and 3D feature embeddings for cross-modal reconstruction, then combines modality-specific reconstruction-error maps through pixel-wise multiplication to produce an anomaly map.
2.1. Multimodal feature extraction and alignment
- Pretrained RGB and point-cloud encoders produce features that are aligned pixel-wise using known 3D-point correspondences.The aligned RGB and point-cloud feature vectors are used as network inputs.Source: §2.1 · p. 2
2.2. Global attention MLP
- GA-MLP pools mean and max patch features into a global token, aligns patches to that token, and projects enhanced embeddings into the other modality.Mean pooling summarizes global context, while max pooling captures local saliency; patch tokens act as queries and the global token supplies keys and values.Source: §2.2 · pp. 2–3
2.3. Stable reconstruciton module
- The SRM jointly optimizes cosine-based cross-modal projection and normalized embedding reconstruction losses to stabilize normal-data reconstruction.The reconstruction loss compensates for magnitude differences not captured by cosine alignment; its weight α is 0.2 in the experiments.Source: §2.3 · p. 3
3. EXPERIMENTS
GRC-Net is evaluated on two multimodal anomaly-detection benchmarks, with comparisons covering overall performance, qualitative localization, and component ablations.
3. EXPERIMENTS
- Evaluation uses pixel-wise aligned RGB images and point clouds from MVTec 3D-AD and Eyecandies, reporting I-AUROC, P-AUROC, and AUPRO.I-AUROC measures image-level detection, P-AUROC pixel-level localization, and AUPRO overlap with anomalous regions at threshold 0.3.Source: §3 · p. 3
3.1. Experimental results
- GRC-Net achieves strong performance on MVTec 3D-AD and Eyecandies, with high image- and pixel-level AUROC.The quantitative comparison covers I-AUROC, P-AUROC, and AUPRO.Source: §3.1 · pp. 3–4, Table 1 · p. 3
- On both datasets, GRC-Net produces cleaner anomaly maps than prior methods, suppressing responses in normal regions and better localizing anomalies.The comparisons describe noisy responses on irregular or curved MVTec surfaces and false responses or blurred boundaries on Eyecandies.Source: §3.1 · p. 4, Figure 2 · p. 4, Figure 3 · p. 4
3.2. Ablation study
- GA-MLP and SRM each improve the baseline, while their combination achieves the best results on both datasets.The ablation attributes global representation consistency to GA-MLP and reconstruction stability and localization improvements to SRM.Source: §3.2 · p. 4, Table 2 · p. 4
Conclusion
GRC-Net combines global representation consistency with stable reconstruction for multimodal anomaly detection. Experiments on two benchmarks report improved detection performance and more accurate localization.
Conclusion
- GRC-Net uses RGB images and point clouds for multimodal anomaly detection.It integrates GA-MLP for global representation consistency and SRM for reconstruction stability.Source: Conclusion · p. 4
- The framework reduces reconstruction noise in normal regions and enlarges the reconstruction gap between normal and anomalous regions.Source: Conclusion · p. 4
- Experiments on MVTec 3D-AD and Eyecandies report improved image- and pixel-level detection performance over existing methods, with more accurate qualitative localization.Source: Conclusion · p. 4