PaperlayerView exact revision on arXiv

Source-linked AI summary

MambaAD: Exploring State Space Models for Multi-class Unsupervised Anomaly Detection

Haoyang He, Yuhu Bai, Jiangning Zhang, Qingdong He, Hongxu Chen, Zhenye Gan, Chengjie Wang, Xiangtai Li, Guanzhong Tian, Lei Xie

arXiv:2404.06564v4cs.CV

TL;DR

Multi-class unsupervised anomaly detection seeks better accuracy and efficiency than separate per-class models while addressing CNNs’ limited long-range modeling and transformers’ quadratic complexity. MambaAD uses a pre-trained encoder and multi-scale Mamba decoder with LSS modules that combine HSS-based global modeling, multi-kernel local modeling, and hybrid scanning. Experiments report state-of-the-art performance across six datasets and seven metrics, although the authors acknowledge that the model is not yet efficient enough and needs further lightweight design.

  • Problem

    Multi-class anomaly detection still needs improved accuracy and efficiency, while CNNs lack long-range modeling and transformers incur quadratic computational complexity.

  • Method

    MambaAD uses a pre-trained encoder and Mamba decoder with multi-scale LSS modules combining HSS blocks for global information and multi-kernel convolutions for local information.

  • Results

    MambaAD achieves state-of-the-art results on six diverse anomaly detection datasets and seven evaluation metrics.

  • Takeaways & Limitations

    MambaAD demonstrates that Mamba-based global modeling can be combined with CNN-based local feature correlation for multi-class anomaly detection.

  • Takeaways & Limitations

    The authors state that MambaAD is not efficient enough and that more lightweight models need to be designed.

Abstract

from arXiv · show

Recent advancements in anomaly detection have seen the efficacy of CNN- and transformer-based approaches. However, CNNs struggle with long-range dependencies, while transformers are burdened by quadratic computational complexity. Mamba-based models, with their superior long-range modeling and linear efficiency, have garnered substantial attention. This study pioneers the application of Mamba to multi-class unsupervised anomaly detection, presenting MambaAD, which consists of a pre-trained encoder and a Mamba decoder featuring (Locality-Enhanced State Space) LSS modules at multi-scales. The proposed LSS module, integrating parallel cascaded (Hybrid State Space) HSS blocks and multi-kernel convolutions operations, effectively captures both long-range and local information. The HSS block, utilizing (Hybrid Scanning) HS encoders, encodes feature maps into five scanning methods and eight directions, thereby strengthening global connections through the (State Space Model) SSM. The use of Hilbert scanning and eight directions significantly improves feature sequence modeling. Comprehensive experiments on six diverse anomaly detection datasets and seven metrics demonstrate state-of-the-art performance, substantiating the method's effectiveness. The code and models are available at https://lewandofskee.github.io/projects/MambaAD.

1 Introduction

Industrial anomaly detection increasingly needs multi-class methods that reduce the training and memory burden of separate per-class models while improving accuracy and efficiency. MambaAD applies Mamba to this setting, combining global and local modeling through a multi-scale decoder with LSS modules.

  • Single-class anomaly detection requires separate models for each class, increasing training and memory usage.
  • CNN-based RD4AD captures local context but lacks long-range dependencies, whereas Transformer-based UniAD provides global modeling with quadratic computational complexity.
  • MambaAD applies Mamba to multi-class anomaly detection, using linear complexity to compute anomaly maps across multiple scales with fewer parameters and lower computational demand.
  • The LSS module combines cascaded Mamba-based HSS blocks for global information with parallel multi-kernel convolutions for local connections.
  • The HSS block uses five scanning methods and eight multi-directional scans to strengthen global modeling across anomaly images with varied categories and morphologies.
  • MambaAD achieves state-of-the-art results on six anomaly detection datasets and seven metrics while maintaining low model parameters and computational complexity.

2 Related Work

Related work spans embedding-, synthesizing-, and reconstruction-based anomaly detection, with multi-class methods addressing the cost and scalability limits of per-category training. MambaAD builds on state space modeling to combine linear-complexity global modeling with CNN-like local detail in multi-scale reconstruction.

  • Unsupervised Anomaly Detection: Unsupervised anomaly detection methods are broadly categorized as embedding-based, synthesizing-based, and reconstruction-based approaches.
  • Unsupervised Anomaly Detection: Synthesizing-based methods can produce anomalies that differ from real-world anomalies, while reconstruction-based methods reconstruct images using encoders and decoders.
  • Multi-class Anomaly Detection: Multi-class anomaly detection avoids the increased time and memory consumption of training separate models as the number of categories grows.
  • State Space Models: State space models efficiently represent long-range dependencies, and Mamba extends this line of work with applications in visual and other domains.
  • MambaAD: MambaAD combines state space models’ global modeling capabilities with CNNs’ detailed local modeling for multi-class unsupervised anomaly detection.
  • MambaAD: The proposed framework reconstructs multi-scale features with a pyramidal auto-encoder, while aggregated multi-scale reconstruction error forms the anomaly map.

3 Method

MambaAD combines state-space modeling with convolutional processing in a multi-scale reconstruction framework for multi-class anomaly detection. Its HSS blocks use hybrid scanning, while LSS modules integrate global and local feature processing.

  • State Space Model preliminaries: State Space Models map one-dimensional input sequences to responses through hidden states using linear ordinary differential equations.The state transition matrix A, input matrix B, and output matrix C define the state-space formulation; discretization uses a zero-order hold with timescale parameter ∆.
  • MambaAD framework: MambaAD uses a pre-trained CNN encoder, Half-FPN bottleneck, and Mamba-based decoder to reconstruct features at three scales.The decoder uses depth configuration [4], and training minimizes summed MSE across the three feature-map scales.
  • Locality-Enhanced State Space module: LSS modules combine cascaded HSS blocks for global information with parallel multi-kernel convolutions for local information.Local and global outputs are concatenated, projected with a 1 × 1 convolution, and added to the input through a residual connection.
  • Hybrid State Space block: HSS blocks encode feature maps with five scanning methods and eight directions before SSM processing, then decode features back to the original orientation.The HS encoder combines features across scanning configurations, while the HS decoder restores orientation and sums the decoded outputs.
  • Hybrid scanning: The five evaluated scanning methods are Sweep, Scan, Z-order, Zigzag, and Hilbert, with Hilbert selected for feature-sequence modeling.The paper states that Hilbert scanning better encodes local and global information and helps address long-range-dependency modeling challenges.

4 Experiments

Experiments evaluate MambaAD across diverse multi-class anomaly-detection datasets, metrics, comparisons, qualitative results, ablations, scanning strategies, and efficiency. Results show strong accuracy, scalability, and parameter-efficiency, while analyses support combining global Mamba modeling with local convolutions.

  • Datasets: MVTec-AD contains 5 texture types, 10 object types, and 5,354 high-resolution images, with 3,629 normal training images and 1,725 test images.
  • Datasets: VisA contains 12 objects and 10,821 images, including 9,621 normal and 1,200 anomalous samples.
  • Datasets: Real-IAD contains 30 object categories and 150K high-resolution images, including 99,721 normal and 51,329 anomalous images.
  • Evaluation: MambaAD reports AU-ROC, AP, F1_max, AU-PRO, and their seven-metric mean mAD for anomaly detection and segmentation.
  • Quantitative results: On MVTec-AD, MambaAD reaches 98.6/99.6/97.6 at image level and 97.7/56.3/59.2/93.1 at pixel level, improving mAD by 2.0 over DiAD.
  • Quantitative results: On VisA, MambaAD exceeds DiAD by 7.5/6.2/4.3 at image level and 2.5/13.3/11.0/15.8 at pixel level, with an 8.7 mAD improvement.
  • Ablation analysis: Incremental ablations raise MVTec-AD mAD from 82.1 with basic Mamba scanning to 86.0 using LSS and HSS, while VisA reaches 78.9.
  • Ablation analysis: The proposed LSS module combines Mamba-based HSS global modeling with parallel CNN branches, achieving the best performance and a 1.6% improvement over individual branches.

5 Conclusion

MambaAD applies Mamba to multi-class anomaly detection through a pre-trained encoder, multi-scale decoder, and LSS modules that combine global and local modeling. Experiments span six datasets and seven metrics, while the authors acknowledge that the model is not yet efficient enough.

  • 5 Conclusion: MambaAD is presented as the first application of Mamba to anomaly detection, using a pre-trained encoder and multi-scale Mamba decoder.The decoder uses LSS modules at different scales and depths.
  • 5 Conclusion: LSS modules combine sequential HSS modules with parallel multi-kernel convolutions to model global feature correlations and local information associations.HS encoders use five scanning patterns and eight directions for feature-sequence modeling.
  • 5 Conclusion: Extensive experiments on six diverse anomaly detection datasets and seven evaluation metrics demonstrate SoTA performance.The paper reports these experiments as evidence of MambaAD's effectiveness.
  • 5 Conclusion: The authors state that MambaAD is not efficient enough and that more lightweight models need to be designed.They describe this work as an initial attempt and foundation for future research.

A More Quantitative Results for Each Category on The MVTec-AD Dataset.

The supplementary MVTec-AD results report image-level detection and pixel-level localization outcomes across all categories, comparing MambaAD with various state-of-the-art methods.

  • A More Quantitative Results for Each Category on The MVTec-AD Dataset.: The supplementary results further demonstrate MambaAD's superiority over various SoTA approaches on MVTec-AD.This claim covers both image-level detection and pixel-level localization comparisons.
  • A More Quantitative Results for Each Category on The MVTec-AD Dataset.: Table A1 compares multi-class MVTec-AD anomaly detection across categories using AU-ROC, AP, and F1_max metrics.The table reports image-level anomaly detection outcomes.
  • A More Quantitative Results for Each Category on The MVTec-AD Dataset.: Table A2 compares multi-class MVTec-AD anomaly localization across categories using AU-ROC, AP, F1_max, and AU-PRO metrics.The table reports pixel-level anomaly localization outcomes.

B More Quantitative Results for Each Category on The VisA Dataset.

The supplementary VisA results report image-level detection and pixel-level localization outcomes across all categories, comparing MambaAD with various state-of-the-art methods.

  • B More Quantitative Results for Each Category on The VisA Dataset.: The supplementary results further demonstrate MambaAD's superiority over various SoTA approaches on VisA.This claim covers both image-level detection and pixel-level localization comparisons.
  • B More Quantitative Results for Each Category on The VisA Dataset.: Table A3 compares multi-class VisA anomaly detection across categories using AU-ROC, AP, and F1_max metrics.The table reports image-level anomaly detection outcomes.
  • B More Quantitative Results for Each Category on The VisA Dataset.: Table A4 compares multi-class VisA anomaly localization across categories using AU-ROC, AP, F1_max, and AU-PRO metrics.The table reports pixel-level anomaly localization outcomes.

C More Quantitative Results for Each Category on The MVTec-3D Dataset.

The supplementary MVTec-3D results report image-level detection and pixel-level localization outcomes across all categories, comparing MambaAD with various state-of-the-art methods.

  • C More Quantitative Results for Each Category on The MVTec-3D Dataset.: The supplementary results further demonstrate MambaAD's superiority over various SoTA approaches on MVTec-3D.This claim covers both image-level detection and pixel-level anomaly localization comparisons.
  • C More Quantitative Results for Each Category on The MVTec-3D Dataset.: Table A5 compares multi-class MVTec-3D anomaly detection across categories using AU-ROC, AP, and F1_max metrics.The table reports image-level anomaly detection outcomes.
  • C More Quantitative Results for Each Category on The MVTec-3D Dataset.: Table A6 compares multi-class MVTec-3D anomaly localization across categories using AU-ROC, AP, F1_max, and AU-PRO metrics.The table reports pixel-level anomaly localization outcomes.

D More Quantitative Results for Each Category on The Uni-Medical Dataset.

The Uni-Medical results cover both image-level anomaly detection and pixel-level anomaly localization across all categories. The reported comparisons show MambaAD outperforming various state-of-the-art approaches.

  • Uni-Medical results span image-level anomaly detection and pixel-level anomaly localization across all categories.Tables A7 and A8 report quantitative outcomes for the two tasks.
  • Table A7 compares multi-class anomaly detection using AU-ROC, AP, and F1_max metrics.
  • Table A8 compares multi-class anomaly localization using AU-ROC, AP, F1_max, and AU-PRO metrics.

E More Quantitative Results for Each Category on The COCO-AD Dataset.

The COCO-AD results cover image-level anomaly detection and pixel-level anomaly localization across all categories. The reported comparisons show MambaAD outperforming various state-of-the-art approaches.

  • COCO-AD results span image-level anomaly detection and pixel-level anomaly localization across all categories.Tables A9 and A10 report quantitative outcomes for the two tasks.
  • Table A9 compares multi-class anomaly detection using AU-ROC, AP, and F1_max metrics.
  • Table A10 compares multi-class anomaly localization using AU-ROC, AP, F1_max, and AU-PRO metrics.

F More Quantitative Results for Each Category on The Real-IAD Dataset.

The supplied results cover multi-class evaluation on Real-IAD and single-class evaluation on MVTec-AD and VisA, reporting detection and localization outcomes.

  • Real-IAD: Real-IAD results report image-level anomaly detection and pixel-level anomaly localization across all categories.Tables A11 and A12 provide the corresponding quantitative outcomes and state superiority over various state-of-the-art approaches.
  • Real-IAD: Table A11 compares Real-IAD multi-class anomaly detection using AU-ROC, AP, and F1_max metrics.
  • MVTec-AD: MVTec-AD results cover single-class anomaly detection and single-class anomaly localization.Tables A13 and A14 report quantitative outcomes for both tasks.
  • Real-IAD: Table A12 compares Real-IAD multi-class anomaly localization using AU-ROC, AP, F1_max, and AU-PRO metrics.
  • MVTec-AD: Table A13 compares MVTec-AD single-class anomaly detection using AU-ROC, AP, and F1_max metrics.
  • MVTec-AD: Table A14 compares MVTec-AD single-class anomaly localization using AU-ROC, AP, F1_max, and AU-PRO metrics.
  • VisA: VisA results cover single-class anomaly detection and single-class anomaly localization.Tables A15 and A16 report quantitative outcomes for both tasks.
  • VisA: Table A15 compares VisA single-class anomaly detection using AU-ROC, AP, and F1_max metrics.
Loading 2404.06564v4…