Hierarchical Vector Quantized Transformer for Multi-class Unsupervised Anomaly Detection

• One-for-all detection avoids the time and memory costs of separate class models, but must capture complex multi-class distributions precisely.Source: §1 · p. 1
• Continuous representations can let anomalous inputs reconstruct well, creating the “identical shortcut” that makes them hard to distinguish.Source: §1 · p. 2
• VQ replaces encoded features with normal-category prototypes, making anomalous regions reconstruct as normal and producing large reconstruction errors.Source: §1 · p. 2, Figure 1 · p. 2
• Hierarchical VQ combines fine-grained and abstract features to address codebook collapse, while prototype-oriented optimal transport regulates prototypes and supports hierarchical anomaly scoring.Source: §1 · p. 2
• HVQ-Trans achieves state-of-the-art anomaly detection and localization, with enhanced interpretability through prototype visualization.Source: §1 · p. 3

2.1 Overview

• HVQ-Trans uses a pre-trained EfficientNet, cascaded Transformer encoder and decoder, hierarchical vector quantization, switching experts, and prototype-oriented scoring for anomaly detection and localization.The overview describes the pipeline from visual-token extraction through calibrated anomaly-score-map refinement.Source: §2.1 · p. 3

2.2 Improving Feature Reconstruction with Hierarchical Vector Quantization

• Hierarchical VQ replaces visual tokens with similar normal prototypes and fuses multi-level features with a global quantized vector to preserve nominal information.The decoder cross-attends to prototypes, which the authors say suppresses abnormal-region signals and limits their transmission into reconstruction.Source: §2.2 · p. 4
• A classifier switches among category-specific codebooks and reconstruction experts; uncertainty in this selection is expected to increase for unseen anomalies.The model uses feature-level rather than pixel-level reconstruction, and the authors associate switching uncertainty with multi-class anomaly detection.Source: §2.2 · pp. 4–5

2.3 Prototype-oriented Learning and Scoring for Anomaly Detection

• Prototype-oriented optimal transport aligns codebooks with normal image features during training and contributes hierarchical feature discrepancies to anomaly scoring at test time.The authors identify Euclidean scoring's focus on the most significant score-map difference as neglecting multi-level differences.Source: §2.3 · pp. 5–6

2.4 Overall Optimization

• The objective combines reconstruction, hierarchical prototypical and commitment losses, POT loss, and classifier cross-entropy, with copied gradients approximating the quantization gradient.The commitment loss keeps encoder outputs close to selected prototypes and discourages frequent switching between prototypes.Source: §2.4 · p. 6

3 Connection with previous works

• Reconstruction-based methods assume normal regions reconstruct well and anomalous regions do not, but most lack a reliable mechanism to enforce larger errors on anomalies.The paper distinguishes these from density-based methods, which estimate the distribution of normal data points.Source: §3 · p. 6
• Memory-based methods recombine and reweight recorded normal patterns, but anomalous representations recovered this way may collapse discrimination.MemAE introduced the memory-matrix paradigm for assembling normal patterns during training.Source: §3 · p. 6
• HVQ-Trans uses a discrete latent space inspired by lossy compression, while its hierarchy supports larger codebooks without collapse and multi-level prototype-oriented anomaly scoring.The method avoids assuming a specific normal-data distribution and learns the image prior dynamically.Source: §3 · p. 6

4.2 Anomaly detection and localization performance on MVTec-AD

• HVQ-Trans surpasses PatchCore and UniAD by 1.6% and 1.5% on average detection, and UniAD by 0.5% on both localization settings.The MVTec-AD comparisons are reported under unified evaluation.Source: Table 1 · p. 7, §4.2 · pp. 7–8
• Hierarchical VQ layers generate more accurate MVTec-AD localization maps than a single VQ layer by restoring abnormal regions with corresponding normal patterns.The qualitative comparison covers both object anomalies and texture damage.Source: Figure 4 · p. 7, §4.2 · p. 8

4.1 Datasets and Metrics

• Evaluation uses AUROC for image-level detection and pixel-wise localization; MVTec-AD and VisA provide normal training data and anomalous test examples, while CIFAR-10 uses five normal and five unseen anomaly classes.The MVTec-AD unified setting uses one model across categories without fine-tuning.Source: §4.1 · pp. 6–7

4.3 Anomaly detection on VisA

• On VisA, HVQ-Trans exceeds UniAD by 1.3% image-AUROC and OmniAL by 5.4% on detection and 2.1% on localization.Qualitative results also show anomaly regions restored to normal appearance, including in multi-instance scenes.Source: Table 2 · p. 8, §4.3 · pp. 8–9, Figure 5 · p. 8

4.4 Anomaly detection on CIFAR-10

• HVQ-Trans surpasses every competitor across each CIFAR-10 split in the many-versus-many setting.The study uses five normal classes and treats the remaining five as unseen anomalies during training.Source: Table 3 · p. 9, §4.4 · p. 9

4.5 Ablation studies

• Removing VQ drops performance from 96.4 to 70.5; hierarchical VQ reduces search complexity and codebook collapse, while POT supports detection and localization.Switching brings slight gains on MVTec-AD but gains up to 3.2% on CIFAR-10.Source: §4.5 · p. 9
• Concatenating multi-level features with global prototypes improves joint detection and localization; higher-abstraction global prototypes may provide efficient latent representations for detection.The hierarchy comparison is reported in Table 5.Source: Table 5 · p. 9, §4.5 · p. 10
• Performance remains above most competitors across prototype counts, and reconstructions using irrelevant-category prototypes follow the supplied prototypes.Prototype visualizations show a Cable region reconstructed as Grid when instructed to use Grid prototypes.Source: Table 6 · p. 9, §4.5 · p. 10, Figure 6 · p. 10

5 Conclusion

• Under the one-for-all setting, HVQ-Trans significantly surpasses competitors on MVTec-AD and VisA and provides visualizations for anomaly localization and detection.The model uses hierarchical discrete prototypes learned from normal training data and a hierarchical prototype-oriented optimal transport module to regulate prototypes and calibrate anomaly scores.Source: §5 · p. 10
• HVQ-Trans is a unified model for multi-class unsupervised anomaly detection under the one-for-all setting.Its latent space is modeled as hierarchical discrete prototypes learned from normal training data.Source: §5 · p. 10
• Training assumes category labels are available, while incorporating clustering methods instead of labels remains future work.The discussion also notes that anomaly detection in video surveillance or social multimedia may raise privacy concerns.Source: §5 · p. 10