PaperlayerView exact revision on arXiv

Source-linked AI summary

Anomaly Detection in 3D Point Clouds using Deep Geometric Descriptors

Paul Bergmann, David Sattlegger

arXiv:2202.11660v1cs.CV

TL;DR

Unsupervised detection and localization of geometric anomalies in 3D point clouds remains comparatively unexplored, particularly for high-resolution data. The paper adapts student–teacher detection to point clouds and self-supervises dense local geometric descriptors. On MVTec 3D-AD, it outperforms the previously leading method by 25.0 points.

  • Problem

    Unsupervised 3D anomaly detection remains comparatively unexplored, and existing methods perform poorly on challenging high-resolution point clouds.

    Source: §1 · p. 1, §2.2 · p. 4

  • Method

    3D-ST trains a student on anomaly-free point clouds to match a pretrained teacher’s dense local geometric descriptors, with self-supervised teacher pretraining through local-geometry reconstruction.

    Source: §1 · pp. 2–3, §3 · p. 5

  • Results

    Our method performs significantly better than all other methods on every MVTec 3D-AD category; the d=128 variant outperforms the previously leading method by 25.0 points.

    Source: §4.2 · p. 10

  • Takeaways & Limitations

    The method outperforms other evaluated methods across PRO-curve integration limits, with particularly large relative differences at lower limits associated with low false-positive rates.

    Source: §4.2 · p. 10

Abstract

from arXiv · show

We present a new method for the unsupervised detection of geometric anomalies in high-resolution 3D point clouds. In particular, we propose an adaptation of the established student-teacher anomaly detection framework to three dimensions. A student network is trained to match the output of a pretrained teacher network on anomaly-free point clouds. When applied to test data, regression errors between the teacher and the student allow reliable localization of anomalous structures. To construct an expressive teacher network that extracts dense local geometric descriptors, we introduce a novel self-supervised pretraining strategy. The teacher is trained by reconstructing local receptive fields and does not require annotations. Extensive experiments on the comprehensive MVTec 3D Anomaly Detection dataset highlight the effectiveness of our approach, which outperforms the next-best method by a large margin. Ablation studies show that our approach meets the requirements of practical applications regarding performance, runtime, and memory consumption.

1. Introduction

Unsupervised anomaly detection in 3D point clouds remains comparatively unexplored, with no established pretraining protocol for the task. The paper adapts student–teacher detection to 3D and introduces self-supervised pretraining for dense local geometric descriptors.

1. Introduction

  • 3D anomaly detection is comparatively unexplored, and existing local-feature methods are ill-suited to high-resolution anomaly localization.The paper identifies the lack of an established 3D pretraining protocol as a gap.Source: §1 · pp. 1–2
  • The work draws on pretrained-descriptor approaches to anomaly detection that have been successful in 2D.The cited 2D approaches outperform methods based on convolutional autoencoders or generative adversarial networks.Source: §1 · p. 2
  • 3D-ST trains a student on anomaly-free point clouds to match pretrained teacher descriptors, then uses regression errors to localize anomalies in high-resolution test samples.The method operates directly on point clouds and performs localization with a single forward pass.Source: §1 · p. 2
  • The teacher’s self-supervised protocol reconstructs local geometry encoded by dense descriptors, enabling anomaly scoring on many input points without intermediate subsampling.It provides explicit control over the receptive field and does not require annotations.Source: §1 · p. 3

2. Related Work

The related work spans 2D anomaly detection, existing 3D anomaly methods, and learned local 3D descriptors. The paper identifies limited 3D anomaly-detection methods and descriptor designs that do not meet its needs for dense, rotation-sensitive features.

2.1 Anomaly Detection in 2D

  • Pretrained-network descriptors are used in 2D anomaly detection because anomalous regions produce descriptors that differ from normal regions; these methods tend to outperform training from scratch.Student–teacher variants train students on anomaly-free images to reproduce pretrained teacher descriptors and detect anomalies through increased regression errors and predictive variance.Source: §2.1 · p. 3

2.2 Anomaly Detection in 3D

  • Only a few methods address unsupervised 3D anomaly detection, and none leverage pretrained feature vectors; MVTec 3D-AD reports poor performance on challenging high-resolution point clouds.Voxel f-AnoGAN and convolutional autoencoder approaches compare input voxels with reconstructions to derive anomaly scores.Source: §2.2 · pp. 3–4

2.3 Learning Deep 3D Descriptors

  • Patch-based local descriptor methods require separate patch processing for each feature, which becomes computationally intractable for many points.The cited approaches include supervised metric learning and unsupervised autoencoder strategies.Source: §2.3 · p. 4
  • Dense descriptor methods use sparse convolutions or self-supervised correspondences, but voxelization can introduce discretization inaccuracies and rotation invariance can suppress anomaly-relevant local rotations.The paper argues for pretraining that is sensitive to local rotations.Source: §2.3 · p. 4

3. Student-Teacher Anomaly Detection in Point Clouds

3D-ST adapts student-teacher anomaly detection to high-resolution 3D point clouds, using a teacher pretrained to describe local geometry and a student trained on anomaly-free data. It localizes anomalies through teacher-student regression errors.

3. Student-Teacher Anomaly Detection in Point Clouds

  • At inference, increased regression errors between student and teacher descriptors indicate anomalous points.The student is trained against the pretrained teacher on anomaly-free point clouds.Source: §3 · p. 5, §3.2 · p. 8

3.1 Self-Supervised Learning of Dense Local Geometric Descriptors

  • The teacher produces a dense local geometric descriptor for each point, with features describing geometry within that point’s receptive field.Its architecture processes high-resolution point clouds without downsampling and permits explicit control over descriptor receptive fields.Source: §3.1 · p. 5
  • Translation-invariant local features use neighboring-point differences, and the authors report that this invariance is crucial for anomaly-detection performance.The local aggregation block combines geometric features of neighboring points with their input features.Source: §3.1 · p. 6
  • Self-supervised pretraining trains a decoder to reconstruct each sampled point’s local receptive field from its teacher descriptor.The reconstruction objective uses Chamfer distance; input receptive-field points are centered by subtracting their mean.Source: §3.1 · p. 7

3.2 Matching Geometric Features for 3D Anomaly Detection

  • For anomaly-detection training, the teacher remains fixed while the student learns to reproduce its descriptors using feature-wise L2 distance.Teacher features are centered and scaled to unit standard deviation; at inference, teacher-student regression errors provide pointwise anomaly scores.Source: §3.2 · p. 8

4. Experiments

Experiments on MVTec 3D-AD show that 3D-ST outperforms the evaluated baselines, with ablations examining its hyperparameters, teacher pretraining, and alternative feature extractors.

4.1 Experiment Setup

  • The MVTec 3D-AD benchmark contains over 4,000 high-resolution scans across 10 industrial product categories, and evaluation trains on anomaly-free samples.Baselines include voxel-based and depth-image counterparts evaluated under the same protocols and hyperparameter settings.Source: §4 · p. 8, §4.1 · p. 9
  • Teacher pretraining uses synthetic scenes assembled from ModelNet10 objects, with 1,000 training and 50 validation point clouds.The ModelNet10 objects are randomly selected, rotated, scaled, and placed at random locations to generate scenes.Source: §4.1 · p. 9

4.2 Experiment Results

  • 25.0 points: 3D-ST with d = 128 outperforms the previously leading method; increasing d from 64 to 128 improves overall performance by 1.5 percentage points.Table 1 reports results by category using area under the PRO curve at an integration limit of 0.3.Source: Table 1 · p. 10, §4.2 · p. 10
  • 3D-ST outperforms all evaluated methods across integration limits, with a particularly large relative difference at lower limits important for real-world applications.Figure 5 marks the 0.3 integration limit and shows performance across limits.Source: §4.2 · p. 10, Figure 5 · p. 11

4.3 Ablation Studies

  • Performance is insensitive to input-point count, descriptor dimension, and neighbor count; it eventually saturates while inference time and memory consumption continue increasing superlinearly.Across the tested settings, mean performance rises monotonically with each hyperparameter and outperforms the best baseline by a large margin.Source: §4.3 · p. 11, Figure 6 · p. 12
  • Random teacher initialization, absolute coordinates, and rotation-invariant augmentation each reduce performance relative to the proposed teacher strategy.The authors interpret the coordinate result as evidence that translation invariance matters and the rotation result as evidence that sensitivity to local rotations is beneficial.Source: §4.3 · p. 11, Figure 7 · p. 13
  • Pretraining on ITODD or 3DMatch does not strongly change performance, while KITTI yields a slight gap that the authors attribute likely to domain shift.The comparison uses industrial, indoor-registration, and autonomous-driving datasets, respectively.Source: §4.3 · p. 12, Figure 7 · p. 13
  • Student networks trained to match PPF-FoldNet or FCGF features outperform the Voxel GAN baseline but do not reach the proposed pretraining strategy's performance.The student output dimension is adapted to match the descriptor dimension of each alternative extractor.Source: §4.3 · pp. 12–13

5. Conclusion

The paper introduces 3D-ST for unsupervised geometric anomaly detection in 3D point clouds, using self-supervised teacher pretraining and student-teacher regression errors to localize anomalies. On MVTec 3D Anomaly Detection, it reports outperforming existing methods by a large margin.

5. Conclusion

  • 3D-ST uses a self-supervised teacher that reconstructs local receptive fields to produce dense geometric descriptors, which a student learns to match on anomaly-free samples.At inference, per-point anomaly scores come from regression errors between associated teacher and student descriptors.Source: §5 · p. 13
  • On the MVTec 3D Anomaly Detection dataset, 3D-ST outperforms all existing methods by a large margin.Source: §5 · p. 13
  • Ablations show computational efficiency and robustness to the choice of hyperparameters and pretraining datasets.Source: §5 · p. 13

Appendix A. Additional Information on Ablation Experiments

The appendix provides numerical ablation results corresponding to plots in the main manuscript.

Appendix A. Additional Information on Ablation Experiments

  • Numerical ablation results are provided for the line and bar plots in the main manuscript.Source: Appendix A · p. 19

A.1 Lower Integration Limits

This appendix reports AU-PRO across multiple false-positive-rate integration limits and descriptor dimensions for MVTec 3D-AD categories.

A.1 Lower Integration Limits

  • Table 2 reports AU-PRO for each MVTec 3D-AD category at four false-positive-rate integration limits.The limits are 0.01, 0.05, 0.10, and 0.20; the paper's majority integration limit is 0.3.Source: §A.1 · p. 19, Table 2 · p. 19
  • The first four rows report results for descriptor dimension d = 64, and the bottom four rows report corresponding results for d = 128.Source: §A.1 · p. 19, Table 2 · p. 19
  • The table's values are also depicted in Figure 5's line plot.Source: §A.1 · p. 19

A.2 Varying Key Model Hyperparameters

The appendix describes an ablation of input-point count, descriptor feature dimension, and nearest-neighbor count for local feature aggregation.

A.2 Varying Key Model Hyperparameters

  • The ablation varies input-point count, geometric-descriptor feature dimension, and nearest-neighbor count for local feature aggregation.Table 3 provides the numerical values corresponding to Figure 6's line plots.Source: §A.2 · p. 19, Table 3 · p. 19

A.3 Modifying the Training Strategy

The section compares training-strategy variations, with results reported in Table 4 and represented by Figure 7.

A.3 Modifying the Training Strategy

  • The experiments vary teacher initialization, absolute point-coordinate inputs, and rotation invariance during anomaly-detection training.Source: §A.3 · p. 20
  • The experiments also compare different pretraining datasets and pretrained feature extractors.Source: §A.3 · p. 20
  • Table 4 reports AU-PRO integrated up to a limit of 0.3 for these training-strategy variations.Source: Table 4 · p. 20, §A.3 · p. 20

Appendix B. Additional Qualitative Results

Figure 8 presents additional qualitative localization results across MVTec 3D-AD categories where the method reliably localizes anomalies.

Appendix B. Additional Qualitative Results

  • Figure 8 shows additional qualitative results for each MVTec 3D-AD category in which the method reliably localizes anomalies.Source: Appendix B · p. 21
  • The top row displays the algorithm’s anomaly scores for each 3D point.Source: Figure 8 · p. 21
  • The bottom row displays ground-truth annotations, with anomalous points shown in red.Source: Figure 8 · p. 21

Related papers

Loading 2202.11660v1…