PaperlayerView exact revision on arXiv

Source-linked AI summary

PointCloud Saliency Maps

Tianhang Zheng, Changyou Chen, Junsong Yuan, Bo Li, Kui Ren

arXiv:1812.01687v6cs.CVcs.AI

TL;DR

Point-cloud recognition lacks automatic point-wise importance measures despite the usefulness of saliency maps. This paper builds a point-cloud saliency map from loss-based point contributions using differentiable point-shifting, and evaluations across PointNet, PointNet++, and DGCNN show it consistently outperforms comparison schemes for targeted point dropping.

  • Problem

    Point-cloud recognition lacks an automatic way to evaluate each point’s importance for classification performance, limiting point- and subset-level saliency assessment.

  • Method

    The method assigns loss-contribution scores to points by approximating point dropping with continuous point shifting, enabling efficient saliency estimation.

  • Results

    The saliency-map-driven point-dropping algorithm consistently outperforms random and critical-subset strategies across PointNet, PointNet++, and DGCNN evaluations.

  • Takeaways & Limitations

    The saliency map quantifies point-level and subset-level importance and identifies points whose removal can improve or degrade recognition performance.

  • Takeaways & Limitations

    Critical-subset theory is not an accurate, exhaustive characterization of point- or subset-level saliency, and the reported comparison uses one vote for simplicity.

Abstract

from arXiv · show

3D point-cloud recognition with PointNet and its variants has received remarkable progress. A missing ingredient, however, is the ability to automatically evaluate point-wise importance w.r.t.\! classification performance, which is usually reflected by a saliency map. A saliency map is an important tool as it allows one to perform further processes on point-cloud data. In this paper, we propose a novel way of characterizing critical points and segments to build point-cloud saliency maps. Our method assigns each point a score reflecting its contribution to the model-recognition loss. The saliency map explicitly explains which points are the key for model recognition. Furthermore, aggregations of highly-scored points indicate important segments/subsets in a point-cloud. Our motivation for constructing a saliency map is by point dropping, which is a non-differentiable operator. To overcome this issue, we approximate point-dropping with a differentiable procedure of shifting points towards the cloud centroid. Consequently, each saliency score can be efficiently measured by the corresponding gradient of the loss w.r.t the point under the spherical coordinates. Extensive evaluations on several state-of-the-art point-cloud recognition models, including PointNet, PointNet++ and DGCNN, demonstrate the veracity and generality of our proposed saliency map. Code for experiments is released on \url{https://github.com/tianzheng4/PointCloud-Saliency-Maps}.

1. Introduction

Point-cloud recognition models are accurate and robust to many point drops, but critical-subset theory does not quantify each point’s importance. The paper addresses this gap with an efficient saliency-map approximation and validates it through targeted point dropping.

  • PointNet, PointNet++, and DGCNN handle irregular point clouds and achieve high classification accuracy.
  • Critical-subset theory explains robustness to non-critical point dropping but does not specify the importance of individual points or subsets.
  • Brute-force evaluation of all point combinations is impractical because its computational complexity scales exponentially with the number of points.
  • The proposed method approximates point dropping with a continuous point-shifting procedure and estimates saliency efficiently with a single backward step.
  • 200/1024 highest-scored points reduce PointNet accuracy to 49.2%/44.3% on 3D-MNIST/ModelNet40, versus 94.8%/87.7% for random dropping.

2. Preliminaries

The preliminaries define point clouds, prediction loss, point contribution, and saliency scores, then explain point-dropping evaluation and the recognition-model families considered. They also identify limitations of critical-subset theory as a complete saliency measure.

  • 2.1. Definition and Notations: A point cloud is represented by 3D points X = {x_i} and a ground-truth label y, while the classifier outputs a probability vector and cross-entropy loss.
  • 2.1. Definition and Notations: A point’s contribution is the loss difference between clouds excluding and including it; positive values indicate that adding the point reduces loss.
  • 2.1. Definition and Notations: The point-cloud saliency map assigns each point a score s_i intended to reflect its contribution to the model’s prediction.
  • 2.1. Definition and Notations: Dropping highest-scored points should degrade recognition, whereas dropping negative-scored points should improve it, providing a verification procedure.
  • 2.2. 3D Point-Cloud Recognition Models: The study focuses on point-cloud-based recognition models, including PointNet, PointNet++, and DGCNN, alongside volume- and multi-view-based approaches.
  • 2.2. 3D Point-Cloud Recognition Models: Critical-subset theory identifies a subset determining PointNet’s max-pooled features and output, but does not provide accurate, exhaustive point- or subset-level saliency.

3. Point-Cloud Saliency Map

The method approximates point dropping by shifting points toward a robust point-cloud center, then derives saliency from loss gradients in spherical coordinates. Experiments show that shifting selected points closely matches dropping them in classification outcomes.

  • 3.1. From Point Dropping to Point Shifting: Point dropping is approximated by shifting points toward the point-cloud center, making the loss change differentiable.The method treats inward shifting as eliminating a point’s effect on classification.
  • 3.1. From Point Dropping to Point Shifting: More than 95% of point-cloud pairs had identical classification results after either dropping or shifting the selected points.The pairs were generated by removing or center-shifting 100/1024 points under furthest, random, or saliency-based selection; identical results may be correct or wrong.
  • 3.2. Gradient-based Saliency Map: Point contribution is measured with the loss gradient along the radial coordinate in a spherical coordinate system.Spherical coordinates represent each point by radius and two angles, avoiding the view-angle problem of the original coordinate system.
  • 3.1. From Point Dropping to Point Shifting: The spherical core is constructed from the coordinate-wise medians of all points for outlier robustness.For each axis, the center uses the median coordinate value.
  • 3.2. Gradient-based Saliency Map: The saliency score includes a positive rescaling parameter α, set to 1 in the experiments, and points are visualized by score rankings.Larger ranking numbers indicate higher saliency scores.

4. Point Dropping Algorithms

The paper evaluates saliency maps through high- and low-score point dropping, including iterative updates that account for dependencies among remaining points. It also compares these procedures with critical-subset strategies.

  • 4.1. Saliency-Map based Point Dropping: Point dropping evaluates saliency by testing whether removing highest- or lowest-scoring points degrades or improves recognition performance.High-drop removes points with the highest scores, while low-drop removes points with the lowest scores.
  • 4.1. Saliency-Map based Point Dropping: The dropped high-saliency points form clusters that correspond to important recognition segments in the object.The remaining fragmented cloud may be recognized as another object after high-drop.
  • 4.1. Saliency-Map based Point Dropping: Iterative saliency-map dropping recomputes scores for the remaining points before removing n/T points at each iteration.This accounts for point dependencies that are ignored when all scores are calculated only once.
  • 4.2. Critical-Subset based Point Dropping: Among critical-subset strategies, iteratively dropping points contributing to at least two max-pooled features performs best, but still underperforms saliency-based dropping.This comparison supports the saliency map as a more accurate point- and subset-level measure.

5. Experiments

Experiments evaluate the saliency-map-driven point-dropping methods on 3D-MNIST and ModelNet40 using PointNet, PointNet++, and DGCNN. Across models, high-drop identifies points whose removal substantially degrades accuracy, while low-drop can improve performance and the method generalizes across architectures.

  • Datasets and Models: The evaluation uses 3D-MNIST and ModelNet40 with PointNet, PointNet++, and DGCNN under dynamic point-number inputs.3D-MNIST contributes 50,000 training and 10,000 testing point clouds of 1,024 points; ModelNet40 uses 9,843 training and 2,468 testing models.
  • Results on PointNet: High-drop reduces PointNet accuracy from 94% ∼95% to 49.2% on 3D-MNIST and from 87% ∼89% to 44.3% on ModelNet40.On ModelNet40, low-drop increases accuracy to 91.4%, whereas high-drop reduces it to 44.3%.
  • Results on PointNet++: PointNet++ reaches 97.2%/59.5% on 3D-MNIST and 91.1%/58.5% on ModelNet40 when the algorithm increases/reduces accuracy.The corresponding random-drop accuracies remain 95% ∼96% and 88 ∼90%.
  • Results on DGCNN: DGCNN reaches 97.2%/76.4% on 3D-MNIST and 91.3%/64.2% on ModelNet40 when the algorithm increases/reduces accuracy.Under random dropping, DGCNN maintains 96% ∼97% and 89% ∼91% accuracy on the two datasets.
  • Parameter Study: With 600/1024 points dropped on PointNet and ModelNet40, random-drop remains over 80% accurate while high-drop reduces accuracy to nearly 0.The parameter study recommends 20 iterations for high-drop and one iteration for low-drop because additional low-drop iterations add computational cost with limited improvement.
  • Generalization and Visualization: PointNet-generated subsets with high-score points removed still cause substantial accuracy degradation when tested on PointNet++ and DGCNN.Visualizations show that high-drop identifies important distinguishing segments, such as a lamp base; DGCNN appears most robust to point dropping among the three models.

6. Conclusion

The paper constructs point-cloud saliency maps by approximating point dropping with continuous point shifting and evaluates them through high- and low-score dropping. The saliency-map-driven procedure consistently outperforms random and critical-subset strategies for measuring point- and subset-level saliency.

  • The method constructs a saliency map to characterize point-level and subset-level saliency in 3D point clouds.
  • Point dropping with the highest or lowest saliency scores serves to verify whether the map identifies points whose removal degrades or improves recognition.
  • High-drop changes originally correct predictions into wrong predictions, whereas low-drop changes originally wrong predictions into correct predictions.
  • The saliency-map-driven point-dropping algorithm consistently outperforms random point dropping and critical-subset-based strategies.
Loading 1812.01687v6…