PaperlayerView exact revision on arXiv

Source-linked AI summary

Adversarial Attack and Defense on Point Sets

Jiancheng Yang, Qiang Zhang, Rongyao Fang, Bingbing Ni, Jinxian Liu, Qi Tian

arXiv:1902.10899v4cs.CVcs.AIcs.CRcs.LG

TL;DR

Point-cloud networks face security and robustness concerns because adversarial perturbations, point additions or removals, and sensing noise can alter decisions. This paper develops attacks, a perturbation-measurement defense, and transferability analyses, finding that pointwise-gradient attacks are effective, defenses detect adversarial examples, and momentum improves transferability.

  • Problem

    Point-cloud networks require stronger security and robustness evidence against adversarial perturbations, sensing noise, and missing or added points in safety-critical applications.

  • Method

    The paper develops pointwise-gradient, point-attachment, and point-detachment attacks alongside perturbation-measurement defenses and transferability analyses across point-cloud networks and grid CNNs.

  • Results

    Pointwise-gradient attacks achieve high success rates, defenses detect adversarial point clouds including defense-targeting attacks, and momentum-enhanced gradients improve attack transferability.

  • Takeaways & Limitations

    The findings support evaluating point-cloud models for adversarial robustness, physical feasibility, detection capability, and cross-architecture transferability.

  • Takeaways & Limitations

    All attack methods are developed in white-box settings, and the point-detach method is designed for PointNet rather than universally across point-cloud networks.

Abstract

from arXiv · show

Emergence of the utility of 3D point cloud data in safety-critical vision tasks (e.g., ADAS) urges researchers to pay more attention to the robustness of 3D representations and deep networks. To this end, we develop an attack and defense scheme, dedicated to 3D point cloud data, for preventing 3D point clouds from manipulated as well as pursuing noise-tolerable 3D representation. A set of novel 3D point cloud attack operations are proposed via pointwise gradient perturbation and adversarial point attachment / detachment. We then develop a flexible perturbation-measurement scheme for 3D point cloud data to detect potential attack data or noisy sensing data. Notably, the proposed defense methods are even effective to detect the adversarial point clouds generated by a proof-of-concept attack directly targeting the defense. Transferability of adversarial attacks between several point cloud networks is addressed, and we propose an momentum-enhanced pointwise gradient to improve the attack transferability. We further analyze the transferability from adversarial point clouds to grid CNNs and the inverse. Extensive experimental results on common point cloud benchmarks demonstrate the validity of the proposed 3D attack and defense framework.

I. INTRODUCTION … B. Adversarial Examples and Adversarial Attack

The paper studies the security and robustness of point-cloud networks, proposing attacks that manipulate points and a perturbation-measurement defense, while analyzing attack transferability across models and representations. It motivates these methods by the efficiency of direct point-set learning and the vulnerability of point-cloud classifiers despite claimed robustness to small corruptions.

  • I. INTRODUCTION: The proposed perturbation-measurement defense detects most adversarial examples generated against PointNet, exceeding 75% detection with only the victim model.The defense measures statistics of perturbed model outputs rather than using perturbed examples directly.
  • I. INTRODUCTION: Momentum-enhanced gradient attacks improve transferability, whereas vanilla gradient-guided attacks transfer poorly; high-accuracy models do not necessarily provide high adversarial robustness.The transferability analysis considers several point-cloud networks and cross-domain transfer between point clouds and grid CNNs.
  • I. INTRODUCTION: The work introduces three point-cloud attack methods, a flexible victim-model defense framework, and transferability analyses between point-cloud networks and grid CNNs.The attacks use pointwise gradient perturbation and adversarial point attachment or detachment.
  • A. Deep Learning on Point Clouds: Point clouds support direct, permutation-invariant, size-varying learning that avoids the computation-intensive rendering and convolution required by voxel-based methods.Point-cloud classification is benchmarked on ModelNet40, while PointNets and DeepSets are cited as successful direct point-set learners.
  • A. Deep Learning on Point Clouds: Although PointNets use critical points and are claimed robust to small corruptions or extra noise points, the paper demonstrates their vulnerability to corruption.For max-pooling aggregation, at most K critical points determine the final output, while remaining points can be removed without changing it.
  • B. Adversarial Examples and Adversarial Attack: The study defines untargeted adversarial examples as perturbed inputs whose predicted class differs from the benign sample under a distance constraint, focusing on white-box point-cloud attacks.White-box attackers have access to model internals and gradients; point clouds differ from images because they are orderless, unstructured sets.
  • B. Adversarial Examples and Adversarial Attack: Because outlier noise is common in 3D sensing, experiments use asymmetric Chamfer pseudo-distance, while point attachment and detachment require a separate measurement of the number of changed points.Chamfer distance is more tolerant of target-set outliers than Hausdorff distance, but neither distance adequately measures point-count changes.

C. Defense on Adversarial Attack … B. Pointwise Gradient Method

The paper studies white-box, untargeted attacks and detect-and-reject defenses for point-cloud networks on ModelNet40 and MNIST, including adversarial transferability. Its pointwise gradient attacks perturb each point under a distance budget, while momentum improves transferability over vanilla gradient descent.

  • C. Defense on Adversarial Attack: The study focuses on PC-Nets using white-box, untargeted attacks on ModelNet40 and MNIST, with a detect-and-reject defense and explicit transferability analysis.It also distinguishes its framework from a parallel point-set attack study by adding point detachment and a novel defense framework [49].
  • A. Principles and Notations: The framework represents point clouds as X ∈ R^N×c, with N points and c-dimensional coordinates, and defines f(·) as class-probability outputs with c*(·) as the true label.For 3D point clouds, c = 3.
  • A. Principles and Notations: An attack transforms an input point cloud X into Xa under budget ϵ and distance metric DC to reduce the attacked model’s classification accuracy.Attack performance is evaluated by post-attack model accuracy.
  • III. ATTACK METHODS: The Pointwise Gradient method generates adversarial point clouds by applying small, iterative gradient-guided perturbations to every point under a Chamfer-distance budget.The method uses the gradient obtained by backpropagating the ground-truth class output in the untargeted setting.
  • B. Pointwise Gradient Method: Using l2-normalized gradients yields better attack performance and more stable convergence, defining the vanilla-gradient Pointwise Gradient method.The method is based on an iterative perturbation formula rather than the single-step sign update associated with FGSM [14].
  • B. Pointwise Gradient Method: Momentum-enhanced Pointwise Gradient accumulates gradients before applying them to the victim sample, extending vanilla gradient descent for transferable adversarial examples.A momentum factor µ is introduced for gradient accumulation and is typically set to 1 in the experiments.
  • B. Pointwise Gradient Method: MPG improves adversarial point-cloud transferability over PG by accumulating gradients with a momentum factor typically set to 1.The momentum-enhanced method is introduced specifically to produce more transferable adversarial examples.

C. Point-Detach Method

The point-detach method models realistic adversarial point loss by removing a limited number of points, motivated by occlusion and scale-related point vanishing in 3D sensing. For PointNets, it ranks critical points by class-dependent importance and greedily detaches the most important ones.

  • Motivation: Point-detach attacks remove a few points to model adversarial point loss, a realistic scenario because 3D sensing commonly loses points through occlusion and scale issues.This contrasts with the Pointwise Gradient Method, which perturbs every point and is difficult to realize physically.
  • PointNets strategy: For PointNets [36], the strategy targets critical points because the model output changes if and only if a missing point is critical.The method defines class-dependent importance to efficiently construct an untargeted attack.
  • Importance estimation: The method estimates the effect of detaching a critical point by replacing each affected channel value with its second-largest activation and applying a first-order Taylor approximation.The feature gradient is sparse because it is nonzero only at critical points.
  • Optimization: A greedy procedure repeatedly detaches the remaining point with greatest true-class-dependent importance until Nd points are removed, recomputing importance after every iteration.This recomputation yields an O(N · Nd) point-detach algorithm.

D. Point-Attach Method · IV. DEFENSE METHODS · A. Principle and Notation

The Point-Attach Method creates adversarial point clouds by attaching a few optimized points near object surfaces while leaving the original points unchanged. The defense framework detects adversarial inputs through repeated perturbation, output-statistic measurement, and thresholding, evaluated with AUROC and DDR.

  • D. Point-Attach Method: The Point-Attach Method attaches a few points at appropriate positions to generate a variant of adversarial point-cloud attack.The attached points are optimized while the original point cloud remains unchanged.
  • D. Point-Attach Method: A Lagrange multiplier restricts attached points to move around object surfaces, with λ = 0.001 used empirically.The optimization updates only attached points and stops when the adversarial budget is exceeded.
  • D. Point-Attach Method: The method has a theoretical guarantee that sufficiently close point attachments can change the classifier’s predicted class for some dataset sample.The proposition bounds both the point-cloud distance and the number of attached points.
  • A. Principle and Notation: The defense aims to detect and reject test samples that may be adversarial point clouds rather than benign inputs.Its premise is that adversarial outputs are less stable under small perturbations than natural outputs.
  • A. Principle and Notation: The framework repeatedly applies perturbations, measures statistics of the resulting outputs, and detects adversarial examples by thresholding those statistics.It uses perturbation measurement rather than the perturbed prediction itself, and instantiates the framework for different attack scenarios.
  • A. Principle and Notation: PointNet serves as the PC-Net in the framework, whose output distributions differ between benign samples and adversarial examples.Statistics are computed over repeated perturbed outputs to capture this difference.
  • A. Principle and Notation: AUROC measures adversarial-versus-natural separability, while DDR measures detected adversarial examples at a specified natural-example rejection rate.AUROC is threshold-free and class-imbalance-insensitive; DDR is evaluated with adversarial examples as the positive class.

B. Perturbation Methods · C. Measurement Methods

The framework perturbs point clouds with Gaussian noise, quantification, or random sampling, then detects adversarial instability using variance- and confidence-based measurements. Max-confidence measurements are theoretically guaranteed under stated local-function and adversarial-density assumptions when paired with Gaussian noising or quantification.

  • B. Perturbation Methods: Gaussian noising adds i.i.d. N(0, σ2) perturbations that help models escape narrow adversarial sub-spaces for subsequent detection.The method is denoted Gσ and uses non-directional noise.
  • B. Perturbation Methods: Quantification converts inputs to multiple low-precision levels from µ/M to µ, increasingly distorting natural examples while destabilizing adversarial outputs.Adversarial examples produce chaotic classifier-output distributions and distinguishable statistics under quantification.
  • B. Perturbation Methods: Random sampling changes the number of points by retaining n < N points sampled without replacement and is especially effective against point-attach attacks.Each point’s inclusion is indicated by a Bernoulli(0.5) variable.
  • C. Measurement Methods: Set-Indiv Variance Measurement averages per-class confidence variances because adversarial examples destabilize outputs and variance is more distinguishable than entropy.This measurement is empirically the most effective in most cases.
  • C. Measurement Methods: Max Confidence-Based Measurement statistically uses maximum class-confidence scores through their average CoA and variance CoV to detect adversarial examples.The approach is inspired by prior out-of-distribution detection methods [18], [27].
  • C. Measurement Methods: Under local continuity and convexity or concavity, with adversarial examples sufficiently sparse near natural examples, max-confidence measurements admit a theoretical effectiveness guarantee.The assumptions formalize a small adversarial proportion within a local neighborhood Dδ around each natural example.
  • C. Measurement Methods: For any sample, CoV detects adversarial examples for convex functions and CoA for concave functions when combined with Gaussian noising or quantification.This is the stated Proposition 3 guarantee.

D. Attack over the Defenses … B. Defense Performance

The proposed attacks substantially compromise PointNet while remaining visually subtle, and the perturbation-measurement defenses detect most generated adversarial point clouds. A defense-targeting EoTPG attack is also tested, yet the defenses remain effective against it.

  • D. Attack over the Defenses: The defense-targeting EoTPG variant averages perturbations over 100 transformations per attack step, yet the proposed defenses still detect its adversarial point clouds.This robustness contrasts with the reported effectiveness of expectation-over-transformation attacks against randomness-based defenses [2], [50].
  • V. RESULTS: The experiments use PointNet with T-Net on ModelNet40, whose official split contains 9,843 training and 2,468 test, attack, and defense examples sampled with 1,024 points.The evaluation also reports attack performance on this benchmark and conducts defense experiments on the resulting adversarial point clouds.
  • A. Attack Performance: Pointwise-Gradient reduces PointNet classification accuracy to 0%, successfully attacking all natural examples.This result is reported on ModelNet40 under attack intensities summarized in Table I.
  • A. Attack Performance: Point-Detach and Point-Attach are less effective than Pointwise-Gradient for accuracy reduction but are more physically feasible and visually imperceptible.Both attacks outperform random point detachment or attachment, and their effectiveness can increase under looser constraints.
  • B. Defense Performance: Kernel-density estimates of −log(SIV) show significantly different natural and adversarial distributions, enabling detection with an appropriate threshold.The KDE analysis includes correct natural examples and adversarial examples, with the latter producing distinguishable statistics.
  • B. Defense Performance: The defenses successfully detect most adversarial examples, exceeding 75% detection, although detection declines as more critical points are detached.The decline occurs because detached critical points reduce output fluctuation; performance is evaluated with optimal perturbation-measurement settings in Table II.
  • B. Defense Performance: CoA and CoV are theoretically expected to detect adversarial examples but show limited empirical performance, with detailed comparisons reported in Appendix D.Defense results are quantified using AUROC and Defense Detection Rate (DDR), under both all-natural and correctly classified natural-example settings.

C. Performance of Attack over the Defenses · VI. TRANSFERABILITY OF ADVERSARIAL EXAMPLES · A. Between Various PC-Nets

The proposed defenses remain effective against EoTPG attacks designed to target them, while adversarial point-cloud transferability varies substantially across PC-Nets and improves with momentum-enhanced gradients.

  • C. Performance of Attack over the Defenses: Defense performance generally improves against specifically designed EoTPG attacks compared with vanilla PG, including attacks directly targeting the defense strategy.EoTPG uses 100 perturbations per attack step with random sampling, quantification, or Gaussian-noising perturbations.
  • VI. TRANSFERABILITY OF ADVERSARIAL EXAMPLES: The transferability study evaluates black-box attacks between PointNet [36], PointNet++ [38], and DGCNN [47], which differ in their kNN-graph constructions.PointNet++ uses hierarchical static kNN graphs, whereas DGCNN dynamically reconstructs them.
  • A. Between Various PC-Nets: PointNet++ is less robust than PointNet under attack despite outperforming it without attack, potentially because attacked examples induce differently structured static kNN graphs.The authors conjecture that these graphs weaken the robustness of hidden representations.
  • A. Between Various PC-Nets: DGCNN is more robust than PointNet and PointNet++ under attack and achieves the highest unattacked performance, partly because dynamic edge convolution causes shattered gradients [2].The comparison varies attack intensity using Chamfer distance DC.
  • A. Between Various PC-Nets: Transferability is evaluated by generating adversarial examples on one source PC-Net and measuring pre-attack and post-attack accuracy on another network.The evaluation uses vanilla Pointwise Gradient examples and an attack budget of DC = 0.03.
  • A. Between Various PC-Nets: Vanilla Pointwise Gradient adversarial point clouds transfer poorly across PC-Nets, making it ineffective as a black-box attack in point-cloud settings.High post-attacking accuracy indicates that only a few adversarial examples successfully transfer to evaluating networks.
  • A. Between Various PC-Nets: Momentum-enhanced Pointwise Gradient (MPG) significantly improves adversarial-example transferability between PC-Nets, enabling more effective black-box attacks.The study introduces MPG after vanilla Pointwise Gradient shows poor transferability across evaluating networks.

B. Between PC-Nets and CNNs · VII. DISCUSSION · VIII. CONCLUSION

The study finds asymmetric transferability between point-cloud and image CNN attacks, while discussing rotation sensitivity, defense errors, limitations, and future directions. It concludes that PG is broadly effective, PA/PD are more physically feasible, and permutation-based defense detects adversarial clouds, including proof-of-concept attacks targeting it.

  • B. Between PC-Nets and CNNs: Point-cloud attacks partially transfer to CNNs, whereas CNN attacks scarcely transfer to PointNet: CNN accuracy falls from 99.1% to 60.8%, while PointNet remains 98.9% accurate when CNN accuracy reaches 0%.PointNet and LeNet-like CNN achieve comparable clean accuracies of 99.2% and 99.1%, respectively.
  • VII. DISCUSSION: Rotation is an ineffective adversarial strategy for PointNet: under a 0.3-rad constraint, attacked accuracy is 76.01%, close to 77.53% for random rotation.The discussion attributes this weak attackability to PointNet’s sensitivity to random rotation and notes training with rotation augmentation and T-Net.
  • VII. DISCUSSION: The defense has two error sources: overlapping adversarial and natural-example distributions, and misclassified natural examples being mistaken for adversarial ones.The first error motivates improved perturbation measurements and hyperparameter tuning; defense performance is better on correctly classified natural examples than on all natural examples.
  • VII. DISCUSSION: The work remains limited because attacks are developed white-box, Point-Detach is designed for PointNet, and a universal point-cloud attack method is still needed.The discussion calls for more black-box attack and defense research despite experiments on black-box transferability.
  • VII. DISCUSSION: Physical adversarial point clouds remain challenging because 3D sensors use diverse formats and introduce illumination and sensing noise, making physical 3D-object generation harder than for 2D images.The paper suggests investigating physically realized adversarial point clouds using 3D printing.
  • VIII. CONCLUSION: The study’s attacks trade effectiveness against physical feasibility: PG is broadly effective across point-cloud networks, whereas PA and PD have lower success rates but greater physical feasibility.The conclusion summarizes three proposed point-cloud attack methods and their differing properties.
  • VIII. CONCLUSION: The permutation-measurement defense detects adversarial point clouds, including those produced by a proof-of-concept attack designed to target the defense.The conclusion reports empirical support for the defense framework.
  • VIII. CONCLUSION: Future work targets physically feasible 3D-object attacks, black-box attacks, stronger defenses, and improved robustness of point-cloud networks.The authors also caution that defenses may not remain strong long term as adversarial-point-cloud safety becomes more important.

APPENDIX A PROOFS … C. On Max Confidence-Based Measurement

The appendix proves that pointwise-gradient and point-attachment attacks can induce misclassification under bounded perturbation, while Max Confidence-Based Measurement detects adversarial samples under stated local-function and sparsity assumptions. For concave and convex PC-Nets, the proofs establish CoA- and CoV-based detection, respectively.

  • A. On Pointwise-Gradient Attack: For pointwise-gradient attacks, some correctly classified point cloud can be transformed into a point cloud within distance ϵ whose predicted class differs from the true class.The bound ϵ is chosen over the dataset, with ϵX = DC(X, X′) for each transformed sample.
  • A. On Pointwise-Gradient Attack: The pointwise-gradient proof formalizes attack construction by changing any sample with label t pointwise into another sample with label t′, then taking the dataset-wide maximum perturbation distance.This establishes the existence of a finite ϵ supporting the attack proposition across the point cloud dataset.
  • B. On Point-Attach Attack: For point-attach attacks, some correctly classified point cloud can be misclassified after attachment within distance ϵ and with at most Na point-count change.The proof attaches one of two differently labeled examples to the other; whichever resulting prediction matches determines the successfully attacked sample.
  • C. On Max Confidence-Based Measurement: Max Confidence-Based Measurement can always detect adversarial versus benign samples using CoV for convex PC-Nets and CoA for concave PC-Nets, with Gaussian Noising or Quantification perturbation.This proposition assumes PC-Nets are locally continuous and convex or concave, and adversarial examples form a sufficiently small proportion of each local neighborhood.
  • C. On Max Confidence-Based Measurement: The concave-case proof models Quantification as uniform-distribution perturbation and separates negative adversarial from positive natural examples using the 0.5 classification threshold.It applies the same perturbation to quantized adversarial and natural samples before comparing their confidence behavior.
  • C. On Max Confidence-Based Measurement: For locally concave PC-Nets, CoA is effective because the benign sample has greater distance from the decision-boundary point than the adversarial sample, yielding CoA(x2) > CoA(x1).The proof uses a shared perturbation and the point c where f(c) = 0.5; the local adversarial-example sparsity assumption gives DC(c, x2) > DC(x1, c).
  • C. On Max Confidence-Based Measurement: For locally convex PC-Nets, CoV is effective because equal perturbation widths and convexity imply CoV(x1) > CoV(x2) for adversarial and benign samples, respectively.The proof compares slopes at a1 and b2, using a1 > b2 and b2 − a2 = b1 − a1.

APPENDIX B EXPERIMENT DETAILS IN THIS STUDY … E. Training of CNN for MNIST

Appendix B details training configurations for PointNet, PointNet++, DGCNN, and CNN experiments on ModelNet40 and MNIST, including preprocessing, optimization, initialization, and implementation choices.

  • A. Training of PointNet for ModelNet40: PointNet on ModelNet40 uses the official topology with cross-entropy loss, zero-centered inputs scaled to a maximum point-to-center L2 distance of 1 m, and standard-normal initialization.Training uses Adam at learning rate 0.001, batch size 32, 250 epochs, shuffled data, PyTorch 1.0, and Titan Xp GPUs.
  • APPENDIX B EXPERIMENT DETAILS IN THIS STUDY: Appendix Table A1 reports additional defense results obtained by applying the perturbation and measurement methods against various attacks.
  • B. Training of PointNet++ for ModelNet40: PointNet++ and DGCNN on ModelNet40 follow their official topologies with cross-entropy loss and zero-centered inputs whose maximum point-to-center L2 distance is 1 m.Both use Adam with initial learning rate 0.001, decay by 0.7 every 200000 steps to a 0.00001 minimum, 250 epochs, shuffled data, TensorFlow 1.4, and Titan Xp GPUs.
  • C. Training of DGCNN for ModelNet40: DGCNN on ModelNet40 follows its official network, uses cross-entropy loss and zero-centered inputs scaled to a maximum point-to-center L2 distance of 1 m, and is trained for 250 epochs.Its Adam learning rate starts at 0.001, decays by 0.7 every 200000 steps to 0.00001, and training uses shuffled data with TensorFlow 1.4 on Titan Xp GPUs.
  • D. Training of PointNet for MNIST: PointNet for MNIST converts each pixel at coordinate (x,y) with gray value c into a point (x,y,c), then applies ModelNet40-style point-cloud processing.The network uses standard-normal initialization, Adam at learning rate 0.001 for 25 epochs, shuffled data, PyTorch 1.0 on Titan Xp GPUs, and cross-validation with grid search for approximate epoch and learning-rate selection.
  • E. Training of CNN for MNIST: The MNIST CNN matches LeNet, uses cross-entropy loss and standard-normal initialization, and is trained with Adam at learning rate 0.001 for 25 epochs on shuffled data.Implementation uses PyTorch 1.0 on Titan Xp GPUs, with cross-validation and grid search selecting approximate epoch and learning-rate values.

APPENDIX C DETAILS OF ROTATION ATTACK · APPENDIX D MORE DEFENSE RESULTS · APPENDIX E VISUALIZATION ON POINTWISE-GRADIENT ATTACK

The appendices detail Euler-angle rotation attacks, enumerate defense-method combinations, and visualize how Pointwise Gradient attack intensity affects point-cloud distortion. Certain examples become visibly distorted once Chamfer distance exceeds 0.02.

  • APPENDIX C DETAILS OF ROTATION ATTACK: The rotation attack parameterizes rotations with Euler angles α, β, and γ around the axes shown in Figure A3.The rotation system uses the x-axis, y′-axis, and x′-axis.
  • APPENDIX C DETAILS OF ROTATION ATTACK: The Euler-angle rotation is constructed from rotation matrices for α, β, and γ, corresponding to sequential axis rotations.The appendix provides the matrix forms used for the three angle parameters.
  • APPENDIX C DETAILS OF ROTATION ATTACK: Following gradient-based iterations, the attack updates θ adversarially while limiting each Euler-angle dimension to 0.3 rad.Here, θ comprises α, β, and γ, following the Pointwise Gradient iteration approach.
  • APPENDIX D MORE DEFENSE RESULTS: Table A1 reports defense AUROC for every combination of Gaussian Noising, Quantification, or Random Sampling with SIV, CoA, or Cov.The appendix expands results across all proposed Perturbation × Measurement combinations.
  • APPENDIX D MORE DEFENSE RESULTS: Appendix D evaluates the proposed defense through the cross-product of three perturbation methods and three measurement methods.The perturbation methods are Gaussian Noising, Quantification, and Random Sampling; the measurement methods are SIV, CoA, and Cov.
  • APPENDIX E VISUALIZATION ON POINTWISE-GRADIENT ATTACK: Chamfer distance above 0.02 visibly distorts certain point-cloud examples under Pointwise Gradient attacks, visualized across attack levels from 0.01 to 0.05.The visualization compares benign samples with adversarial examples at varying Chamfer distances.
Loading 1902.10899v4…