PaperlayerView exact revision on arXiv

Source-linked AI summary

PointMixup: Augmentation for Point Clouds

Yunlu Chen, Vincent Tao Hu, Efstratios Gavves, Thomas Mensink, Pascal Mettes, Pengwan Yang, Cees G. M. Snoek

arXiv:2008.06374v1cs.CV

TL;DR

Point-cloud mixup is difficult because unordered point sets lack one-to-one correspondences. PointMixup uses optimal assignments to define shortest-path linear interpolations, and experiments report benefits for classification, especially with scarce data, plus greater robustness to perturbations. The method remains agnostic to the classifier function while focusing on augmentation.

  • Problem

    Image-domain mixup cannot directly extend to point clouds because their unordered points lack one-to-one correspondences.

  • Method

    PointMixup uses an optimal assignment between point sets to construct shortest-path linear interpolations and enable Mixup and Manifold Mixup for point clouds.

  • Results

    PointMixup outperforms baseline interpolations and regularizers, with accuracy 56.5 versus 35.1 for the baseline under noise σ = 0.05.

  • Takeaways & Limitations

    PointMixup provides model-agnostic augmentation that improves point-cloud classification and robustness across perturbations and learning settings.

  • Takeaways & Limitations

    The paper remains agnostic to the choice of mapping function h and focuses on data augmentation rather than a specific classifier architecture.

Abstract

from arXiv · show

This paper introduces data augmentation for point clouds by interpolation between examples. Data augmentation by interpolation has shown to be a simple and effective approach in the image domain. Such a mixup is however not directly transferable to point clouds, as we do not have a one-to-one correspondence between the points of two different objects. In this paper, we define data augmentation between point clouds as a shortest path linear interpolation. To that end, we introduce PointMixup, an interpolation method that generates new examples through an optimal assignment of the path function between two point clouds. We prove that our PointMixup finds the shortest path between two point clouds and that the interpolation is assignment invariant and linear. With the definition of interpolation, PointMixup allows to introduce strong interpolation-based regularizers such as mixup and manifold mixup to the point cloud domain. Experimentally, we show the potential of PointMixup for point cloud classification, especially when examples are scarce, as well as increased robustness to noise and geometric transformations to points. The code for PointMixup and the experimental details are publicly available.

1 Introduction

Point cloud augmentation is difficult because unordered point sets lack the point-to-point correspondence that makes image mixup straightforward. PointMixup addresses this by interpolating point clouds along shortest paths defined through optimal assignments, enabling mixup-style regularization and empirical gains.

  • Point cloud datasets are often smaller, increasing the importance of augmentation for classification accuracy and robustness.
  • PointMixup generates interpolated point clouds by solving an optimal bijective assignment that minimizes total point displacement.
  • Image mixup does not directly transfer to point clouds because unordered point sets lack one-to-one point correspondences.
  • The method defines point-cloud augmentation as shortest-path linear interpolation and supports both Mixup and Manifold Mixup regularizers.
  • The authors prove shortest-path, assignment-invariance, and linearity properties, then evaluate benefits across classification, few-shot, and semi-supervised learning.

2 Related Work

Prior work established permutation-invariant point-cloud networks and interpolation-based regularization in deep learning, while PointMixup extends Mixup to unordered point sets. Related augmentation approaches include auto-augmentation, adversarial training, and semi-supervised interpolation frameworks.

  • PointNet and later architectures process unordered point sets using permutation-invariant operations, with PointNet++ adding hierarchical structure through sampling and grouping.
  • Other point-cloud augmentation work uses auto-augmentation and a complicated adversarial training procedure, whereas PointMixup uses interpolation.
  • Mixup regularizes networks by interpolating examples and labels, while Manifold Mixup interpolates hidden representations at multiple layers.
  • PointMixup introduces optimal-assignment Mixup to handle the permutation-invariant nature of point clouds.
  • PointMixup can be integrated into semi-supervised frameworks that mix labeled and unlabeled data or enforce interpolation consistency.

3 Point cloud augmentation by interpolation

The paper defines point-cloud augmentation as shortest-path interpolation under Earth Mover’s Distance, using optimal point assignments to adapt Mixup and Manifold Mixup to unordered point sets.

  • Motivation: Point-cloud interpolation is challenging because points are unordered and lack the one-to-one correspondence that makes image Mixup straightforward.The goal is to generate interpolated point clouds while preserving permutation invariance.
  • Shortest-path interpolation: A shortest-path interpolation requires every intermediate cloud to satisfy d(S1, S(λ)) + d(S(λ), S2) = d(S1, S2).This property is intended to support a unique label distribution for interpolated data.
  • Distance metric: Earth Mover’s Distance supplies the point-cloud metric by minimizing total displacement through a bijective assignment between points in two clouds.The metric captures geometry, local details, and density distributions.
  • PointMixup: PointMixup, also called Optimal Assignment interpolation, uses the EMD-optimal assignment to define Mixup-like interpolation between point clouds.The same strategy can be applied to latent point-cloud representations for Manifold Mixup.
  • Analysis: The analysis addresses the non-trivial possibility that a shorter path could exist because optimal assignments involving mixed clouds are initially unknown.The proof uses the EMD triangle inequality and establishes the shortest-path property for all λ.
  • Analysis: PointMixup is proved to be a shortest-path linear interpolation in (S, dEMD), with assignment invariance ensuring intermediate paths remain part of the source path.Linearity also aligns the input interpolation ratio with the label interpolation ratio.

4 Experiments

Experiments evaluate PointMixup across interpolation settings, baselines, regularizers, perturbations, architectures, datasets, and limited-label regimes. PointMixup generally improves classification, robustness, and semi-supervised performance, while producing sensible mixed examples.

  • Experimental setup: Experiments use ModelNet40 and ScanObjectNN, including pre-aligned, unaligned, and reduced-training-data settings.ModelNet40 contains 12,311 CAD models across 40 categories; ScanObjectNN contains 2,902 real-world objects across 15 categories.
  • Ablation studies: The evaluation compares PointMixup with random-assignment and point-sampling interpolation baselines, alongside multiple regularization variants.The ablations cover interpolation ratios, baseline interpolations, other regularizers, and robustness to noise.
  • Baseline comparison: Across alignment variants and dataset sizes, PointMixup obtains favorable results compared with excluding interpolation and both interpolation baselines.The comparison uses ModelNet40 with PointNet++ and includes reduced training sets.
  • Regularization comparison: PointMixup with manifold mixup obtains the highest scores among the evaluated regularizers, outperforming label smoothing and other mixup variants.The comparison includes Mixup, Manifold Mixup, input-only mixing, fixed-layer latent mixing, and label smoothing.
  • Robustness: 56.5 versus 35.1 accuracy under noise with σ = 0.05, and 72.9 versus 59.2 under scaling by two, illustrate greater robustness with manifold mixup.The evaluation also tests point-location noise, translation, and rotations.
  • Qualitative analysis: Qualitative interpolations combine characteristics of source classes into sensible point clouds with mixed labels for cups, tables, chairs, and cars.Figure 5 presents eight examples across these object combinations.
  • Generalization across models and data: PointMixup improves results for PointNet and DGCNN, and also benefits real-world ScanObjectNN scans rather than only virtual point clouds.These experiments support applicability across architectures and datasets.
  • Beyond standard classification: With 800 labeled examples, semi-supervised PointMixup raises accuracy from 73.5% to 82.0% by incorporating additional unlabeled training examples.The semi-supervised evaluation uses 400, 600, and 800 labeled ModelNet40 point clouds within Interpolation Consistency Training.

5 Conclusion

PointMixup defines shortest-path interpolation for point clouds through optimal assignment and supports Mixup-style augmentation across classification settings. Experiments report benefits for scarce-data regimes, robustness, and broad applicability.

  • PointMixup interpolates point clouds through optimal assignment, producing shortest-path linear interpolation that respects their permutation-invariant structure.The method also extends to latent point-cloud representations for Manifold Mixup.
  • The approach outperforms baseline interpolations and regularizers across point-cloud classification experiments.The conclusion reports increased robustness to noise and geometric transformations.
  • PointMixup shows potential in semi-supervised and few-shot settings, particularly when labeled examples are scarce.The paper describes the method as generally applicable to point-based networks and datasets.

A Proofs for the properties of PointMixup interpolation

The appendix proves that PointMixup's optimal-assignment interpolation follows a shortest path and preserves assignment consistency and linearity. The proofs use Earth Mover's Distance and explicit assignments between source, intermediate, and target point clouds.

  • The appendix provides detailed proofs for the shortest-path, assignment-invariance, and linearity properties stated for PointMixup.
  • Shortest path property: The shortest-path proof represents each interpolated point as u_i = (1 − λ) · x_i + λ · y_φ*(i), using the optimal assignment φ* from S1 to S2.The argument compares transportation distances under the optimal and identical assignments and invokes the EMD triangle inequality.
  • Assignment invariance property: The assignment-invariance proof shows that φ_I remains optimal between S1 and an intermediate cloud S(λ1).The proof derives equality conditions across the relevant EMD inequalities and extends the argument to the intermediate-to-target assignment.
  • Linearity: The linearity proof uses the interpolated points u_k = (1 − λ1) · x_k + λ1 · y_φ*(k) and corresponding intermediate representations.It relies on the established optimal assignments between intermediate clouds and the endpoint cloud.

B Few-shot learning with PointMixUp

The few-shot procedure integrates PointMixup into episodic ProtoNet training by augmenting labeled support examples before query prediction. It learns episode-specific feature-extractor weights from mixed examples and then evaluates the query set.

  • PointMixup is evaluated for few-shot classification, where models generalize to unseen classes from only a few labeled examples.ProtoNet is the baseline and PointNet++ serves as the feature extractor.
  • Episodic ProtoNet setup: Each episode samples classes, forms labeled support and unlabeled query sets, and computes class prototypes from embedded support examples.ProtoNet classifies query examples using distances to the class prototypes.
  • PointMixup training procedure: PointMixup constructs mixed examples from the support set and updates episode-specific weights before predicting query labels.The updated feature extractor is used to recompute prototypes and classify query examples.
  • Training and testing distinction: Testing omits the query-based weight-learning updates used during episodic training.The training algorithm adds PointMixup between prototype calculation and query prediction.

C Further Discussion on Interpolation Variants

The discussion compares Optimal Assignment interpolation with Random Assignment and Point Sampling. Optimal Assignment performs best, while Random Assignment violates the shortest-path definition and can produce entangled, chaotic mixed clouds.

  • Interpolation variants: Optimal Assignment interpolation performs best, followed by Point Sampling, while Random Assignment performs worst among the compared strategies.The comparison is framed in terms of both theoretical advantages and practical performance.
  • Choice of interpolation: The variant analysis is used to validate choosing Optimal Assignment interpolation for PointMixup.
  • Random Assignment: Random Assignment lacks a shortest-path interpolation definition, allowing mixed points from different source examples to entangle.The discussion links this behavior to chaotic mixed examples that are difficult to recognize from source-class features.

Point Sampling interpolation: yet another shortest path interpolation

Point Sampling interpolation is argued to approximately define a shortest path on (S, dEMD) when point clouds contain many points, though the shortest path need not be unique.

  • Point Sampling interpolation can produce mixed examples whose source classes remain recognizable.This observation is reported from Fig. 3 in the main paper.
  • As N →∞, Point Sampling interpolation approximately defines a shortest path on the metric space (S, dEMD).The shortest path between fixed initial and final point clouds is not necessarily unique.
  • Point Sampling interpolation randomly samples points from each source cloud according to the sampling frequency λ.The construction samples without replacement from each set.
  • Table 5 reports interpolation-strategy accuracy percentages for PointNet in the unaligned setting.The table caption states that Point Sampling interpolation fails with PointNet, a density-invariant model.
  • Optimal assignment matches identical points as pairs, so those pairs contribute zero to the overall EMD distance.

N dEMD(S2, S1)

The derivation establishes that Point Sampling interpolation is approximately a shortest-path linear interpolation under EMD, with linearity in the mix ratio.

  • Point Sampling interpolation satisfies dEMD(S_PS, S2) = dEMD(S1, S2), which proves its shortest-path property.
  • Its linearity with respect to dEMD also holds, making Point Sampling interpolation approximately a shortest-path linear interpolation.

Point Sampling interpolation: limitations

Point Sampling interpolation changes local density rather than underlying shape, limiting its behavior with density-invariant architectures; OA interpolation instead smoothly morphs shape and is presented as more generalizable.

  • Point Sampling interpolation: limitations: The mix ratio λ changes local density distribution but does not vary the underlying shape in Point Sampling interpolation.
  • Point Sampling interpolation: limitations: Point Sampling interpolation fails with PointNet because max pooling aggregates information from all points and is ideally invariant to point density.
  • Point Sampling interpolation: limitations: PointNet++ can work with Point Sampling interpolation because farthest point sampling is sensitive to local density differences, despite global density invariance.Its sampling and grouping stage selects different farthest-point groups when local density changes.
  • Point Sampling interpolation: limitations: Mixup based on Point Sampling interpolation may remain limited because it does not work well with PointNet as a basic component in PointNet++.
  • Point Sampling interpolation: limitations: OA interpolation smoothly morphs the underlying shape and is not limited by point-density invariance, making it a more generalizable strategy.
Loading 2008.06374v1…