PaperlayerView exact revision on arXiv

Source-linked AI summary

DeepGMR: Learning Latent Gaussian Mixture Models for Registration

Wentao Yuan, Ben Eckart, Kihwan Kim, Varun Jampani, Dieter Fox, Jan Kautz

arXiv:2008.09088v1cs.CV

TL;DR

Point cloud registration must align 3D data despite large transformations, noise, and time constraints. DeepGMR learns pose-invariant point-to-distribution correspondences and uses differentiable compute blocks to recover transformations, outperforming geometry-based and learning-based baselines across varied settings while remaining efficient, generalizable, and noise-robust.

  • Problem

    Existing registration methods struggle with large transformations, noise, and time constraints, despite registration’s importance in 3D vision, graphics, and robotics.

  • Method

    DeepGMR learns pose-invariant point-to-distribution correspondences, estimates GMM parameters from point clouds, and recovers the optimal transformation through differentiable compute blocks without iterative EM.

  • Results

    DeepGMR outperforms state-of-the-art geometry-based and learning-based baselines across varied data settings, including noisy, new-category, and real-world data.

  • Takeaways & Limitations

    DeepGMR provides a global registration method for accurate and efficient alignment of 3D data, while integrating neural networks with probabilistic registration.

  • Takeaways & Limitations

    Point-to-point global methods can suffer on noisy point clouds, while iterative EM-based registration is computationally expensive and may get stuck in local minima for large transformations.

Abstract

from arXiv · show

Point cloud registration is a fundamental problem in 3D computer vision, graphics and robotics. For the last few decades, existing registration algorithms have struggled in situations with large transformations, noise, and time constraints. In this paper, we introduce Deep Gaussian Mixture Registration (DeepGMR), the first learning-based registration method that explicitly leverages a probabilistic registration paradigm by formulating registration as the minimization of KL-divergence between two probability distributions modeled as mixtures of Gaussians. We design a neural network that extracts pose-invariant correspondences between raw point clouds and Gaussian Mixture Model (GMM) parameters and two differentiable compute blocks that recover the optimal transformation from matched GMM parameters. This construction allows the network learn an SE(3)-invariant feature space, producing a global registration method that is real-time, generalizable, and robust to noise. Across synthetic and real-world data, our proposed method shows favorable performance when compared with state-of-the-art geometry-based and learning-based registration methods.

1 Introduction

DeepGMR addresses global point-cloud registration under large transformations, noise, and efficiency constraints by learning pose-invariant point-to-distribution correspondences. It combines learned GMM representations with closed-form, differentiable transformation recovery for robust and efficient alignment.

  • Point-cloud registration aligns input point clouds into a common coordinate frame for 3D vision, graphics, and robotics applications.
  • Existing methods struggle with global registration because local geometry matching cannot handle large transformations without good initialization, while other global methods can be slow or require accurate normals.
  • Noisy point clouds make point-to-point and feature-level correspondences unreliable, while distribution matching suffers from view-dependent GMM parameters and local convergence.
  • DeepGMR learns pose-invariant point-to-distribution parameter correspondences and recovers the optimal transformation in a single step without iterative EM.
  • DeepGMR runs at 20-50 frames per second with memory growing linearly in the number of points, tolerates noise and differing input sizes, and is fully differentiable.Its gradients can be obtained with a single backward pass, enabling integration into larger gradient-based optimization procedures.
  • Experiments on challenging synthetic and real-world data show favorable performance against geometry-based and learning-based state-of-the-art methods.The evaluated data include large transformations, noise, and scene-level real-world point clouds.

2 Related Work

Prior registration work spans efficient but initialization-sensitive local methods and global methods that are pose-invariant but often costly. DeepGMR is positioned as a global, learning-based probabilistic alternative using direct point-to-distribution correspondences.

  • Local and Global Registration: Local registration methods are efficient and effective when transformations are small, whereas global methods are invariant to initial conditions but often sacrifice efficiency.
  • Probabilistic Registration: Probabilistic local methods commonly use GMMs and EM, with later work targeting batch registration, density variation, viewing angle, and computational efficiency.
  • Baselines: The experiments compare DeepGMR with Trimmed ICP, HGMR, FGR, RANSAC, and TEASER++ as representatives of local, probabilistic, and geometry-based global registration.
  • Learning-Based Registration: Learning-based methods have distinct limitations: PointNetLK and PRNet remain local under large transformations, while DCP’s point-to-point matching suffers on noisy point clouds.
  • DeepGMR: DeepGMR is characterized as a global method and the first learning-based probabilistic registration method.

3 GMM-Based Point Cloud Registration

GMM-based registration models point clouds as probability distributions and alternates between fitting mixture parameters and optimizing a rigid transformation. This probabilistic formulation handles noise and outliers but conventional EM is slow and can fail under large transformations.

  • GMM formulation: A GMM represents a multimodal 3D probability distribution as a weighted sum of Gaussian components with weights, means, and covariances.Its parameters are Θ = {(π_j, µ_j, Σ_j)} for J components.
  • Registration objective: Registration is formulated as fitting a target GMM and then finding the SE(3) transformation that aligns the source point cloud to it.Both steps maximize point-cloud likelihood with respect to different parameters.
  • EM optimization: EM alternates E-steps that estimate latent point-to-component correspondences with M-steps that maximize expected joint likelihood to update parameters.The correspondence variables make the likelihood optimization tractable through a lower bound.
  • Advantages and limitations: Compared with point-to-point matching, the probabilistic formulation provides a differentiable way to handle noise and outliers.It replaces exact point matches with soft assignments based on component likelihoods.
  • Advantages and limitations: Iterative EM is computationally expensive and can get stuck in local minima when transformations are large because locality-based assignments create spurious associations.The method motivates learned pose-invariant point-to-GMM correspondences as a replacement for this association step.

4 DeepGMR

DeepGMR replaces conventional locality-based GMM correspondence estimation with a learned, pose-invariant network and computes mixture parameters and transformations in closed form. This yields a probabilistic registration pipeline that uses latent Gaussian components rather than noisy point-to-point matches.

  • 4 DeepGMR: DeepGMR combines a permutation-invariant point network for correspondences with differentiable blocks that compute GMM parameters and transformations.The network is trained by backpropagating a transformation loss through both compute blocks.
  • 4.1 Correspondence Network fψ: The correspondence network replaces the EM E-step and predicts non-local point-to-component associations without relying on Mahalanobis distances.This permits data-driven contextual associations and requires only a single iteration.
  • 4.2 MΘ Compute Block: The MΘ block converts correspondence probabilities and point coordinates into GMM parameters in closed form, with spatial overlap providing an inductive bias.The mapping is (Γ, P) → Θ.
  • 4.2 MΘ Compute Block: Isotropic covariances enable a closed-form transformation solver, while the experiments fix the number of components at J = 16.The component count is much smaller than the number of points, supporting time and memory efficiency.
  • 4.3 MT Compute Block: The MT block minimizes KL-divergence between transformed source and target latent distributions and recovers the optimal transformation in closed form.The formulation reduces alignment to matching latent component centroids and uses a weighted SVD solution.
  • 4.3 MT Compute Block: Using J = 16 and N = 1024, DeepGMR matches latent Gaussian components instead of point pairs, reducing problem dimension and avoiding the need for exact noisy correspondences.The paper contrasts this probabilistic matching with learning-based point-to-point SVD approaches.
  • 4.4 Implementation: The framework is architecture-agnostic and uses rotation-invariant input features with a PointNet segmentation backbone in the experiments.Training computes forward and inverse transformations and minimizes transformation error against ground truth.

5 Experimental Results

DeepGMR is evaluated on synthetic and real-world point clouds under noise, unseen geometry, large transformations, and efficiency constraints. It performs robustly across challenging settings while avoiding common correspondence failures and achieving very fast registration.

  • Datasets and evaluation: DeepGMR is evaluated on synthetic ModelNet40 and real-world Augmented ICL-NUIM point clouds using RMSE and recall with threshold 0.2.The evaluation compares geometry-based and learning-based registration baselines.
  • Quantitative results: DeepGMR achieves the best performance across datasets, including noisy, unseen-geometry, and real-world settings.The reported results attribute this performance to robust data association in challenging cases.
  • Quantitative results: 13.3s versus 11ms: TEASER++ matches DeepGMR on synthetic data but takes over 1000 times longer per instance.The comparison is reported for TEASER++ and DeepGMR registration time.
  • Failure-mode analysis: Global data association lets DeepGMR associate points with distant components using whole-shape context despite repetitive structures and symmetry.Distance-based matching and local descriptors can confuse correspondences in these cases.
  • Failure-mode analysis: Probabilistic optimization helps DeepGMR recover alignments from sparse regions, whereas thin parts can be ignored as outliers by distance-minimizing objectives.The mug handle is given as an example of a sparse but alignment-critical region.
  • Time efficiency: DeepGMR is the fastest among evaluated methods because it uses non-iterative registration through correspondences to a low-dimensional latent probability distribution.Timing varies with hardware across the compared methods, and some baselines exceed 10 seconds or run out of memory.

6 Conclusion

The paper presents DeepGMR as a learning-based probabilistic registration method that performs robustly across varied data settings. Its design combines efficient global alignment with a latent GMM-based registration paradigm.

  • DeepGMR is presented as a first attempt toward learning-based probabilistic registration.
  • DeepGMR outperforms state-of-the-art geometry-based and learning-based baselines across varied data settings, including noise, unseen categories, and real-world data.The authors also identify applications requiring accurate and efficient global alignment of 3D data.
  • The method provides a way to integrate 3D neural networks into a probabilistic registration paradigm.

DeepGMR: Learning Latent Gaussian Mixture Models for Registration –Supplementary Material–

The listed authors are Wentao Yuan, Ben Eckart, Kihwan Kim, Varun Jampani, Dieter Fox, and Jan Kautz, affiliated with the University of Washington and NVIDIA.

  • The authors are Wentao Yuan, Ben Eckart, Kihwan Kim, Varun Jampani, Dieter Fox, and Jan Kautz.
  • The listed affiliations are the University of Washington and NVIDIA.

1 Overview

The supplementary overview organizes additional technical details, experiments, visualizations, and limitation discussion that support the main paper.

  • The supplement provides additional details, discussions, and experiments supporting the original submission.
  • Section 2 focuses on derivations and proofs for the MT compute block.
  • Section 3 covers error distributions, ablations, robustness tests, and category-specific results.
  • Section 4 presents additional registration visualizations and learned latent GMMs, while Section 5 discusses limitations and future directions.

2 Additional Derivation

The derivation establishes equivalences that reduce probabilistic registration to expected log-likelihood minimization and reduce many point-component distances to component-level distances. It then derives the weighted rigid transformation in closed form using weighted Umeyama’s method.

  • KL-divergence equivalence: Minimizing KL-divergence from the transformed source GMM to the target is equivalent to maximizing transformed-source expected log likelihood under the target GMM.The transformed source distribution’s negative entropy is invariant to the transformation, leaving the cross-entropy term.
  • KL-divergence equivalence: A sufficiently large transformed source point cloud approximates the expected log likelihood through the law of large numbers, yielding the main-paper objective.The transformed source points are sampled independently from the transformed source GMM.
  • Single-sum reduction: The correspondence-derived GMM statistics reduce NJ point-component distance pairs to J component-level distance pairs.The reduction uses the correspondence matrix and its weighted sums of source points and transformed source points.
  • Single-sum reduction: Assuming isotropic target-component covariances, the transformation-dependent objective reduces to a single weighted term because the remaining term is invariant to T.The proof uses the linearity of the rigid transformation and the correspondence-derived moment identities.
  • Closed-form transformation: Weighted Umeyama’s method computes the optimal SE(3) rotation and translation from the centered cross-covariance matrix’s SVD while enforcing det R*=+1.The determinant constraint prevents reflections.

3 Additional Quantitative Results

Additional experiments characterize DeepGMR’s error distributions, ablations, density robustness, transformation-magnitude robustness, and category-wise behavior. The results indicate stable performance across several challenging conditions, with specific design choices affecting performance differently.

  • Full Error Distribution: CDF curves expose complete RMSE error distributions, while recall@0.2 is the curve’s recall value at threshold x = 0.2.A curve point gives the fraction of test instances with RMSE below its threshold; diamonds mark the maximum error when reached.
  • Full Error Distribution: Probabilistic data-association methods, including DeepGMR and HGMR, are reported as unaffected by noise, unlike point-to-point and feature-correspondence methods.The comparison covers ICP, DCP, PRNet, FGR, and PointNetLK alongside the probabilistic methods.
  • Full Error Distribution: Learning-based methods other than DeepGMR perform significantly worse on unseen categories under unrestricted rotation.The observation concerns PointNetLK, DCP, and PRNet.
  • Ablation Studies: Performance saturates for J > 16, so the experiments use J = 16 GMM components.The ablation varies the number of components and evaluates models with RMSE CDFs on ModelNet noisy.
  • Ablation Studies: Raw XYZ input still outperforms DCP, although RRI features further improve DeepGMR’s performance.The input ablation compares RRI features with raw xyz coordinates.
  • Ablation Studies: The RMSE-trained and MSE-trained models perform almost identically, indicating low sensitivity to the particular loss-function choice.The MSE-trained model has slightly lower maximum error.
  • Robustness Tests: DeepGMR is unaffected by point clouds up to 4 times denser than training data and only slightly worse up to 4 times sparser.The sparse-data degradation is attributed to missing geometric details.
  • Robustness Tests: DeepGMR’s pose-invariant correspondences preserve performance as transformation magnitude changes, whereas local methods degrade significantly and FGR and PRNet become less stable.DCP is also reported to share the invariance property, while FGR and PRNet rely on feature matching or keypoint detection.

4 Additional Qualitative Results

Qualitative results show DeepGMR learning consistent point-to-GMM correspondences across views and handling challenging geometric cases that trap other methods in local minima. The correspondences are learned end-to-end without explicit correspondence supervision.

  • Qualitative Results: DeepGMR handles repetitive structures, undersampled thin parts, and non-planar geometry that trap other methods in local minima.The qualitative examples come from ModelNet noisy and ICL-NUIM point clouds.
  • Qualitative Results: Different object parts are assigned consistently to different GMM components across views.The visualizations show the learned GMMs and correspondences.
  • Qualitative Results: The correspondence structure is learned end-to-end with the registration objective without explicit correspondence supervision.The learned assignments connect points to latent GMM components.

5 Future Work

DeepGMR does not explicitly model partial overlap because it estimates correspondences for all points and GMM components. Preliminary partial-data experiments show competitive performance with refinement, but a substantial gap remains relative to complete overlap.

  • Limitation: DeepGMR’s all-point, all-component correspondence model does not explicitly represent partial overlap, where only subsets should correspond.Partial overlap is defined as IoU below 1 after alignment.
  • Partial-overlap results: On artificially generated ModelNet40 partial data, DeepGMR remains competitive, and DeepGMR initialized ICP achieves the best performance.The partial clouds approximate an orthographic depth-camera rendering procedure.
  • Future work: Partial-overlap performance remains far below complete-overlap performance, motivating future integration with methods designed for partial overlap.The paper mentions attention mechanisms as one example of such techniques.
  • Qualitative evidence: The qualitative figures compare registration examples, including ModelNet40 noisy and ICL-NUIM results, with RMSE labels shown for examples in Fig. 5.Fig. 6 visualizes source and target point-to-GMM assignments and component uncertainty through sphere radii.
Loading 2008.09088v1…