PaperlayerView exact revision on arXiv

Source-linked AI summary

You Only Hypothesize Once: Point Cloud Registration with Rotation-equivariant Descriptors

Haiping Wang, Yuan Liu, Zhen Dong, Wenping Wang

arXiv:2109.00182v2cs.CV

TL;DR

Point-cloud registration must recover accurate transformations between unaligned partial scans, but conventional rotation handling relies on fragile local reference frames. YOHO uses icosahedral-group features to combine rotation-invariant matching with rotation-equivariant rotation estimation, enabling one-correspondence RANSAC hypotheses. Across multiple datasets, it achieves state-of-the-art registration with far fewer RANSAC iterations, while descriptor construction remains computationally costly and planar overlaps can make rotation estimation ambiguous.

  • Problem

    PCA-based local reference frames used for rotation-invariant descriptors are ambiguous and sensitive to noise and point-density variation.

  • Method

    YOHO builds descriptors from icosahedral-group features, pooling for rotation invariance and using feature-map permutations for rotation estimation from one matched pair.

  • Results

    YOHO achieves state-of-the-art or better-or-comparable registration accuracy across multiple datasets with far fewer RANSAC iterations than existing methods.

  • Takeaways & Limitations

    Combining rotation-invariant matching with rotation-equivariant rotation estimates reduces the transformation search space and improves registration accuracy and efficiency.

  • Takeaways & Limitations

    YOHO may fail on mainly planar overlap regions, and YOHO-Desc construction is less efficient because it requires 60 backbone forward passes.

Abstract

from arXiv · show

In this paper, we propose a novel local descriptor-based framework, called You Only Hypothesize Once (YOHO), for the registration of two unaligned point clouds. In contrast to most existing local descriptors which rely on a fragile local reference frame to gain rotation invariance, the proposed descriptor achieves the rotation invariance by recent technologies of group equivariant feature learning, which brings more robustness to point density and noise. Meanwhile, the descriptor in YOHO also has a rotation equivariant part, which enables us to estimate the registration from just one correspondence hypothesis. Such property reduces the searching space for feasible transformations, thus greatly improves both the accuracy and the efficiency of YOHO. Extensive experiments show that YOHO achieves superior performances with much fewer needed RANSAC iterations on four widely-used datasets, the 3DMatch/3DLoMatch datasets, the ETH dataset and the WHU-TLS dataset. More details are shown in our project page: https://hpwang-whu.github.io/YOHO/.

1 INTRODUCTION

YOHO addresses fragile rotation handling in point-cloud registration with a descriptor that combines rotation invariance and equivariance. This enables one-correspondence transformation hypotheses and strong registration accuracy with far fewer RANSAC iterations.

  • Point-cloud registration seeks accurate SE(3) transformations between unaligned partial scans for tasks including reconstruction, pose estimation, AR/VR, and autonomous driving.
  • PCA-based local reference frames are ambiguous and sensitive to noise and density variation, risking incorrect alignment between corresponding points.The paper illustrates this sensitivity by downsampling patches and adding noise, each of which changes the constructed frame.
  • YOHO uses group-equivariant feature learning on SO(3), avoiding reliance on an external local reference frame while targeting robustness to noise and density variation.YOHO-Desc is built from group features defined on the icosahedral group.
  • Rotation-invariant features find descriptor matches, while rotation-equivariant features estimate a plausible rotation from one matched point pair.The group-feature permutation supports rotation estimation, and pooling removes the permutation for rotation invariance.
  • YOHO achieves better or comparable registration accuracy on four benchmarks with only 1000 RANSAC iterations, whereas existing methods commonly use 50000.The paper attributes the reduced search to using estimated rotations in RANSAC.

2 RELATED WORK

Prior registration methods use handcrafted, PCA-based, or group-convolution descriptors, while other approaches jointly process point clouds or directly regress transformations. YOHO differs by using equivariant features to estimate rotations within registration.

  • Traditional and learning-based feature methods construct local descriptors, but PCA neighborhood alignment is not robust to noise or density variations.Handcrafted invariant features can also discard substantial information.
  • Group-convolution methods extract rotation-invariant descriptors, but the cited comparison says EPN does not use estimated rotations in RANSAC.YOHO uses those rotations for faster and more accurate partial-scan alignment.
  • Only a few methods estimate rotation from one descriptor pair; RelativeNet directly regresses rotation, whereas YOHO aligns icosahedral-group feature maps by permutation.The paper contrasts YOHO's approach with RelativeNet's reported generalization limitation on unseen data.
  • Direct registration methods jointly consider both point clouds to solve transformation parameters or condition descriptors for more accurate correspondences.
  • YOHO applies rotation-equivariant feature learning to general point-cloud registration, whereas related works mainly target rotation invariance for recognition or shape analysis.

3 METHOD

YOHO constructs rotation-invariant and rotation-equivariant descriptors on the icosahedral group, then uses correspondence-level rotation estimates to narrow transformation search in modified RANSAC.

  • Overview: YOHO extracts local descriptors from two unaligned point clouds, matches them into correspondences, and estimates coarse and refined rotations for each correspondence.The estimated rotations are subsequently used by modified RANSAC algorithms to identify correct transformations.
  • Preliminary: The icosahedral group is the largest discrete finite subgroup of SO(3), containing 60 rotations that preserve a regular icosahedron.YOHO defines group features and local convolution neighborhoods on this finite rotation group.
  • Descriptor construction: YOHO constructs group features by rotating each local neighborhood through icosahedral-group elements and processing the rotated sets with a shared backbone.The resulting group feature is stored as a 60 × n0 matrix, with rows corresponding to group rotations; PointNet, FCGF, or D3Feats can serve as the backbone.
  • Descriptor construction: Localized icosahedral group convolutions preserve rotation equivariance, representing input rotations as permutations of group-feature rows.The convolution uses a neighborhood set H and learns local patterns on the icosahedral group.
  • Descriptor construction: Average pooling over the final group feature produces a descriptor invariant to all rotations in the icosahedral group.The descriptor is not strictly invariant outside this group, although the authors state that the discrete invariance provides an inductive bias for learning broader invariance.
  • Rotation estimation: For each matched correspondence, YOHO finds a coarse rotation by testing 60 group-feature permutations, then regresses a residual to obtain a refined rotation.The residual regressor applies group convolutions, average pooling, and an MLP, and composes the residual with the coarse rotation.
  • Transformation estimation: Modified RANSAC restricts triplets to correspondences sharing a coarse rotation, while one-shot estimation directly computes a transformation from one correspondence and its refined rotation.With n correspondences, one-shot estimation yields n hypotheses instead of n × (n−1) × (n−2)/6 triplet hypotheses.

4 EXPERIMENTS

YOHO is evaluated across indoor and outdoor registration benchmarks, with ablations examining rotation invariance, rotation verification, group convolution, and one-shot estimation. It generally improves accuracy and efficiency with substantially fewer RANSAC iterations, while retaining limitations on planar overlaps and descriptor construction cost.

  • Experimental protocol: YOHO is evaluated on 3DMatch, 3DLoMatch, ETH, and WHU-TLS, using FMR, IR, and RR under established registration protocols.The experiments compare YOHO-C and YOHO-O with learning-based descriptors and the matcher Predator.
  • 3DMatch/3DLoMatch results: With 50 times fewer RANSAC iterations, YOHO outperforms all baseline methods on 3DMatch and 3DLoMatch.The reported comparison attributes the RR improvement mainly to rotation equivariance reducing the transformation search space.
  • ETH results: YOHO underperforms SpinNet under strict ETH thresholds but outperforms it in IR and RR under the looser setting.The strict setting uses τc=0.1m and τr=0.2m; the looser setting uses τc=0.2m and τr=0.5m.
  • ETH results: YOHO-C+ICP outperforms SpinNet+ICP at all reported thresholds, including the strictest τr=0.05m setting.ICP post-processing improves the accuracy limited by YOHO’s 0.15m FCGF voxel downsampling.
  • Ablation studies: Group-feature invariance improves robustness, CRV achieves higher RR with 50 times fewer iterations, and group convolution improves FMR by 4.9% and RR by 9.6%.YOHO-O adds refined rotations for one-shot transformation estimation and brings further RR improvements.
  • Analysis: YOHO finds true transformations in less than 400 iterations, and its total registration time remains shortest despite 60 backbone forward passes for descriptor extraction.The advantage is especially pronounced on 3DLoMatch, where baseline triplet sampling is disadvantaged by inlier ratios below 0.1.
  • Limitations: YOHO has two reported limitations: planar overlap can make rotation estimation ambiguous, and YOHO-Desc construction is less efficient because of 60 backbone forward passes.The paper suggests group simplification, SO(2) restrictions, or advanced equivariance learning as possible future optimizations.

5 CONCLUSION

YOHO combines rotation-invariant matching with rotation-equivariant rotation estimation for partial point-cloud registration. Its estimated rotations improve subsequent RANSAC robustness and accuracy while reducing required iterations.

  • YOHO uses a descriptor with both rotation invariance and rotation equivariance.The invariant component builds correspondences, while the equivariant component estimates a rotation for each correspondence pair.
  • Estimated rotations help RANSAC find correct transformations robustly and accurately.
  • YOHO achieves state-of-the-art performance across multiple datasets with much fewer RANSAC iterations.

A PROOF OF ROTATION EQUIVARIANCE

The proof establishes rotation equivariance by showing that rotating the input permutes group-feature rows, and that group convolution preserves this permutation structure through successive layers.

  • Rotating the input point set by an icosahedral-group element transforms its group feature through a corresponding group action.The feature is computed over the icosahedral group, whose closure supports the subsequent composition argument.
  • For the initial feature, rotation permutes rows because each row represents a different group rotation.The transformed feature is expressed as P_m ◦ f_0, where P_m is the permutation operator associated with m.
  • If f_k is rotation-equivariant, group convolution produces an output f_k+1 that is also rotation-equivariant.The proof uses the group-convolution definition and closure of the group operation.

B IMPLEMENTATION DETAILS

YOHO trains its backbone, group feature embedder, and rotation residual regressor sequentially, while evaluation uses official baseline implementations and correspondence-based inlier counting.

  • The backbone, Group Feature Embedder, and Rotation Residual Regressor are trained sequentially.The supplied training description specifies separate stages for the backbone and group feature embedder, with the regressor included in the sequential pipeline.
  • The backbone is trained for 80 epochs, followed by 10 epochs for the group feature embedder with batch size 32.The backbone uses rotation augmentation in [0°, 50°], whereas the embedder uses augmentation at all angles.
  • Baseline methods are evaluated using their official codes and models, with estimated correspondences used for inlier counting in both modified RANSAC procedures.

C COMPARISON WITH DIRECT REGISTRATION METHODS

The paper compares YOHO with direct partial-point-cloud registration methods on 3DMatch and 3DLoMatch, using successful-alignment rates and additional reported metric tables.

  • YOHO is compared with eight direct registration methods on the 3DMatch and 3DLoMatch datasets.The listed baselines are Go-ICP, FGR, TEASER, PointNetLK, 3DRegNet, DGR, DHV, and PointDSC.
  • Successful alignment is defined by rotation-error and translation-error thresholds between predicted and ground-truth transformations.The supplied passage begins specifying these thresholds but does not include their values.
  • Table 7 reports the comparison, while Tables 8 and 9 report backbone performance and YOHO-PN results under specified metric and iteration settings.Table 9 also defines the meanings of YOHO-C, YOHO-O, and total time cost.

D POINTNET AS BACKBONE

YOHO-PN uses a simple PointNet backbone while improving registration performance and efficiency across the reported comparisons.

  • Backbone and setup: YOHO-PN processes randomly sampled 1024-point local patches with a 10-layer PointNet backbone.Planar keypoints are filtered using a minimum PCA eigenvalue threshold of ≥0.03 to avoid rotation ambiguities.
  • Performance: YOHO-PN brings great improvements to the simple PointNet backbone and outperforms several baselines in registration recall.The reported baselines include methods [7] [31] [43].
  • Performance: In the strict ETH setting, YOHO-PN achieves better FMR/IR than YOHO-FCGF, while YOHO-PN-C achieves the best reported RR among the compared baselines.The strict setting uses τc = 0.1m and τr = 0.2m.
  • Efficiency: YOHO-PN costs less time than SpinNet and PerfectMatch despite using a local-patch descriptor.This comparison concerns other local patch-based descriptors.

E ROBUSTNESS ANALYSIS

YOHO-PN is evaluated under controlled density and noise variations, where its descriptor maintains matching robustness better than the compared rotation-invariant baselines.

  • Experimental setup: The robustness experiment uses 3000 synthesized 3DMatch patch pairs, each containing 4096 points within a 0.3m radius.Random rotations and different noise and density variation levels are applied to every point cloud.
  • Evaluation: Matching recall is measured as the ratio of correct patch pairs while density and noise levels vary.The compared methods are YOHO-PN, PCA+PointNet, PerfectMatch, and LMVD.
  • Findings: YOHO-Desc remains nearly unchanged or declines more slowly as density decreases and noise increases than the baselines.The authors interpret this as evidence for robustness from YOHO’s rotation-invariance construction.

F RESULTS ON THE WHU-TLS DATASET

On the unseen WHU-TLS dataset, YOHO is reported to generalize better than SpinNet under the specified evaluation settings.

  • WHU-TLS results: YOHO generalizes better than SpinNet to the unseen WHU-TLS dataset.The comparison is reported using the detailed results for different WHU-TLS scenes.
  • Evaluation settings: The WHU-TLS evaluation uses τc = 0.5m, τr = 1m, and 5000 keypoints.YOHO-PN uses a 6m patch radius, YOHO uses 0.8m downsampling voxels, and SpinNet is evaluated with a 6m patch radius.
  • Comparison with EPN: YOHO and EPN differ in that YOHO uses estimated rotations within modified RANSAC for partial-scan alignment, whereas EPN uses vanilla RANSAC.The paper identifies this as the key distinction between the methods.
Loading 2109.00182v2…