Source-linked AI summary
CoFiNet: Reliable Coarse-to-fine Correspondences for Robust Point Cloud Registration
Hao Yu, Fu Li, Mahdi Saleh, Benjamin Busam, Slobodan Ilic
TL;DR
Point-cloud registration needs reliable correspondences, but sparse keypoints can be difficult to repeat consistently. CoFiNet replaces keypoint detection with coarse-to-fine node-to-point matching and reports superior benchmark performance, especially in low-overlap settings, while using fewer parameters.
Problem
Sparse keypoint-based correspondence retrieval struggles to guarantee repeatability when extracting reliable matches for point-cloud registration.
Method
CoFiNet hierarchically matches uniformly down-sampled nodes using overlap-based weighting, then expands proposals into patches and refines point matches with density-adaptive optimal transport.
Results
CoFiNet outperforms existing methods on indoor and outdoor benchmarks, with especially strong gains on low-overlap 3DLoMatch and substantially fewer parameters than a state-of-the-art comparison.
Takeaways & Limitations
Detection-free coarse-to-fine correspondence extraction provides a robust alternative to keypoint-based matching for unordered and irregular point clouds.
Takeaways & Limitations
CoFiNet lacks explicit coarse-scale outlier rejection, struggles in non-distinctive regions, and produces point correspondences that may be insufficiently sparse for RANSAC.
Abstract
from arXiv · showhide
We study the problem of extracting correspondences between a pair of point clouds for registration. For correspondence retrieval, existing works benefit from matching sparse keypoints detected from dense points but usually struggle to guarantee their repeatability. To address this issue, we present CoFiNet - Coarse-to-Fine Network which extracts hierarchical correspondences from coarse to fine without keypoint detection. On a coarse scale and guided by a weighting scheme, our model firstly learns to match down-sampled nodes whose vicinity points share more overlap, which significantly shrinks the search space of a consecutive stage. On a finer scale, node proposals are consecutively expanded to patches that consist of groups of points together with associated descriptors. Point correspondences are then refined from the overlap areas of corresponding patches, by a density-adaptive matching module capable to deal with varying point density. Extensive evaluation of CoFiNet on both indoor and outdoor standard benchmarks shows our superiority over existing methods. Especially on 3DLoMatch where point clouds share less overlap, CoFiNet significantly outperforms state-of-the-art approaches by at least 5% on Registration Recall, with at most two-third of their parameters.
1 Introduction
CoFiNet addresses unreliable keypoint repeatability by extracting registration correspondences hierarchically, from coarse nodes to fine points, without keypoint detection. Its weighting and density-adaptive refinement mechanisms target low-overlap and irregular point-cloud matching.
- Reliable correspondences are central to point-cloud registration and other vision tasks, but unordered, irregular point clouds make correspondence extraction challenging.
- Two-stage registration methods commonly match descriptors on sparse nodes before estimating pose, but keypoint sparsity can undermine repeatability.
- Coarse-to-fine matching avoids direct keypoint detection, addressing a gap in 3D point-cloud methods where keypoint detectors often perform poorly.
- CoFiNet weights coarse node matching by local overlap ratios, prioritizing nodes with more-overlapping vicinities and shrinking the refinement search space.
- The framework refines coarse proposals to point correspondences with a density-adaptive optimal-transport module robust to varying point density.
- CoFiNet is presented as detection-free and uses substantially fewer parameters than a state-of-the-art comparison method on indoor and outdoor benchmarks.It uses around two-third and one-fourth of [12]'s parameters on indoor and outdoor benchmarks, respectively.
2 Related Work
Related work spans learned local descriptors, learned 3D keypoint detectors, and coarse-to-fine correspondence methods developed primarily for 2D matching.
- Learned local descriptors: Learned descriptor methods encode local 3D patches or raw points to support correspondence search.
- Learned 3D keypoint detectors: Learned 3D keypoint detectors estimate salient points or jointly perform detection and description, but reported issues include degeneration and weak low-overlap robustness.
- Coarse-to-fine correspondences: 2D methods such as DRC-Net, Patch2Pix, and LoFTR establish correspondences progressively from coarse regions or patches to finer points.
3 Methodology
CoFiNet takes unordered point clouds, proposes coarse node correspondences, expands them into patches, and refines them to point-level matches using density-aware optimal transport.
- The method extracts point correspondences first and uses them with RANSAC to estimate the rigid transformation between two unordered point sets.
- Coarse-scale Matching: Shared KPConv encoders down-sample raw points into uniformly distributed nodes and jointly learn associated features for coarse matching.
- Coarse-scale Matching: Self- and cross-attention aggregate global context across nodes before strengthened features produce a similarity matrix and coarse correspondence proposals.
- Coarse-scale Matching: Slack rows and columns allow nodes without sufficiently overlapping counterparts to remain unmatched, while Sinkhorn optimization converts similarities into matching confidences.
- Point-level Refinement: Coarse node correspondences are expanded into point patches by assigning points to their nearest nodes in geometric space.
- Point-level Refinement: The density-adaptive module handles repeated or occluded patch points by solving an optimal transport problem rather than biasing matches toward slack entries.
- Training combines coarse- and fine-scale losses as L = Lc + λLf, balancing supervision at both correspondence levels.
- Coarse supervision weights node pairs by mutual local overlap ratios, including slack weights for unmatched regions.
X Y( eGP
The fine-scale loss supervises confidence predictions for valid point pairs while suppressing repeated samples and the artificial slack entry.
- Repeatedly sampled points and the slack entry are assigned zero supervision weight, preventing their training influence on fine-scale matching.
- The fine-scale objective weights log confidence scores by the correspondence target matrix, with 0 · log(0) defined as 0.
4 Results
CoFiNet is evaluated on indoor 3DMatch and 3DLoMatch benchmarks using correspondence-quality, registration, and ablation analyses. It shows particularly strong performance in low-overlap settings while using fewer parameters than competing methods.
- 4.1 3DMatch and 3DLoMatch: CoFiNet is evaluated on 3DMatch, 3DLoMatch, and odometryKITTI, covering indoor scenes with over 30% or 10%–30% overlap and outdoor odometry.The experiments compare indoor and outdoor benchmark scenarios.
- 4.1 3DMatch and 3DLoMatch: CoFiNet outperforms most prior methods on Inlier Ratio and performs best on both datasets when sampling 250 correspondences.The authors attribute this robustness to avoiding keypoint detection and report that fewer sampled correspondences yield higher Inlier Ratio.
- 4.1 3DMatch and 3DLoMatch: CoFiNet produces more reliable correspondences than PREDATOR on both 3DMatch and 3DLoMatch when poses are solved directly by SVD without RANSAC.FCGF and D3Feat fail on 3DLoMatch in this evaluation, where lower overlap makes reliable correspondences harder to obtain.
- 4.1 3DMatch and 3DLoMatch: CoFiNet significantly outperforms other methods on Feature Matching Recall, with a margin exceeding 4% on low-overlap 3DLoMatch.The result indicates that CoFiNet finds at least 5% inlier correspondences for more test cases in that setting.
- 4.1 3DMatch and 3DLoMatch: On Registration Recall, CoFiNet slightly trails PREDATOR on 3DMatch but surpasses all competitors on 3DLoMatch while using the fewest parameters.Registration Recall is identified as the metric that better reflects final registration performance.
- 4.2 Ablation Study: Ablations show that refinement and the overlap-proportional weighting scheme improve performance, while retaining at least a minimum number of coarse correspondences raises Registration Recall despite slightly lowering Inlier Ratio.Using only coarse correspondences performs worse on all metrics, and the minimum-count strategy supplies more correspondences for RANSAC.
5 Conclusion
CoFiNet extracts reliable point-cloud correspondences through a keypoint-free coarse-to-fine strategy. It uses overlap-guided coarse matching and density-adaptive refinement, matching or surpassing existing methods across indoor and outdoor benchmarks.
- CoFiNet directly consumes unordered, irregularly sampled point sets and proposes correspondences without keypoint detection.
- At the coarse level, overlap-proportional weighting matches nodes with overlapping vicinity areas and shrinks the subsequent refinement search space.
- At the fine level, density-adaptive matching handles varying point density and mitigates repeated-sampling effects.
- CoFiNet stays on par with state-of-the-art approaches on 3DMatch and KITTI while surpassing them on 3DLoMatch with significantly fewer parameters.
A Appendix
The appendix provides implementation, evaluation, dataset, efficiency, limitation, broader-impact, and qualitative-registration details.
- The supplementary material covers network architectures, evaluation metrics, implementation details, datasets, inference time, limitations, broader impact, and qualitative registration results.
A.1 Network Architectures
CoFiNet uses a KPConv-based encoder-decoder with additional attention networks for context aggregation. Its coarse-to-fine design supports a lightweight encoder and substantially reduces parameters relative to [12].
- CoFiNet mainly uses an encoder-decoder architecture based on KPConv operations, supplemented by two attention-based networks for context aggregation.
- Compared with [12], the design reduces parameters by around 2M on 3DMatch/3DLoMatch and over 20M on KITTI.
A.2 Evaluation Metrics
The evaluation uses correspondence-quality metrics and registration performance metrics, with formulas defining their dataset-level aggregation. The architecture passage also specifies four-head attention modules.
- Evaluation Metrics: Inlier Ratio measures the fraction of predicted correspondences whose residual under the ground-truth transformation is below a threshold.
- Evaluation Metrics: Feature Matching Recall measures the fraction of point-cloud pairs whose Inlier Ratio exceeds τ2 = 5%, indicating recoverability by robust pose estimation.
- Evaluation Metrics: Registration Recall measures the fraction of point-cloud pairs whose registration RMSE is below τ3 = 0.2m.
- Evaluation Protocol: The evaluation follows the original 3DMatch protocol, excluding immediately adjacent point clouds with very high overlap ratios.
- Network Architecture: Self- and cross-attention modules use four heads in their multi-head attention components.
A.3 Implementation Details
CoFiNet is implemented and trained end-to-end with a compact training setup on a single GPU.
- CoFiNet is implemented in PyTorch and trained end-to-end on a single RTX 2080Ti GPU.
- Training runs for 20 epochs on 3DMatch/3DLoMatch and KITTI using Adam with an initial learning rate of 3e-4.
- The learning rate is exponentially decayed by 0.05 after each epoch, with batch size 1 in all experiments.
- The encoder and decoder architectures resemble [12] but use significantly fewer parameters.
A.4 Data
The evaluation uses indoor 3DMatch variants and outdoor KITTI data, with established training, testing, and pose-refinement protocols.
- 3DMatch and 3DLoMatch: 3DMatch contains 62 scenes, split into 46 training, 8 validation, and 8 testing scenes.
- 3DMatch and 3DLoMatch: 3DLoMatch additionally includes point-cloud pairs with overlaps between 10% and 30%, unlike original 3DMatch pairs with over 30% overlap.
- OdometryKITTI: KITTI provides 1,358 training pairs, 180 validation pairs, and 555 testing pairs from 11 driving sequences scanned with a Velodyne HDL-64 laser scanner.
- Runtime evaluation: Model-runtime results are averaged over 1,623 point-cloud pairs from the 3DMatch testing set, including time to write results to disk.
A.5 Timings
CoFiNet is evaluated for inference and registration behavior through runtime comparisons, qualitative correspondence visualizations, and estimated-versus-ground-truth alignments.
- Runtime comparisons: CoFiNet reduces RANSAC runtime significantly compared with PREDATOR, especially when more correspondences are used for pose estimation.
- Runtime comparisons: CoFiNet directly proposes point correspondences, whereas PREDATOR outputs dense descriptors and extracts correspondences during RANSAC.
- Limitations: CoFiNet has no explicit coarse-scale outlier rejection, so false coarse correspondences can expand into false point correspondences and lower fine-level Inlier Ratio.
- Limitations: The method struggles in non-distinctive regions and produces correspondences that are less sparse, potentially affecting RANSAC-based registration.
- Correspondence visualization: Figure 6 visualizes 250 correspondences in selected columns, coloring correct matches green and incorrect matches red.
- Qualitative registration: Relative poses in the qualitative registration examples are estimated by RANSAC from CoFiNet-extracted correspondences.
- Qualitative registration: Figure 7 compares input point-cloud pairs, estimated registration, and ground-truth alignment across two examples per dataset.