Source-linked AI summary
MinkLoc3D: Point Cloud Based Large-Scale Place Recognition
Jacek Komorowski
TL;DR
Large-scale 3D place recognition needs discriminative descriptors, but PointNet-based pipelines have limited ability to capture local geometric structures. MinkLoc3D uses sparse voxelization, sparse convolutions, an FPN-based extractor, and Generalized-Mean pooling, achieving strong benchmark results with a simpler and more efficient model. The authors identify benchmark saturation and the need for larger, more diverse datasets, while leaving full 6DoF localization as future work.
Problem
Existing point cloud place-recognition methods rely on PointNet-based local features that are weak at capturing local geometric structures, motivating an alternative representation and architecture.
Method
MinkLoc3D uses a sparse voxelized point cloud representation, sparse convolutions with an FPN-based local feature extractor, and Generalized-Mean pooling for global descriptors.
Results
MinkLoc3D outperforms prior cloud-based place-recognition methods, with AR@1% 0.8–2.9 p.p. higher than LPD-Net across evaluation subsets and 5–18 p.p. higher than other methods.
Takeaways & Limitations
Sparse voxelized representations and sparse convolutions show potential for efficiently extracting discriminative 3D point cloud features, while simpler pooling can support a lightweight model.
Takeaways & Limitations
The standard benchmarks are close to saturation, so further progress requires larger and more diverse datasets; full 6DoF localization remains future work.
Abstract
from arXiv · showhide
The paper presents a learning-based method for computing a discriminative 3D point cloud descriptor for place recognition purposes. Existing methods, such as PointNetVLAD, are based on unordered point cloud representation. They use PointNet as the first processing step to extract local features, which are later aggregated into a global descriptor. The PointNet architecture is not well suited to capture local geometric structures. Thus, state-of-the-art methods enhance vanilla PointNet architecture by adding different mechanism to capture local contextual information, such as graph convolutional networks or using hand-crafted features. We present an alternative approach, dubbed MinkLoc3D, to compute a discriminative 3D point cloud descriptor, based on a sparse voxelized point cloud representation and sparse 3D convolutions. The proposed method has a simple and efficient architecture. Evaluation on standard benchmarks proves that MinkLoc3D outperforms current state-of-the-art. Our code is publicly available on the project website: https://github.com/jac99/MinkLoc3D
1. Introduction
The paper targets discriminative, low-dimensional 3D point cloud descriptors for place recognition, where query descriptors are matched against geo-tagged database clouds. MinkLoc3D replaces increasingly complex unordered-point approaches with sparse voxelization, sparse convolutions, and simple feature aggregation, achieving strong benchmark performance.
- Place recognition localizes a query point cloud by searching geo-tagged database clouds with the closest descriptors.
- The method is positioned for applications including robotics, autonomous driving, and augmented reality.
- PointNetVLAD extracts local features with PointNet and aggregates them with NetVLAD, but PointNet is not well suited to informative local features in large, complex scenes.
- Increasing complexity among unordered-point methods motivates investigating an alternate point cloud representation and network architecture.
- MinkLoc3D quantizes input clouds into a sparse voxelized representation, uses an FPN-based local feature network, and applies Generalized-Mean pooling to form a global descriptor.
- MinkLoc3D achieves state-of-the-art results on standard 3D place recognition benchmarks, outperforming PointNetVLAD by a large margin and improving over LPD-Net with simpler, more computationally effective architecture.
2. Related work
Prior 3D point cloud recognition methods commonly use unordered point processing, but PointNet-based local features miss local geometric structure. Related approaches add hierarchical, attention, graph, handcrafted, or multi-level contextual mechanisms, while sparse voxelization offers convolutional alternatives.
- Volumetric representations support 3D convolutions but become inefficient because memory requirements grow cubically with spatial resolution.
- PointNet processes each point independently and uses symmetric max pooling, preserving permutation independence but limiting capture of local geometric structures and fine-grained patterns.
- PointNet++ addresses PointNet’s limitations through hierarchical processing.
- Sparse voxelized representations enable 3D convolutions to capture local structures, while Minkowski Engine 2 improves sparse-convolution efficiency through coordinate hashing.
- PointNetVLAD combines PointNet local features with NetVLAD aggregation, while later methods add mechanisms for extracting local contextual information.
- PCAN uses multiscale sampling, grouping, and attention; DAGC uses dynamic graph convolutions and dual attention for multiscale contextual aggregation.
- LPD-Net combines ten handcrafted point features with PointNet and graph neural networks before NetVLAD aggregation.
- DH3D jointly performs global place recognition and local 6DoF pose refinement using FlexConv, squeeze-and-excitation blocks, and NetVLAD.
3. MinkLoc3D: global point cloud descriptor for place recognition
MinkLoc3D computes a global point-cloud descriptor by quantizing input coordinates into a sparse tensor, extracting local features with a 3D FPN, and aggregating them with GeM pooling. Its training uses metric learning with efficient batch-hard mining, dynamic batch sizing, and data augmentation.
- Network architecture: MinkLoc3D transforms the input point cloud into a sparse single-channel tensor before local feature extraction.Non-empty voxels are represented in the sparse tensor and passed to the local feature extraction network.
- Feature aggregation: Generalized-mean pooling converts the sparse local feature map into a global descriptor vector.GeM generalizes global max and average pooling, with a learnable pooling parameter p.
- Network architecture: A MinkowskiNet-inspired 3D FPN extracts local features through bottom-up convolutional blocks and top-down upsampling with lateral skip connections.The bottom-up path reduces spatial resolution while increasing receptive fields; the top-down path concatenates upsampled and skipped features.
- Network training: Triplet margin loss trains embeddings by separating anchor-positive pairs from anchor-negative pairs by a margin.Distances use the Euclidean norm, and optimization uses stochastic gradient descent with Adam.
- Network training: Batch-hard negative mining replaces prior candidate-heavy triplet construction with batches containing structurally similar pairs and similarity masks.The method uses hash-based indexing to identify structurally similar, dissimilar, or indefinite pairs.
- Network training: Dynamic batch sizing increases the batch when the proportion of active triplets falls below a threshold, while augmentation reduces overfitting.Augmentation includes jitter, translation, random point removal, and random erasing.
4. Experimental results
Experiments evaluate MinkLoc3D on multiple LiDAR datasets under baseline and refined training scenarios using Average Recall metrics. MinkLoc3D achieves strong benchmark performance, faster inference than LPD-Net, and robustness under challenging visual-localization conditions.
- Datasets and evaluation methodology: The evaluation uses modified Oxford RobotCar and three in-house LiDAR datasets, with baseline and refined training scenarios.The refined scenario trains on Oxford and in-house data, whereas the baseline trains only on Oxford data.
- Datasets and evaluation methodology: Queries are localized when one of the top N retrieved clouds lies within 25 meters of ground truth, reported as AR@1 and AR@1%.Recall@N is the percentage of correctly localized queries.
- Results and discussion: 3.0 p.p. higher AR@1% than LPD-Net on Oxford, while remaining comparable across in-house datasets despite differing LiDAR characteristics.MinkLoc3D is 1.0 and 0.6 p.p. worse on U.S. and B.D., respectively, but 0.7 p.p. better on R.A.
- Results and discussion: 0.8–2.9 p.p. higher AR@1% than LPD-Net across all refined-dataset evaluation subsets, and 5–18 p.p. higher than other methods.Average Recall plots show MinkLoc3D outperforming previous methods on every evaluation subset.
- Results and discussion: 22 ms inference time versus 26 ms for LPD-Net, while MinkLoc3D uses only 1.5 million trainable parameters.LPD-Net also requires preprocessing to compute ten handcrafted features.
- Results and discussion: GeM pooling produces the most discriminative descriptors and best generalization, while more parameterized NetVLAD variants perform worse on in-house data.The paper attributes this to overfitting risk from millions of NetVLAD parameters and effective local features from the extraction network.
- Results and discussion: AR@1% remains between 97.3 and 98.0% on Oxford and 90.3 and 93.2% on in-house datasets for descriptor sizes from 64 to 512.Performance deteriorates when the descriptor size falls to 32.
- Results and discussion: On RobotCar Seasons, image-based methods perform better in daytime, whereas MinkLoc3D surpasses all image-based methods except NetVLAD+SP+SG at night.The comparison uses small, downsampled 4096-point clouds and nearest-neighbor poses for the LiDAR methods.
5. Conclusion
MinkLoc3D advances 3D point-cloud place recognition with sparse convolutions and a 3D FPN, while its benchmarks indicate limited remaining headroom for standard evaluation datasets.
- MinkLoc3D is a 3D point-cloud descriptor based on sparse voxelization and a 3D FPN architecture.
- Sparse convolutional architecture produces informative local features for constructing a discriminative global descriptor.
- Improved training enables efficient and effective learning with larger batch sizes, supporting descriptor discriminability and generalization.
- The method advances prior point-cloud place-recognition methods through extensive experimental evaluation.
- AR@1% between 96.7% and 99.4% on the Refined Dataset indicates that standard benchmarks are close to saturation.The paper states that larger and more diverse datasets are needed for further progress.