PaperlayerView exact revision on arXiv

Source-linked AI summary

Improving Point Cloud Based Place Recognition with Ranking-based Loss and Large Batch Training

Jacek Komorowski

arXiv:2203.00972v2cs.CV

TL;DR

Place-recognition methods have focused heavily on increasingly complex architectures, while reported gains on a popular benchmark have stagnated. This paper combines an enhanced sparse-voxel 3D convolutional descriptor network with a modified differentiable Average Precision loss and large-batch training; it reports consistent gains over state of the art.

  • Problem

    Recent place-recognition research has focused largely on complex network architectures, while quantitative improvement between competitive methods has been limited.

    Source: §I · p. 1

  • Method

    The paper enhances a sparse-voxel 3D convolutional descriptor network with channel attention and trains it using Truncated Smooth-AP with multistage backpropagation for very large batches.

    Source: §I · p. 1, A. Network architecture · p. 2, 1) Loss function · p. 3

  • Results

    The method consistently improves over state of the art; on Oxford under the Refined evaluation protocol, Average Recall@1 is 96.9%, 1.9 p.p. above runner-up TransLoc3D.

    Source: 1) Comparison with state of the art · p. 4, §V · p. 5

  • Takeaways & Limitations

    The authors conclude that standard benchmarks are nearly saturated and call for larger datasets covering more diverse environments.

    Source: §V · p. 5

    Standard benchmarks are near their reported metric limits, constraining room for further improvement on those benchmarks and motivating larger, more diverse datasets.

    Source: §V · p. 5

Abstract

from arXiv · show

The paper presents a simple and effective learning-based method for computing a discriminative 3D point cloud descriptor for place recognition purposes. Recent state-of-the-art methods have relatively complex architectures such as multi-scale oyramid of point Transformers combined with a pyramid of feature aggregation modules. Our method uses a simple and efficient 3D convolutional feature extraction, based on a sparse voxelized representation, enhanced with channel attention blocks. We employ recent advances in image retrieval and propose a modified version of a loss function based on a differentiable average precision approximation. Such loss function requires training with very large batches for the best results. This is enabled by using multistaged backpropagation. Experimental evaluation on the popular benchmarks proves the effectiveness of our approach, with a consistent improvement over the state of the art

I. INTRODUCTION

The paper frames place recognition as retrieval of nearby point-cloud descriptors and addresses limited recent benchmark gains by improving both network architecture and training. It builds on sparse-voxel convolutional methods while comparing its approach with more complex Transformer-based designs.

I. INTRODUCTION

  • Place recognition is treated as instance retrieval: the query descriptor is matched to database descriptors, and retrieved geo-tagged clouds provide localization coordinates.The network computes a global descriptor for the query point cloud, and the database search returns the closest descriptors.Source: §I · p. 1, Figure 1 · p. 1
  • Recent gains had stagnated, partly because top results on the popular PointNetVLAD benchmark approach the reported metric limits.Earlier PointNet-like approaches – gave way to sparse-voxel 3D convolution and then Transformer variants –[8].Source: §I · p. 1
  • An enhanced MinkLoc3D network paired with ranking-based optimization and very large batches is reported to outperform contemporary, more complex architectures.The method combines a modified differentiable Average Precision loss with multistage backpropagation to train on batches of thousands of point clouds.Source: §I · p. 1

A. Point cloud-based place recognition using learned descriptors

  • MinkLoc3D uses sparse-voxel 3D convolutions and FPN feature extraction with GeM pooling, while recent methods add Transformer-based feature extraction or multi-scale aggregation, –[8].The paper investigates improvements to both the architecture and training process rather than focusing on architecture alone.Source: A. Point cloud-based place recognition… · p. 2

B. Loss function in the retrieval task

Retrieval metrics such as Average Recall are not directly differentiable, so training commonly uses surrogate losses. The paper motivates differentiable ranking losses and multistage backpropagation for large-batch training.

B. Loss function in the retrieval task

  • Average Recall is non-differentiable and cannot be directly optimized with stochastic gradient descent, creating a mismatch between training losses and retrieval evaluation.Prior point-cloud place-recognition work used lazy quadruplet or triplet losses, while surveys found the advantage of more sophisticated losses to be limited and dataset-dependent.Source: B. Loss function in the retrieval task · p. 2
  • Differentiable ranking-metric approximations require batches comparable in size to evaluation sets, but thousands of high-resolution inputs can exceed GPU memory constraints.Multistage backpropagation computes descriptor-level loss gradients separately, then applies the chain rule to obtain network-weight gradients; each batch element passes through the network twice, increasing training time.Source: B. Loss function in the retrieval task · p. 2
  • The paper proposes Truncated SoftAP, a modified differentiable Average Precision loss, combined with multistage backpropagation to train place-recognition descriptors on very large batches.The passage reports that this combination leads to state-of-the-art results.Source: B. Loss function in the retrieval task · p. 2

III. METHOD DESCRIPTION

The method localizes a query point cloud by comparing its descriptor with descriptors in a database of geo-tagged point clouds.

III. METHOD DESCRIPTION

  • Place recognition searches a geo-tagged point-cloud database for descriptors closest to the query descriptor in Euclidean distance.The method section presents this as the descriptor-based route to localization.Source: §III · p. 2
  • The returned database point clouds have known coordinates that can support localization of the query.Localization is inferred from the retrieved matches.Source: §III · p. 2
  • The method section illustrates the descriptor-matching procedure with Fig. 1.The figure depicts searching for database point clouds with descriptors closest to the query descriptor.Source: §III · p. 2

A. Network architecture

The descriptor network enhances MinkLoc3D with a wider, deeper sparse 3D convolutional architecture and ECA channel-attention layers. Its FPN-style feature extraction combines bottom-up and top-down features before GeM pooling produces the descriptor.

A. Network architecture

  • The architecture extends MinkLoc3D by adding convolutional and transposed-convolution blocks, increasing channel counts, and incorporating ECA channel-attention layers,.ECA layers were originally proposed for 2D convolutional networks.Source: A. Network architecture · p. 2
  • The input point cloud is quantized into a single-channel sparse tensor, then processed by five bottom-up convolutional blocks with increasing receptive fields.Starting from Conv1, each block reduces spatial resolution by two; the FPN design produces feature maps with increased receptive fields to capture higher-level semantic information.Source: A. Network architecture · p. 2, Figure 2 · p. 3
  • Two transposed-convolution layers upsample higher-level features and add them to corresponding bottom-up features, with 1x1x1 convolutions unifying channel counts before merging.The architecture overview labels convolutional blocks, transposed convolutions, and 1Conv layers, and gives stride/channel pairs for blocks.Source: A. Network architecture · p. 2, Figure 2 · p. 3
  • The resulting feature map is aggregated by generalized-mean pooling to produce descriptor D [15].The architecture table identifies BN as batch normalization, ECA as Efficient Channel Attention, and [...] as a residual block.Source: A. Network architecture · p. 3, Table I · p. 3

B. Network training

The method modifies Smooth-AP for place recognition by optimizing only the top-k positives, rather than ranking every positive. The batch loss uses a smooth approximation of average precision based on descriptor distances.

1) Loss function:

  • Truncated Smooth-AP maximizes average precision over only the top-k positives, since place recognition needs only a small number of correct top-ranked candidates.Training positives are point clouds acquired within 10 m of the query; negatives are acquired more than 50 m away.Source: 1) Loss function · p. 3
  • The modified loss computes N × k × N similarity matrices instead of Smooth-AP’s N × P × N matrices, where k ≪ P, reducing computation and allowing larger batches.The paper reports that the larger batches further improve accuracy.Source: 1) Loss function · p. 3
  • For each query, the smooth average-precision approximation compares a positive’s soft rank among the top-k positives with its rank among all positives and negatives.The approximation uses descriptor distances and a differentiable function G; the batch loss averages the query-level results.Source: 1) Loss function · p. 3
  • The loss is minimized with stochastic gradient descent using the Adam optimizer.The batch loss is computed across m training elements.Source: 1) Loss function · p. 3

2) Training with large batches:

Multistaged backpropagation permits computing the ranking-based loss over large batches without processing all point clouds with gradients simultaneously. It recalculates descriptors with gradients enabled and accumulates their contributions before updating network weights.

2) Training with large batches:

  • Multistaged backpropagation addresses GPU-memory limits that prevent direct processing of the large batches needed for ranking-based losses.The paper says these losses work best with batches comparable in size to the evaluation set.Source: 2) Training with large batches · p. 3
  • The algorithm first computes each batch element’s descriptor without gradients, then computes the loss and its gradient with respect to all descriptors.It next recalculates each descriptor with gradients enabled and uses the chain rule to obtain gradients with respect to network weights.Source: 2) Training with large batches · p. 3
  • Gradients are accumulated one batch element at a time before the network weights are updated.The chain rule combines the earlier descriptor-level computations to obtain weight gradients.Source: 2) Training with large batches · pp. 3–4

3) Implementation details:

Implementation uses 0.01 coordinate quantization and 256-dimensional global descriptors. The paper also reports data augmentation and the hardware and software used for experiments.

3) Implementation details:

  • A 0.01 coordinate quantization step yields 200 voxels per spatial coordinate for points normalized to [−1, 1].The global descriptor has 256 dimensions, matching the other evaluated methods.Source: 3) Implementation details · p. 4
  • Training augmentation applies random jitter, small random translations, and removal of up to 10% of randomly chosen points.Jitter is drawn from N (0; 0.001), and translation from U (0, 0.01).Source: 3) Implementation details · p. 4
  • Experiments ran on a single RTX 2080Ti GPU with a 12-core Ryzen Threadripper 1920X and 64 GB RAM, using PyTorch 1.10 and MinkowskiEngine 0.5.4.The training parameters for the baseline and refined protocols are listed in Table II.Source: 3) Implementation details · p. 4, Table II · p. 4

IV. EXPERIMENTAL RESULTS

The evaluation uses outdoor LiDAR place-recognition datasets and baseline and refined training protocols. It measures retrieval-based localization using Average Recall at rank 1 and at 1% of database size.

A. Datasets and evaluation methodology

  • The benchmark includes Oxford RobotCar and three in-house areas, with LiDAR point clouds collected along repeated routes; clouds are ground-plane filtered and downsampled to 4096 points.Oxford scans come from a 2D LiDAR, while the in-house datasets use a 3D LiDAR.Source: A. Datasets and evaluation methodology · p. 4
  • The baseline protocol trains on Oxford’s training split, while the refined protocol trains on Oxford and in-house training splits.The refined protocol uses a larger and more diverse training set; Table III presents training and test split counts.Source: A. Datasets and evaluation methodology · p. 4, Table III · p. 4
  • A query is localized successfully when at least one of its top-N retrieved clouds lies within 25 m of its ground-truth position.The reported measures are Average Recall@1 and Average Recall@1%, with the latter using the top 1% of the database.Source: A. Datasets and evaluation methodology · p. 4

B. Results and discussion

The descriptor consistently improves on state of the art across the reported datasets and metrics. The refined protocol shows larger gains, while the figures illustrate nearest-neighbour retrievals and failure cases.

B. Results and discussion

1) Comparison with state of the art.:

  • Under the Baseline protocol, Average Recall@1 reaches 96.3% on Oxford, a 1.3 p.p. gain, and improves by 0.8 to 2.2 p.p. across the three in-house datasets.Training uses only the Oxford training split; the authors note a performance drop on in-house datasets and attribute it to different LiDAR characteristics.Source: 1) Comparison with state of the art · p. 4
  • 96.9% Average Recall@1 on Oxford under the Refined protocol outperforms runner-up TransLoc3D by 1.9 p.p.; on the three in-house datasets, the method outperforms TransLoc3D by 1.0 to 2.8 p.p.Source: 1) Comparison with state of the art · pp. 4–5
  • 99.3% average AR@1% across all four datasets under the Refined protocol is a 0.4% improvement over runner-up TransLoc3D.Oxford and Business District each reach 99.1%, with gains of 0.6 p.p. and 2.7 p.p., respectively.Source: 1) Comparison with state of the art · p. 5

2) Ablation study:

The ablation study finds that architecture and training improvements each raise mean AR@1, and their combination yields the largest gain over the baseline.

2) Ablation study:

  • 2.1 p.p. gain raises mean AR@1 to 96.6% by improving the network architecture without changing the loss.Source: 2) Ablation study · p. 5
  • 3.4 p.p. overall gain over the baseline reaches 97.9% AR@1 when improved architecture and training are combined.Source: 2) Ablation study · p. 5
  • 2.5 p.p. gain raises baseline-architecture AR@1 to 97.0% with TruncatedSoftAP, large batches, and multistage backpropagation.Source: 2) Ablation study · p. 5
  • TruncatedSoftAP performs best with 2048-element batches; using 256-element batches yields AR@1 about 1 p.p. lower.Source: 2) Ablation study · p. 5

V. CONCLUSION

The paper presents a point-cloud descriptor using a relatively simple 3D convolutional architecture and an improved training procedure. It reports consistent state-of-the-art gains, while concluding that standard benchmarks are saturated and further progress needs broader datasets.

V. CONCLUSION

  • The method computes a discriminative point-cloud descriptor for place recognition with a relatively simple 3D convolutional architecture and improved training procedure.Source: §V · p. 5
  • Standard benchmarks are saturated, with little room for improvement; the authors call for larger-area datasets spanning more diverse environments to support further progress.The reported Refined-protocol AR@1 values range from 96.9% to 99.4%.Source: §V · p. 5

Related papers

Loading 2203.00972v2…