Source-linked AI summary
ProposalContrast: Unsupervised Pre-training for LiDAR-based 3D Object Detection
Junbo Yin, Dingfu Zhou, Liangjun Zhang, Jin Fang, Cheng-Zhong Xu, Jianbing Shen, Wenguan Wang
TL;DR
Prior point-cloud SSL contrasts whole scenes or individual points and voxels, although 3D detection calls for region-level representations. ProposalContrast encodes proposal geometry and jointly separates proposals and clusters. The paper reports evaluation across detectors and datasets, including pre-trained PV-RCNN outperforming a scratch model with half the annotations.
Problem
Existing point-cloud SSL approaches contrast whole scenes or points and voxels, while the paper identifies proposal-level representations as better suited to 3D detection.
Source: §1 · p. 2Method
ProposalContrast encodes geometric relations within region proposals and jointly optimizes inter-proposal discrimination and inter-cluster separation.
Source: §1 · p. 3, §3.2 · p. 7Results
With half the annotations, pre-trained PV-RCNN outperforms a scratch model trained with full annotations.
Source: §1 · p. 3Takeaways & Limitations
The reported annotation-efficiency result indicates that ProposalContrast can reduce the annotation burden for 3D detection.
Source: §1 · p. 3
Abstract
from arXiv · showhide
Existing approaches for unsupervised point cloud pre-training are constrained to either scene-level or point/voxel-level instance discrimination. Scene-level methods tend to lose local details that are crucial for recognizing the road objects, while point/voxel-level methods inherently suffer from limited receptive field that is incapable of perceiving large objects or context environments. Considering region-level representations are more suitable for 3D object detection, we devise a new unsupervised point cloud pre-training framework, called ProposalContrast, that learns robust 3D representations by contrasting region proposals. Specifically, with an exhaustive set of region proposals sampled from each point cloud, geometric point relations within each proposal are modeled for creating expressive proposal representations. To better accommodate 3D detection properties, ProposalContrast optimizes with both inter-cluster and inter-proposal separation, i.e., sharpening the discriminativeness of proposal representations across semantic classes and object instances. The generalizability and transferability of ProposalContrast are verified on various 3D detectors (i.e., PV-RCNN, CenterPoint, PointPillars and PointRCNN) and datasets (i.e., KITTI, Waymo and ONCE).
1 Introduction
Point-cloud SSL has used scene-level or point/voxel-level discrimination, but the paper argues proposal-level representations better match 3D detection. ProposalContrast models proposal geometry and jointly learns instance- and class-discriminative representations; its reported evaluations span multiple detectors and datasets.
1 Introduction
- Scene-level SSL struggles to describe road-object locality, while point/voxel-level methods emphasize fine details without object-level characteristics.The paper argues that proposal-level representations better suit 3D detection because objects span multiple points and voxels.Source: §1 · pp. 2–3
- ProposalContrast encodes point geometry within each region proposal and jointly optimizes instance discrimination and cluster-based class discrimination.Inter-proposal discrimination encourages instance-specific characteristics; inter-cluster separation encourages shared patterns that support semantic recognition.Source: §1 · p. 3
- With half the annotations, pre-trained PV-RCNN outperforms a scratch model trained with full annotations.The paper presents this as evidence that ProposalContrast can reduce annotation costs.Source: §1 · p. 3
- The paper evaluates generalizability across PV-RCNN, CenterPoint, PointPillars, and PointRCNN, and transferability across Waymo, KITTI, and ONCE.It also reports annotation savings for pre-trained PV-RCNN.Source: §1 · p. 3
2 Related Work
The related work describes trade-offs between grid-based and point-based detectors and limitations in prior point-cloud SSL. ProposalContrast instead contrasts automatically mined region proposals in self-driving scenes and discriminates between proposals and clusters.
2 Related Work
- Grid-based detectors are efficient but lose information through quantization, while point-based methods are computationally costly on large point clouds.Hybrid voxel- and point-based detectors aim to combine advantages of both approaches.Source: §2 · p. 3
- Prior scene-level SSL neglects object-centric structure, while point/voxel-level methods struggle to acquire object-level representations compatible with 3D detection.The paper also notes that several prior methods overlook semantic relations among instances.Source: §2 · p. 4
- Unlike indoor region-level work using synthetic CAD objects, ProposalContrast mines potential object instances from raw self-driving point clouds.It contrasts proposal instances and clusters to capture point-cloud properties relevant to 3D detection.Source: §2 · p. 4
3 Approach
ProposalContrast pre-trains on paired region proposals, encoding within-proposal geometry and jointly optimizing proposal-level instance discrimination and cluster-based class consistency. Its approach comprises proposal generation, attention-based encoding, and the two complementary training objectives.
3.1 Overview of ProposalContrast
- Paired augmented views are mapped to spherical proposals centered on diverse sampled points, then encoded and jointly optimized with inter-proposal discrimination and inter-cluster separation.The pipeline records point correspondences across views; spherical proposals are used instead of dense 3D bounding-box proposals.Source: §3.1 · pp. 5–6
3.2 Region Proposal Encoding Module
- Bilinear interpolation initializes proposal-point features; cross-attention uses the center feature as query, neighbor-feature differences as keys, and point coordinates for explicit position.Attention aggregates information from the proposal’s keys so the center becomes aware of informative neighborhoods.Source: §3.2 · p. 7
3.3 Joint Optimization of Inter-proposal Discrimination and Inter-cluster Separation
- Inter-proposal discrimination pulls representations of the same proposal across views together and pushes representations of different proposals apart.This proposal-level contrastive task targets instance-sensitive representations without bounding-box or semantic annotations during pre-training.Source: §3.3 · p. 8
- Inter-cluster separation groups proposals into pseudo-classes and maximizes agreement between cross-view cluster assignments; the overall loss combines it with inter-proposal discrimination.The two objectives target class-discriminative representations for recognition and instance-sensitive representations for localization.Source: §3.3 · pp. 7–9
4 Experimental Results
Experiments evaluate Waymo pre-training transferred across KITTI, Waymo, and ONCE, using multiple detectors and labeled-data budgets. Reported comparisons show gains over scratch and concurrent self-supervised methods, while ablations examine proposal encoding and joint objectives.
4.1 Pre-training Settings
- Pre-training uses the full unlabeled Waymo training set before fine-tuning; experiments span grid- and point-based backbones, with 2,048 proposals and 16 points per proposal.The reported Waymo training set contains 158,361 frames.Source: §4.1 · p. 9
4.2 Transfer Learning Settings and Results
- On KITTI, pre-training improves PointRCNN and PV-RCNN over scratch and concurrent methods; half-label models achieve performance comparable to full-label counterparts.The reported PV-RCNN comparison exceeds STRL by 1.46% and GCC-3D by 1.66%.Source: §4.2 · pp. 10–11, Table 1 · p. 10
- For PV-RCNN, ProposalContrast improves scratch by 3.05% APH and beats GCC-3D by 1.10% APH on average; CenterPoint gains 2.9% and 1.06% APH.Both detectors are evaluated with 20% of Waymo training samples.Source: §4.2 · p. 11, Table 2 · p. 11
- With 1% labeled data, pre-training gains are 8.26% and 16.95% APH across the two CenterPoint backbones.Improvements are reported for both PointPillars and VoxelNet under every label setting.Source: §4.2 · p. 12, Table 3 · p. 12
- On ONCE validation, the pre-trained CenterPoint improves its baseline by 2.00% mAP.The improved CenterPoint is reported to achieve the best performance among the compared state-of-the-art detectors.Source: Table 4 · p. 12, §4.2 · p. 13
4.3 Ablation Study
- Proposal-wise pre-training transfers better than reimplemented PointContrast and DepthContrast under the shared VoxelNet-backbone CenterPoint setup.The ablation evaluates transfer to 3D detection in large-scale LiDAR point clouds.Source: §4.3 · pp. 13–14
- Attentive proposal encoding adds +0.81 mAPH over MaxPool; IPD alone raises baseline 2.60% mAPH, removing SKC drops 2.44 mAPH, and joint learning improves further.The tested proposal setting is N = 2048 and r = 1.0 m.Source: §4.3 · p. 14
5 Conclusion
ProposalContrast is a proposal-wise pre-training framework for LiDAR-based 3D object detection that models proposal geometry and jointly optimizes inter-proposal discrimination and inter-cluster separation. Experiments across prevalent 3D detectors and datasets show its effectiveness.
5 Conclusion
- ProposalContrast jointly optimizes inter-proposal discrimination and inter-cluster separation to build comprehensive proposal representations.Its proposal encoding models intrinsic geometry by considering relationships among points inside proposals.Source: §5 · p. 14
- Its generation module samples dense, diverse spherical proposals with different augmented views.The framework also includes a region proposal encoding module that abstracts proposal features.Source: §5 · p. 14
- Extensive experiments across diverse prevalent 3D detectors and datasets show the model's effectiveness.Source: §5 · p. 14