Weakly Supervised 3D Object Detection from Lidar Point Cloud

1 Introduction

• Precisely labeled 3D data is costly to create, taking hundreds of hours to annotate one hour of driving data, and remains limited in size and variety.The paper identifies the demand for massive, high-quality labeled data as a major challenge for 3D object detection.Source: §1 · p. 2
• Stage 1 predicts horizontal object centers and foreground points from center-clicked BEV maps, then generates cylindrical proposals; Stage 2 predicts 3D parameters and confidence scores from those proposals.Stage 2 uses a few precisely annotated object instances, rather than exhaustively labeled scenes.Source: §1 · p. 2
• Center-click supervision is about 40–50× faster than cuboid labeling, and precisely labeled instances cover 25% of objects in weakly labeled scenes.That precise-instance supervision is about 3% of the supervision used in current leading models.Source: §1 · pp. 2–3
• The method is also designed as an annotation tool: automatically generated KITTI annotations let retrained PointPillars and PointRCNN retain more than 94% of original performance.With active center-click guidance, the retrained detectors reach above 96% of original performance.Source: §1 · p. 3
• 85−95% of fully supervised state-of-the-arts is achieved on KITTI Car using 500 weakly annotated scenes and 534 precisely labeled vehicle instances.The cited fully supervised methods use 3,712 precisely annotated scenes and 15,654 vehicle instances.Source: §1 · p. 3

2 Related Work

• Fig. 2 presents KITTI Car moderate-regime performance against annotation effort and is described as showing promising results with far less annotation demand.The figure caption refers readers to §5.2 for the results.Source: Figure 2 · p. 3

Learning Point Cloud Representations

• Voxel methods use volumetric grids and CNNs but can lose information through coarse quantization, whereas point-based methods operate on raw clouds and preserve original information.The paper notes that point-based methods had recently become popular.Source: §2 · pp. 3–4

3D Object Detection

• 3D detection work includes image-based, point-cloud-based, and sensor-fusion methods, with depth estimation, limited receptive fields, or slower multimodal processing noted as challenges.Point-cloud approaches include BEV projection, voxel CNNs, and direct point-wise features.Source: §2 · p. 4

Click Supervision

• Prior click-supervision work addressed 2D segmentation and detection, while this paper explores center clicks on BEV maps for 3D object detection..Source: §2 · p. 4

3D Object Annotation

• Earlier 3D annotation tools used 2D seeds for segment and centroid inference or differentiable template matching; this work differs in its weak supervision and dual-work mode.The cited prior approaches are and [14], respectively.Source: §2 · p. 4

3 Data Annotation Strategy for Our Weak Supervision

• Precise labeling requires navigating a 3D scene, placing a rough cuboid and orientation arrow, then adjusting projected 2D boxes across orthographic views.The paper describes the view switches and gradual corrections as laborious and expensive.Source: §3 · pp. 4–5
• Weak BEV center clicks paired with a few precisely labeled 3D instances provide the training supervision; most clicks can be completed in two actions without using a 3D view.For weak annotation, annotators click roughly in the camera front view and then refine the center on a zoomed BEV map.Source: §3 · p. 5
• Click supervision reduces the time required for traditional precise annotation by 15∼45×.The paper reports relabeling KITTI's 3,712 scenes in about 11 hours, or 2.5 seconds per instance.Source: §3 · pp. 5–6
• Average BEV center errors are about 0.25 m on x and 0.75 m on z, which the paper attributes to LiDAR capturing objects better from the side than from the back..Source: §3 · p. 6

4.1 Learn to Generate Cylindrical Proposals from Click Annotations

• Stage-1 uses click-derived pseudo foreground targets and horizontal-center prediction to generate cylindrical proposals, then applies center-aware suppression to remove nearby duplicates.Proposals have a 4 m radius on the x-z plane and unlimited extent along y; suppression ranks predicted centers by their supporting points’ foreground scores.Source: §4.1 · pp. 6–8

4.2 Learn to Refine Proposals from A Few Well-Labeled Instances

• Stage-2 processes cylindrical proposals through initial cuboid generation and final refinement, which also predicts confidence.The initial and final networks use cuboid regression; the confidence head is trained against an IoU-based target.Source: §4.2 · pp. 8–9

4.3 Implementation Detail

• The implementation samples 16K scene points for Stage-1 and 512 points per proposal or cuboid for Stage-2, and reports inference at about 0.2 s per scene.The default training setup uses 500 weakly annotated scenes and precise labels for 534 vehicle instances.Source: §4.3 · pp. 9–10

5.1 Experimental Setup

• KITTI uses train/val splits of 3,712 and 3,769 images, reports BEV and 3D average precision, and defaults to a 0.7 IoU threshold.The val set is used only for evaluation; outcomes are reported for easy, moderate, and hard regimes.Source: §5.1 · p. 10

5.2 Quantitative and Qualitative Performance

• On KITTI val (Car), the method is comparable to fully supervised leaders and significantly outperforms retrained PointRCNN and PointPillars under comparable scene- and instance-count settings.Training uses 500 center-click scenes and 534 precisely annotated instances; Table 1 reports the KITTI val Car evaluation.Source: §5.2 · p. 11, Table 1 · p. 11
• On KITTI test (Car), the method is competitive with PIXOR and VoxelNet; Fig. 6 visualizes Car detections in yellow, with images used only for visualization.The authors note room for improvement relative to strongly supervised methods.Source: Table 2 · p. 11, §5.2 · p. 12, Figure 6 · p. 12
• Pedestrian results are described as promising using 951 click-labeled scenes and 515 precisely annotated instances; Table 3 reports KITTI val pedestrian evaluation.Evaluation uses a 0.5 IoU threshold.Source: §5.2 · p. 12, Table 3 · p. 12

5.3 Diagnostic Experiment

• Precise BEV annotations yield only marginal improvements over noisy annotations, while additional training data produces gradual performance boosts.The ablations are conducted on the val set because KITTI test ground truth is unavailable and server access is limited.Source: §5.3 · pp. 12–13, Table 4 · p. 13

5.4 Performance as An Annotation Tool

• The annotator supports automatic predictions at about 0.1 seconds per car and active center-click annotation at about 2.6 seconds per car.In active mode, 25 nearby sampled centers seed proposals, and the highest-confidence Stage-2 cuboid is selected.Source: §5.4 · p. 13
• The model produces more accurate annotations than [14], especially in active mode, at 0.1–2.6 seconds per instance.The comparison reports annotation quality on KITTI val and follows the prior methods' 0.5 3D IoU criterion.Source: §5.4 · pp. 13–14, Table 5 · p. 14
• PointPillars and PointRCNN trained on the model's relabeled KITTI train set suffer only small performance drops.Table 6 compares the detectors trained with different annotation sources on KITTI val.Source: §5.4 · p. 14, Table 6 · p. 14

6 Conclusion and Discussion

• The work is an early attempt to train a 3D object detector with limited and weak supervision.The conclusion frames the approach as a direction for further innovation.Source: §6 · p. 14
• The detector is extended into an annotation tool whose performance is examined in both automatic and active modes.Source: §6 · p. 14
• KITTI experiments demonstrate impressive results, while the authors also identify room for improvement.Source: §6 · p. 14

Lidar Point Cloud Supplementary Material

• Section A gives implementation details for adapting the model to 3D pedestrian detection.Source: Lidar Point Cloud Supplementary Material · p. 17
• Section B presents visual results for pedestrian detection on the KITTI validation set.The listed results use point-cloud and image materials elsewhere in the supplement.Source: Lidar Point Cloud Supplementary Material · p. 17
• Section C compares annotations from the trained model in automatic and active modes, and Section D discusses representative failure cases.Source: Lidar Point Cloud Supplementary Material · p. 17

A Weakly Supervised 3D Pedestrian Detection

• Stage 1 uses weak BEV annotations from 951 scenes, while Stage 2 uses 515 of the 2,257 pedestrian instances in those scenes.The prior fully supervised setup uses all 951 exhaustively annotated scenes and 2,257 samples.Source: A Weakly Supervised 3D Pedestrian… · p. 17
• Pedestrian foreground masks use 0.4 m-radius pillars, and cylindrical proposals use a 1 m radius with training centers within 0.5 m of ground truth.The paper contrasts these choices with ellipsoid-shaped soft foreground targets and a 4 m proposal radius for cars.Source: A Weakly Supervised 3D Pedestrian… · p. 17
• Stage 2 training for pedestrians is reduced to 20K iterations, while Stage 1 uses the same parameters as for cars.The car Stage 2 process uses 50K iterations.Source: A Weakly Supervised 3D Pedestrian… · p. 17
• The supplement shows pedestrian detections on KITTI validation image and point-cloud pairs, with boxes depicted in yellow.Source: Figure 1 · p. 18

• For non-occluded pedestrians at reasonable distances with enough points, the model outputs remarkably accurate 3D bounding boxes.Source: B Qualitative Results on KITTI val Set… · p. 18
• The model correctly predicts some highly occluded pedestrians and works well in several crowded scenes.Source: B Qualitative Results on KITTI val Set… · p. 18
• These examples indicate the detector can adapt beyond vehicles to challenging autonomous-driving classes under less and easily acquired supervision.Source: B Qualitative Results on KITTI val Set… · p. 18

• In most cases, automatic mode obtains high-quality annotations, while active mode lets annotators correct inferior or missing predictions with extra center clicks.Source: C Annotation Results on KITTI val Set… · p. 18
• Human clicks on desired object centers generate better proposals around those points, leading to improved predictions.Source: C Annotation Results on KITTI val Set… · p. 18
• Figure 2 depicts labeled 3D boxes in yellow and highlights improved annotations with red circles.Source: Figure 2 · p. 19

• Heavy occlusion can lead to incorrect vehicle height predictions.The paper suggests more contextual information may help.Source: D Failure Cases on KITTI val Set… · p. 19, Figure 3 · p. 20
• Vehicle-like background objects can confuse the car detector, while extremely sparse foreground points make accurate predictions difficult.Source: D Failure Cases on KITTI val Set… · p. 19, Figure 3 · p. 20
• Car failure examples are shown as inaccurate predictions circled in red on image and point-cloud pairs.Source: Figure 3 · p. 20
• For pedestrians, cylindrical obstacles can produce false positives, and two close, highly occluded people can be merged into one prediction.Source: D Failure Cases on KITTI val Set… · p. 20, Figure 4 · p. 20