Source-linked AI summary
PCAN: 3D Attention Map Learning Using Contextual Information for Point Cloud Based Retrieval
Wenxiao Zhang, Chunxia Xiao
TL;DR
Point-cloud place recognition requires discriminative global descriptors while reducing sensitivity to changing visual conditions and irrelevant points. PCAN predicts context-based attention scores for local point features and applies them during aggregation; experiments report improved accuracy over existing state-of-the-art methods, with performance varying across datasets and failure cases where all points receive equal scores.
Problem
Point-cloud place recognition must efficiently encode local features into a discriminative global descriptor, while visual localization based on images is sensitive to changing illumination conditions.
Method
PCAN uses contextual point features, multiscale neighborhood aggregation, and a per-point attention map applied during NetVLAD aggregation to weight local features.
Results
PCAN improves existing state-of-the-art point-cloud place-recognition accuracy, with a baseline recall of 83.81% at top 1%, exceeding PointNetVLAD by 2.8% on Oxford datasets.
Takeaways & Limitations
PCAN can identify significant spatial regions using only localization tags and can be integrated into other point-cloud architectures.
Takeaways & Limitations
On in-house datasets, complex scene structures and fewer training examples make it harder to learn accurate attention maps, and some failures assign score 1 to every point.
Abstract
from arXiv · showhide
Point cloud based retrieval for place recognition is an emerging problem in vision field. The main challenge is how to find an efficient way to encode the local features into a discriminative global descriptor. In this paper, we propose a Point Contextual Attention Network (PCAN), which can predict the significance of each local point feature based on point context. Our network makes it possible to pay more attention to the task-relevent features when aggregating local features. Experiments on various benchmark datasets show that the proposed network can provide outperformance than current state-of-the-art approaches.
1. Introduction
Point cloud retrieval addresses illumination-sensitive visual localization by matching 3D representations. PCAN uses contextual information to reweight local point features, emphasizing task-relevant regions during global descriptor aggregation.
- Visual localization supports augmented reality, robot navigation, and autonomous driving, but image-based retrieval is affected by illumination, weather, and seasonal changes.
- Point cloud retrieval matches global descriptors of 3D point clouds and avoids relying on color information that can cause false matches across illumination conditions.
- The network applies its attention map during NetVLAD aggregation to produce a more discriminative global descriptor.
- Context helps distinguish geometrically similar points, such as a sphere representing a street lamp near a holder versus floating noise without nearby structure.
- PCAN predicts a per-point attention score from contextual information to reweight local features before global aggregation.
- PCAN uses ball-query search with different query radii to aggregate multiscale contextual features because convolution cannot directly handle permutation-sensitive point clouds.
2. Related work
Earlier 3D descriptors primarily target local geometric representation, while learned global retrieval methods combine point-cloud feature extraction with aggregation. PointNetVLAD provides a baseline but does not discriminate local features according to their contribution to the global representation.
- Handcrafted descriptors include Spin Images, Geometry Histograms, PFH, FPFH, and SHOT, but are often task-specific and sensitive to noisy or incomplete sensor data.
- Handcrafted local descriptors are difficult to use for global feature extraction because large data volumes make the computation impractical.
- PointNetVLAD combines PointNet and NetVLAD to extract global descriptors from scanned 3D point clouds for retrieval.
- PointNetVLAD improves aggregation efficiency over vanilla PointNet but does not discriminate local features that positively contribute to the final global representation.
3. Network
PCAN learns a per-point attention map from local features and contextual information, then applies these weights during NetVLAD aggregation to form a global descriptor.
- Network overview: PCAN extracts per-point local features with PointNet and outputs an N × 1 attention map.The attention map assigns one score to each point.
- Feature aggregation: PCAN reweights the VLAD representation during NetVLAD aggregation rather than directly multiplying local features by attention scores.The reweighted local feature map is aggregated into a global descriptor, then compacted with a fully connected layer and normalized.
- Contextual attention map: The network uses ball-query search with different radii to aggregate multi-scale contextual information around points.This replaces direct convolution for orderless point clouds and follows PointNet++-style sampling and grouping.
- Contextual attention map: Three SAG layers group neighboring points with different ball-query radii and the same sampling number before concatenating multi-scale features.A subsequent SAG layer accumulates features, while FP layers recover the original point count.
- Contextual attention map: Two fully connected layers produce the final attention map, whose sigmoid activation yields per-point weights.The FP layers first restore the original number of points.
- Feature aggregation: The soft-assignment operation remains the same as in PointNetVLAD while the learned attention modifies the VLAD aggregation.This keeps the attention mechanism as the principal architectural change.
4. Implements Details
PCAN uses a layer configuration reported in Table 1 and adopts PointNetVLAD’s lazy quadruplet loss and largely shared training settings for fair comparison.
- Layer configuration: Table 1 reports the parameters of each layer in the PCAN architecture.SAG and FP layer number sets denote succeeding PointNet fully connected layers and their widths.
- Training: PCAN uses the same lazy quadruplet loss as PointNetVLAD and mostly follows its training settings for fair comparison.Additional training-setting details are referenced in the supplementary material.
5. Experiments
Experiments evaluate PCAN on LiDAR point-cloud retrieval benchmarks using baseline and refined training settings, retrieval visualizations, and analyses of attention behavior, failure cases, and contextual information. PCAN improves baseline retrieval over comparison methods, while performance depends on dataset structure and training-data availability.
- 5.1. Evaluation Datasets: The evaluation uses Oxford RobotCar and three in-house LiDAR datasets: university sector, residential area, and business district.The benchmark contains submaps from different areas, with ground planes removed and clouds downsampled to 4096 points.
- 5.2. Results: PCAN’s baseline achieves 83.81% average recall at top 1%, exceeding PointNetVLAD by 2.8% and outperforming other methods across datasets.The baseline network also exceeds PointNetVLAD on indoor datasets by 1.2%–1.5%.
- 5.2. Results: Refined training on Oxford, U.S., and R.A. still outperforms PointNetVLAD on Oxford and improves over the Oxford-only baseline, but is similar to PointNetVLAD on in-house datasets.The authors associate the Oxford improvement with reduced overfitting from adding U.S. and R.A. training data.
- 5.3. Results Visualization: PCAN assigns lower attention to unstable tree leaves and floating strips while emphasizing buildings or corner structures that support retrieval.Visualizations show PointNetVLAD retrieving false matches with similar unstable structures despite differing building geometry.
- 5.3. Results Visualization: On in-house scenes, PCAN can suppress distant, floating, or street-center noise even though it was trained only on Oxford data.The discussion attributes this residual ability to the baseline’s small performance advantage over PointNetVLAD on in-house datasets.
- 5.4. Discussion: PCAN sometimes assigns score 1 to every point, degrading to PointNetVLAD, especially with more complex scenes and less in-house training data.The in-house datasets contain 6,671 training point clouds versus 21,711 for Oxford, making accurate attention maps harder to learn across datasets.
- 5.5. The efficiency of context information: PCAN outperforms the context-free PAN on almost all datasets, with Oxford margins of 1.9% at top 1% and 3.6% at top 1.The comparison supports a contribution from contextual information in identifying task-relevant features.
6. Conclusion
PCAN learns point-cloud representations with a context-aware attention map and improves reported point-cloud place-recognition accuracy without additional supervision. Its flexible design also supports integration with other point-cloud tasks.
- PCAN learns point-cloud representations using a context-aware feature-based attention map.
- Experiments show PCAN improves existing state-of-the-art accuracy for point-cloud-based place recognition.
- PCAN uses only localization tags to obtain the attention map without additional supervision.
- The visualization results show that PCAN predicts the significance of spatial regions effectively.
- PCAN can be integrated into other architectures for additional point-cloud tasks such as object recognition.