Source-linked AI summary
Semantic Graph Based Place Recognition for 3D Point Clouds
Xin Kong, Xuemeng Yang, Guangyao Zhai, Xiangrui Zhao, Xianfang Zeng, Mengmeng Wang, Yong Liu, Wanlong Li, Feng Wen
TL;DR
3D point-cloud place recognition needs descriptors robust to occlusion and viewpoint changes. The paper represents scenes as semantic graphs and matches them with a graph similarity network. On KITTI, it reports state-of-the-art performance, particularly for reverse loops and robustness to occlusion and viewpoint changes.
Problem
3D point-cloud place recognition remains an open issue because effective descriptors must be robust to occlusion and viewpoint changes.
Method
The method represents point-cloud scenes as semantic graphs capturing semantic objects and their topological relations, then estimates graph-matching similarity with a learned network.
Results
KITTI odometry experiments achieve state-of-the-art performance, especially for reverse loop closure detection and robustness to occlusion and viewpoint changes.
Takeaways & Limitations
Semantic-level representation offers a promising direction for exploiting semantic information in large-scale 3D point-cloud place recognition.
Abstract
from arXiv · showhide
Due to the difficulty in generating the effective descriptors which are robust to occlusion and viewpoint changes, place recognition for 3D point cloud remains an open issue. Unlike most of the existing methods that focus on extracting local, global, and statistical features of raw point clouds, our method aims at the semantic level that can be superior in terms of robustness to environmental changes. Inspired by the perspective of humans, who recognize scenes through identifying semantic objects and capturing their relations, this paper presents a novel semantic graph based approach for place recognition. First, we propose a novel semantic graph representation for the point cloud scenes by reserving the semantic and topological information of the raw point cloud. Thus, place recognition is modeled as a graph matching problem. Then we design a fast and effective graph similarity network to compute the similarity. Exhaustive evaluations on the KITTI dataset show that our approach is robust to the occlusion as well as viewpoint changes and outperforms the state-of-the-art methods with a large margin. Our code is available at: \url{https://github.com/kxhit/SG_PR}.
I. INTRODUCTION
Place recognition supports loop closure in SLAM, but point-cloud descriptors remain vulnerable to occlusion and viewpoint changes. The paper addresses this with semantic graphs and graph similarity learning, reporting strong KITTI performance, especially for reverse loops.
- Loop closure detection recognizes previously visited places, helping reduce cumulative odometry drift and improve global mapping and localization.
- Existing LiDAR methods extract low-level local or global descriptors from raw point clouds, which are sensitive to occlusion and rotation.
- The proposed representation aggregates semantic information and topological relations between semantic objects into a graph.
- The method estimates graph-matching similarity with an effective and efficient network for loop closure detection.
- KITTI odometry experiments achieve state-of-the-art performance, especially for reverse loops and robustness to occlusion and viewpoint changes.
II. RELATED WORK
Prior 3D point-cloud place-recognition methods use local descriptors, global descriptors, or object-level segments. The paper identifies limitations in descriptor distinctiveness, spatial information, partial-point robustness, and modeling relations among objects.
- Local descriptor based methods: Local descriptor methods match keypoint features, but local structures may lack sufficient descriptive power for reliable place matching.
- Global descriptor based methods: Global descriptor methods encode shape, density, binary signatures, or learned features, but can lose spatial detail or degrade with partial point loss.
- Segments based methods: Segment-based methods identify object-level elements using 3D CNN features and nearest-neighbor matching in feature space.
- Segments based methods: Segment-based approaches require a dense local map and do not account for relations among objects.
- The proposed approach addresses these limitations with a concise semantic graph and graph similarity rather than Euclidean distance.
III. METHODOLOGY
The method represents 3D point-cloud scenes as semantic graphs that preserve semantic information and topological relations, then uses learned graph similarity for matching.
- The authors describe this as the first use of topological semantic graph representation and graph matching for loop closure detection.
A. Semantic Graph Representation
The semantic graph represents point-cloud scenes with instance nodes that retain semantic labels and topological information, producing a concise scene representation.
- Semantic segmentation supplies point-level semantic information, with instances formed by clustering points sharing the same semantic label.
- Each node denotes an instance in the scene and preserves its semantic and topological information.
- The graph retains semantic information and topological relations of semantic instances while reducing the redundancy of dense LiDAR point clouds.
- Each scene instance becomes a node storing its semantic category and centroid coordinates as a four-dimensional node feature.
B. Graph Similarity Network
The graph similarity network builds permutation-invariant node and graph embeddings, models interactions between graph embeddings, and predicts scene similarity.
- The node-embedding architecture obtains spatial and semantic node features separately before concatenating them into the node embedding.
- EdgeConv captures local geometric information with dynamically updated neighborhoods rather than a fixed adjacency matrix.
- Node features combine centroid information with one-hot semantic labels, while k-nearest-neighbor relations provide local feature context.
- Spatial and semantic features are aggregated separately and concatenated into each node’s final embedding.
- An attention module learns which node embeddings should receive greater weight when forming the graph embedding.
- A Neural Tensor Network estimates relations between graph-level embeddings through a bilinear tensor layer across multiple dimensions.
- Fully connected layers reduce the similarity vector to one score in [0, 1] and train the model as binary classification with binary cross-entropy.
A. Dataset and Implementation Details
The evaluation uses selected KITTI odometry sequences with pose-based loop-closure labels, fixed-size graph inputs, and F1 max as the principal score.
- Dataset: KITTI odometry provides 11 LiDAR sequences with ground-truth poses; sequences 00, 02, 05, 06, 07, and 08 containing loop closures are evaluated.Positive pairs are within 3 m, negative pairs exceed 20 m, and positives must be more than 30 s apart to count as true loop closures.
- Implementation: The SemanticKITTI labels are merged into 12 categories, while scenes contain 10–70 nodes and are padded to 100 nodes for batch training.The graph uses k = 10 nearest neighbors, and fake nodes receive zero embeddings.
- Evaluation metric: Evaluation reports F1 max scores on the KITTI dataset.The supplied table caption identifies F1 max as the reported score.
- Pair sampling: The evaluation reserves all positive pairs and randomly samples a proportion of negative pairs.
B. Place Recognition Performance
The semantic graph approach achieves strong KITTI place-recognition performance, including reverse-loop settings where global-descriptor methods degrade, while attention visualizations show interpretable node weighting.
- Baselines: The evaluation compares Ours-RN and Ours-SK with M2DP, Scan Context, and pretrained or KITTI-retrained PointNetVLAD.Ours-RN uses RangeNet++ predictions, while Ours-SK uses SemanticKITTI labels.
- Qualitative analysis: Graph visualizations use node numbers for semantic categories and color intensity for attention weights, with deeper colors indicating higher weights.Fake zero-padding nodes appear as -1 and have zero one-hot encodings.
- Robustness evaluation: Table II reports F1 max scores under 30° random occlusion and z-axis rotation, compared with standard results in Table I.
- Quantitative results: The method’s mean F1 max score outperforms existing methods, with especially consistent performance on sequence 08 reverse loops.M2DP, PV-PRE, and PV-KITTI show severe degradation under viewpoint variation, whereas the proposed approach remains consistent.
- Qualitative analysis: Nodes closer to the graph center receive higher attention, whereas fake zero-padding nodes receive lower attention.
- Qualitative analysis: Similarity visualizations show distinct and accurate predictions around a selected scene, with nearby scenes expected to have higher similarity.The visualization colors trajectories according to similarity scores.
C. Robustness Test
Robustness tests examine occlusion, viewpoint changes, and distance-threshold choices; the proposed approach is reported as least affected by occlusion and consistently satisfactory across thresholds.
- Occlusion: Under 30° azimuthal point removal, all methods degrade, but the proposed methods are least affected.Occlusion corresponds to disappearance of some semantic-graph nodes, and the graph similarity network is described as robust to missing nodes.
- Similarity visualization: The similarity visualization compares one selected scene with scenes from each sequence and expects only scenes within a distance threshold to appear similar.The reported predictions are distinct and accurate.
- Viewpoint changes: Viewpoint testing randomly rotates point clouds, exposing substantial drops in methods lacking rotation invariance.The section identifies viewpoint change as a crucial condition for loop closure recognition.
- Distance thresholds: The distance-threshold study trains models for 3 m, 5 m, and 10 m positive-pair thresholds and reports satisfactory performance across them.Precision-recall curves for these settings are shown for KITTI00.
D. Efficiency
The graph similarity network is designed for efficient inference, producing similarity scores for many places in one pass and operating at about 9 ms per batch.
- Evaluation setup: Precision-recall curves on KITTI00 are evaluated under different distance thresholds.
- Inference: The network obtains similarity scores for N places in a single pass instead of comparing places one by one.The descriptors are 192-dimensional for M2DP, 256 for PointNetVLAD, 20×60 for Scan Context, and 100×4 for the proposed method.
- Inference: With batch size 128, inference takes about 9 ms and uses 2820 MB GPU memory on an NVIDIA GeForce RTX 2080 Ti.The authors describe this runtime as applicable to real-time robotics systems.
V. CONCLUSION
The paper proposes a semantic graph approach for large-scale 3D point-cloud place recognition and reports feasibility and robustness, especially for reverse loops. Future work will investigate unsupervised semantic feature learning for place recognition.
- The proposed approach performs place recognition at the semantic level rather than extracting only local, global, or statistical raw-point-cloud features.It is presented as a promising direction for exploiting semantic information in place recognition.
- Exhaustive evaluations demonstrate the approach’s feasibility and robustness, especially for reverse loops.
- Future work will investigate unsupervised semantic feature learning for point clouds and its availability for place recognition.