Source-linked AI summary
MSSP: Multi-Scale Spatially-Constrained Partition for Unsupervised Semantic Segmentation of 3D Point Clouds
Zhenghao Zhang, Xinjie Wang, Wei Wang, Jun Zhang, Hanyun Wang
TL;DR
Unsupervised 3D segmentation must avoid expensive point-level labels, while fixed-granularity superpoint methods miss hierarchical scene structure. MSSP combines multi-scale spectral descriptors with adjacency-constrained clustering, achieving the best unsupervised mIoU on S3DIS and ScanNet; its ablation shows that spatial regularization is needed for multi-scale features to help.
Problem
Unsupervised 3D segmentation seeks meaningful clusters without labels, but existing superpoint methods use fixed-granularity spectral analysis that misses hierarchical scene structure.
Method
MSSP constructs multi-scale superpoint descriptors and restricts clustering merges to physically adjacent regions through spatially constrained clustering.
Results
MSSP achieves the best mIoU among unsupervised methods on S3DIS and ScanNet, with particularly significant gains on S3DIS.
Takeaways & Limitations
Multi-scale features become effective when paired with spatial constraints, revealing spatial coherence as a prerequisite for their use in superpoint clustering.
Abstract
from arXiv · showhide
3D point cloud semantic segmentation is essential for real-world spatial understanding, yet the prohibitive cost of human annotations motivates unsupervised approaches that require no labels. Existing superpoint-based methods typically rely on spectral analysis at a fixed granularity, failing to capture the hierarchical semantic structures inherent in complex indoor scenes. To bridge this gap, we present a Multi-Scale Spatially-Constrained Partition (MSSP) framework that combines multi-scale spectral analysis with spatially-constrained clustering. Multi-scale spectral analysis constructs enriched superpoint descriptors across multiple clustering granularities; however, the resulting high-dimensional feature space calls for a structural prior to translate into cleaner segmentation. Spatially-constrained clustering supplies this prior by restricting superpoint merging to physically adjacent regions, imposing the spatial coherence needed for multi-scale features to be effective. Extensive experiments on S3DIS and ScanNet show that MSSP achieves the best mIoU among unsupervised methods on the main benchmarks, with particularly significant gains on S3DIS. Notably, our ablation reveals a regularize-then-enrich interaction: multi-scale features alone do not improve final segmentation, yet become highly effective when coupled with spatial regularization, underscoring that spatial coherence is aprerequisite for multi-scale representations in superpoint clustering.
1 Introduction
MSSP addresses the limits of fixed-granularity unsupervised superpoint clustering by combining multi-scale spectral analysis with spatially constrained grouping. Experiments report the best unsupervised mIoU on S3DIS and ScanNet, with especially significant gains on S3DIS.
- Unsupervised 3D segmentation avoids costly point-level annotation, which can take approximately 22.3 minutes for one indoor scene.
- Existing methods pool superpoint features at one clustering granularity, limiting their ability to capture semantic structures spanning different spatial extents.
- Fine clusters capture local parts while coarse clusters capture whole objects, motivating descriptors that combine multiple semantic granularities.
- Ablation reveals a regularize-then-enrich interaction: neither component alone improves over baseline, but their combination yields significant gains.
- MSSP combines multi-scale spectral analysis with spatially constrained clustering that restricts merges to physically adjacent regions.
- MSSP achieves the best mIoU among unsupervised methods on ScanNet and S3DIS, with particularly significant gains on S3DIS.
2 Related Work
Related work progresses from geometric superpoint growth and 2D-feature distillation toward spectral clustering, but existing methods remain limited by single-scale superpoint pooling. MSSP is positioned against this gap in unsupervised 3D segmentation.
- Fully unsupervised 3D segmentation is motivated by the annotation requirements of conventional, weakly supervised, and cross-modal approaches.
- GrowSP discovers semantic primitives from geometric features without labels, while GrowSP++ adds DINOv2 feature distillation but retains single-scale pooling.
- MSSP's framework contains separate multi-scale fusion and spatially constrained grouping branches that converge during primitive assignment.
- PointDC distills 2D self-supervised features into 3D, whereas LogoSP applies spectral analysis to a local-global superpoint graph built from distilled DINOv2 features.
- Spectral clustering uses graph-Laplacian eigenvectors as global-structure features, but unsupervised superpoint methods still generally operate at one scale.
3 Method
MSSP distills image features into 3D superpoint descriptors, enriches them across clustering granularities, and discovers primitives at fine spatial resolution. Spatial adjacency constrains grouping so the resulting pseudo-labels remain geometrically coherent.
- DINOv2 image features are projected onto 3D points, distilled by a sparse convolutional backbone, and average-pooled within initial superpoints.
- MSSP builds a superpoint adjacency graph using 6-neighbor voxel connectivity to constrain agglomerative clustering to physically adjacent merges.
- The constrained clustering produces approximately C spatially connected groups, with C=80 on S3DIS and C=40 on ScanNet.
- Multi-scale fusion uses semantic granularity as the scale parameter, with fewer clusters representing broader groups and more clusters preserving finer structure.
- Each scale replaces a raw superpoint feature with its normalized cluster centroid, and the final descriptor concatenates all scales.
- Spectral analysis operates on all initial superpoints rather than merged superpoints, providing finer spatial resolution for semantic primitive discovery.
- Primitive assignment uses original-space cosine similarity, propagates labels to points as pseudo-labels, and trains a classifier with cross-entropy.
- Multi-scale features and spatial constraints are constructed independently but interact through a regularize-then-enrich design that supports coherent groupings.
4 Experiments and Analysis
MSSP is evaluated on ScanNet and S3DIS using standard and cross-dataset benchmarks, qualitative comparisons, and component ablations. It achieves strong unsupervised results, while experiments show that multi-scale analysis is effective primarily when paired with spatial constraints.
- ScanNet results: MSSP achieves the best unsupervised ScanNet OA, mIoU, and mAcc, reaching 70.7, 36.2, and 51.3 respectively.These correspond to improvements of +6.0 OA, +0.4 mIoU, and +0.5 mAcc over LogoSP.
- S3DIS results: Under S3DIS Area 5, MSSP reaches the best unsupervised mIoU of 49.2, improving by +2.6 over GrowSP++ and +2.7 over LogoSP.Under 6-fold cross-validation, MSSP obtains 46.9 mIoU, close to GrowSP++ at 47.1 and above LogoSP at 46.3.
- Qualitative results: Qualitatively, MSSP produces more coherent segments than LogoSP, especially for structurally complex classes and object boundaries.The improvement is visually more pronounced on S3DIS, whose larger scenes contain objects spanning more varied spatial extents.
- Cross-dataset generalization: MSSP outperforms LogoSP by 4.8 mIoU when trained on ScanNet and tested on S3DIS, while the reverse direction yields only a marginal +0.2 improvement.The evaluation trains on one dataset and tests on the other without fine-tuning.
- Component sensitivity analysis: Adding the third clustering scale raises ScanNet mIoU to 36.2, a +1.4 gain over the best pair of scales.The reported scales are fine, medium, and coarse, whose combination captures complementary spatial extents.
5 Conclusion
MSSP combines multi-scale spectral analysis with spatially-constrained clustering for unsupervised 3D point-cloud semantic segmentation. It achieves the best mIoU among unsupervised methods on S3DIS Area-5 and ScanNet, though with higher computational cost than the single-scale baseline.
- 5 Conclusion: MSSP builds enriched superpoint descriptors across multiple clustering granularities and applies spatially-constrained clustering as structural regularization.The method combines multi-scale spectral analysis with spatial constraints on superpoint merging.
- 5 Conclusion: MSSP achieves the best mIoU among unsupervised methods on the S3DIS Area-5 and ScanNet benchmarks.This is reported as the principal benchmark outcome.
- 5 Conclusion: Multi-scale features alone do not improve final segmentation but become effective when combined with spatial constraints.The ablation identifies a regularize-then-enrich interaction between the two components.
- 5 Conclusion: MSSP has higher computational cost than the single-scale baseline.The conclusion identifies computational cost as a trade-off of the approach.