Source-linked AI summary
AdaCoSeg: Adaptive Shape Co-Segmentation with Group Consistency Loss
Chenyang Zhu, Kai Xu, Siddhartha Chaudhuri, Li Yi, Leonidas Guibas, Hao Zhang
TL;DR
AdaCoSeg addresses contextual shape co-segmentation, where a shape’s labeling should adapt to the set containing it. It combines an offline part prior with runtime, set-specific optimization under a matrix-rank group consistency loss, producing consistent segmentations without consistent ground-truth labels. The method is reported as robust to geometric and structural variation and superior to state-of-the-art methods.
Problem
Shape co-segmentation requires consistent part correspondences across a collection, but unlike single-instance segmentation, the desired labeling can depend on the input set.
Method
AdaCoSeg first proposes parts with an offline pretrained prior, then jointly optimizes per-shape labels at runtime using a matrix-rank group consistency loss.
Results
AdaCoSeg produces consistent co-segmentations and is reported to be robust to large geometric and structural variations, with performance superior to state-of-the-art methods.
Takeaways & Limitations
The framework delivers set-tailored consistent segmentations using inconsistent part supervision and no consistent ground-truth co-segmentations.
Takeaways & Limitations
The online co-segmentation network is recomputed for each input set, does not generalize to new inputs, and the part prior is not trained across object categories.
Abstract
from arXiv · showhide
We introduce AdaCoSeg, a deep neural network architecture for adaptive co-segmentation of a set of 3D shapes represented as point clouds. Differently from the familiar single-instance segmentation problem, co-segmentation is intrinsically contextual: how a shape is segmented can vary depending on the set it is in. Hence, our network features an adaptive learning module to produce a consistent shape segmentation which adapts to a set. Specifically, given an input set of unsegmented shapes, we first employ an offline pre-trained part prior network to propose per-shape parts. Then, the co-segmentation network iteratively and} jointly optimizes the part labelings across the set subjected to a novel group consistency loss defined by matrix ranks. While the part prior network can be trained with noisy and inconsistently segmented shapes, the final output of AdaCoSeg is a consistent part labeling for the input set, with each shape segmented into up to (a user-specified) K parts. Overall, our method is weakly supervised, producing segmentations tailored to the test set, without consistent ground-truth segmentations. We show qualitative and quantitative results from AdaCoSeg and evaluate it via ablation studies and comparisons to state-of-the-art co-segmentation methods.
1. Introduction
AdaCoSeg treats shape co-segmentation as an adaptive, contextual problem: the same shape may receive different segmentations depending on its input set. It combines a weakly supervised part prior with runtime optimization using group consistency.
- 1. Introduction: The same shape can be segmented differently across sets, while parts remain semantically consistent within each set for applications such as part reshuffling.This contextual behavior distinguishes co-segmentation from fixed-label single-shape segmentation.
- 1. Introduction: AdaCoSeg is introduced as a deep neural network for adaptive co-segmentation of point-cloud shape sets.It produces consistent segmentations tailored to the specific input collection.
- 1. Introduction: The network proposes per-shape parts, then jointly optimizes their labelings with a group consistency loss based on matrix-rank estimates.The output uses up to K parts, where K is user-specified.
- 1. Introduction: AdaCoSeg is weakly supervised: an offline part prior can be trained on noisy, inconsistent segmentations, while runtime co-segmentation needs no consistent ground-truth labels.The prior denoises proposed parts, and the adaptive stage is optimized for each input set.
- 1. Introduction: The paper contributes the first DNN for adaptive shape co-segmentation, a low-rank group consistency loss, and a training framework without ground-truth consistent labels.Evaluation includes qualitative and quantitative results, ablations, and comparisons with state-of-the-art methods.
2. Related work
Prior shape co-segmentation methods commonly combine over-segmentation, handcrafted features, and heuristic clustering, while newer deep approaches impose different restrictions. AdaCoSeg is positioned as adaptive because its consistency optimization can adjust proposed part labelings for the input set.
- 2. Related work: Most supervised deep shape-segmentation models target fixed semantic labels, so a shape’s segmentation does not adapt to its surrounding set.This fixed-label behavior differs from the contextual setting of co-segmentation.
- 2. Related work: Traditional shape co-segmentation methods often embed and cluster over-segmented patches using handcrafted features, which can produce unnatural results under complex part variation.These methods are generally unsupervised but rely on heuristic pipelines.
- 2. Related work: Shu et al. learn per-part features with deep auto-encoders, but their co-segmentation module is not deep and strictly preserves first-stage proposals.AdaCoSeg instead allows its consistency loss to adjust part labelings.
- 2. Related work: BAE-NET [5] learns universal parts through branched autoencoding without explicit group-consistency optimization, and tends to underperform AdaCoSeg on small sets or with large part discrepancies.The comparison identifies explicit group consistency as a key distinction.
3. Overview
AdaCoSeg uses a two-stage point-cloud architecture: a part prior refines noisy part proposals, and an unsupervised co-segmentation network optimizes consistent labels for the current shape set.
- 3. Overview: The architecture comprises a part prior network followed by a co-segmentation network operating on point-set 3D shapes.Shape segmentation is formulated as point labeling.
- 3. Overview: The part prior network maps noisy binary point labels to regularized part labels and is trained using perturbed segmentations from ComplementMe [25].Its purpose is to denoise or refine proposed individual-shape parts.
- 3. Overview: The co-segmentation network learns weights by backpropagation over the input shape set using a group consistency loss, without consistent ground-truth segmentations.It outputs semantically consistent K-way labelings, with K acting as an upper bound on part count.
- 3. Overview: Each predicted part is refined by the pretrained prior and converted into part-aware features whose same-label features across shapes form a part feature matrix.These representations support consistency optimization across the set.
4. Method
AdaCoSeg combines an offline denoising prior with a runtime network that jointly refines K-way segmentations across a test set using group consistency. The method relies on noisy, inconsistently segmented data for prior training and SVD-based consistency optimization without ground-truth labels at runtime.
- Part Prior Network: The offline stage trains a weak prior on noisy, unlabeled part structures to denoise proposed parts across shapes and granularities.The prior is trained on ComplementMe data, where component structures are available but segmentations are inconsistent and unlabeled.
- Part Prior Network: The part prior network encodes a noisy foreground and whole shape with PointNet++ MSG and MRG features, then predicts a clean foreground probability for each point.Foreground features are pooled and combined with per-point MRG features before binary classification.
- Part Prior Network: Noise-injected parts train the prior to recover clean indicator functions using negative log-likelihood and Adam optimization.Training randomly inserts background points and removes foreground points by approximately 20–30%.
- Co-segmentation Network: At runtime, a shared K-way classifier proposes parts for each test shape, which the frozen prior refines before part features are extracted.The network operates on the input set rather than learning to generalize to new shape collections.
- Co-segmentation Network: For each abstract label, foreground descriptors across shapes form a similarity matrix whose rank-based energy drives joint consistency optimization.The rank is approximated using the second singular value computed by SVD, and gradient descent iteratively refines the segmentation.
5. Results and Evaluations
AdaCoSeg is evaluated through benchmark comparisons, rank-based consistency analysis, adaptivity tests, and ablations. It achieves strong accuracy while adapting segmentation granularity and input-dependent part structure.
- Discriminative power of matrix ranks: Matrix-rank scores distinguish collections with one, two, and three labels, whereas mean squared error cannot separate two- and three-label collections.The rank-based metric therefore more accurately reflects consistency of a part collection.
- Control, adaptivity, and generalization: AdaCoSeg adapts granularity through K, while the discovered part count can be lower than K and coarse-to-fine nesting is not guaranteed.The same shapes are evaluated with K = 2, 3, and 4.
- Control, adaptivity, and generalization: For fixed K, different input collections induce different segmentations, with detected part types changing according to the structures present in each collection.Chair collections with and without arms produce different four-part assignments.
- Quantitative evaluation: AdaCoSeg achieves the greatest overall accuracy, measured by the lowest Rand Index, compared with prior methods despite using point clouds while they access mesh data.The comparison uses category-specific weak priors and evaluates categories shared with ComplementMe.
- Ablation study: AdaCoSeg improves noisy training segmentations, and its ablations show that the part prior, denoising, and contrastive group-loss term are important for optimization and performance.Removing the part prior or contrastive term prevents substantial loss reduction, while MSG performs better than MRG for part encoding.
- Comparison to BAE-NET: With explicit group-consistency optimization, AdaCoSeg generally outperforms BAE-NET on small test sets and when part discrepancies are strong.The comparison considers BAE-NET using one perfect exemplar.
6. Conclusion, limitation, and future work
AdaCoSeg combines an offline part prior with an adaptive online co-segmentation network that iteratively minimizes group consistency loss, producing consistent segmentations without ground-truth consistent labels. Experiments show robustness to substantial geometric and structural variation and superiority to state-of-the-art methods, while the approach remains input-set-specific and category-specific.
- Conclusion: AdaCoSeg is robust to large geometric and structural variations in input sets and outperforms state-of-the-art co-segmentation methods.The reported experiments support this overall comparison without supplying quantitative values here.
- Conclusion: The method requires no ground-truth consistent co-segmentations because its offline prior and online co-segmentation stages address separate tasks using different datasets.The prior denoises individual-shape part proposals, while the online stage optimizes consistency for a specific test set.
- Limitation: AdaCoSeg does not generalize its online network to new inputs because weights are recomputed for every input set, and it is not trained end-to-end.These choices are deliberate for set-specific unsupervised optimization, although an end-to-end deep co-segmentation network is identified as desirable.
- Limitation and future work: The part prior is trained per object category rather than across categories, limiting cross-category part learning despite handling some intra-category variation.The authors identify cross-category part learning as future work and note that mixed-category learning is more challenging.
- Future work: Future work includes online learning that incrementally updates models as new shapes arrive, potentially maintaining multiple segmentation templates.The current network cannot continuously update its weights as new shapes enter the input stream.