Source-linked AI summary
PointDAN: A Multi-Scale 3D Domain Adaption Network for Point Cloud Representation
Can Qin, Haoxuan You, Lichen Wang, C. -C. Jay Kuo, Yun Fu
TL;DR
3D point-cloud domain adaptation lacks established direct methods and must account for local geometric structure alongside domain shifts. PointDAN jointly aligns local and global features and introduces PointDA-10; experiments report superiority over state-of-the-art general-purpose DA methods.
Problem
Direct domain adaptation for 3D point clouds is limited, although local geometric information is important and source-target distributions differ.
Method
PointDAN uses multi-scale local and global alignment, with Self-Adaptive nodes, node features, adversarial global alignment, and a shared feature mapping.
Results
PointDAN outperforms general-purpose baseline methods on all PointDA-10 adaptation scenarios and is reported superior to state-of-the-art DA methods.
Takeaways & Limitations
PointDAN and PointDA-10 provide a point-based framework and benchmark for evaluating unsupervised domain adaptation across 3D object domains.
Takeaways & Limitations
The UDA formulation assumes source and target domains share the same label space, while their input distributions differ.
Abstract
from arXiv · showhide
Domain Adaptation (DA) approaches achieved significant improvements in a wide range of machine learning and computer vision tasks (i.e., classification, detection, and segmentation). However, as far as we are aware, there are few methods yet to achieve domain adaptation directly on 3D point cloud data. The unique challenge of point cloud data lies in its abundant spatial geometric information, and the semantics of the whole object is contributed by including regional geometric structures. Specifically, most general-purpose DA methods that struggle for global feature alignment and ignore local geometric information are not suitable for 3D domain alignment. In this paper, we propose a novel 3D Domain Adaptation Network for point cloud data (PointDAN). PointDAN jointly aligns the global and local features in multi-level. For local alignment, we propose Self-Adaptive (SA) node module with an adjusted receptive field to model the discriminative local structures for aligning domains. To represent hierarchically scaled features, node-attention module is further introduced to weight the relationship of SA nodes across objects and domains. For global alignment, an adversarial-training strategy is employed to learn and align global features across domains. Since there is no common evaluation benchmark for 3D point cloud DA scenario, we build a general benchmark (i.e., PointDA-10) extracted from three popular 3D object/scene datasets (i.e., ModelNet, ShapeNet and ScanNet) for cross-domain 3D objects classification fashion. Extensive experiments on PointDA-10 illustrate the superiority of our model over the state-of-the-art general-purpose DA methods.
1 Introduction
3D point-cloud learning is limited by expensive labeling and domain shifts, while existing DA methods largely overlook the local geometry needed for 3D alignment. PointDAN addresses this gap with multi-scale alignment and a new benchmark.
- Motivation: 3D vision DNNs require massive labeled datasets, making point-cloud annotation time-consuming and expensive.This labeling burden limits real-world promotion of 3D vision systems.
- Domain adaptation: Domain adaptation transfers knowledge from label-rich source domains to label-scarce target domains despite distribution shifts.Existing approaches include shared-subspace mapping, instance-distance minimization, and adversarial training.
- PointDAN: PointDAN jointly aligns global and local point-cloud features end-to-end for unsupervised domain adaptation.Its local component uses Self-Adaptive nodes and node attention, while adversarial training aligns global features.
- Benchmark: PointDA-10 is a new benchmark for 3D point-cloud domain adaptation, constructed from ModelNet, ShapeNet, and ScanNet.The benchmark supports evaluation across overlapping object categories from three popular datasets.
- PointDAN: PointDAN introduces Self-Adaptive nodes with node attention to use local geometric information and dynamically gather regional structures across domains.The method is designed to align local distributions while preserving geometry-sensitive features.
2 Related Works
Point-cloud research spans multiple 3D representations, but local geometric information remains important for object understanding and domain alignment. PointDAN provides a point-based framework that aligns local structures and global correlations across domains.
- 3D vision understanding: 3D vision uses multi-view, voxel-grid, mesh, and point-cloud representations, with point clouds preserving spatial information through 3D coordinates.Point clouds can be acquired directly from LiDAR for applications including scene segmentation and autonomous driving.
- Point-cloud representations: PointNet handles point clouds directly with permutation-invariant processing, but it ignores local geometric information important for describing 3D objects.PointNet++ and PointCNN subsequently aggregate local features using hierarchical or convolution-like structures.
- Domain adaptation: Point-cloud labeling is difficult because objects require inspection from multiple angles and scans may contain missing or occluded parts.These issues motivate point-based unsupervised domain adaptation between labeled source and unlabeled target data.
- Domain adaptation: Unsupervised domain adaptation addresses source-target distribution shifts by aligning marginal or conditional feature distributions in a shared space.Deep networks and adversarial training are common strategies for learning domain-invariant representations.
- 3D domain adaptation: Most UDA methods target 2D vision and global image features, whereas 3D tasks require regional geometry; prior point-cloud approaches often project data into 2D.The related-work discussion identifies a need for more generalized point-based 3D UDA.
- PointDAN: PointDAN preserves local structures, explores global correlations among local features, and uses adversarial training to align source-target distributions at both levels.This framework is presented as a generalized 3D point-based UDA approach.
3 Proposed Model
PointDAN defines unsupervised adaptation for 3D point clouds by aligning local geometric and global features jointly. Its local module adapts node receptive fields and weights node contributions, while global alignment uses classifier discrepancy with source classification and MMD objectives.
- 3.1 Problem Definition and Notation: PointDAN formulates 3D point-cloud UDA with labeled source data, unlabeled target data, shared label space, and domain distributions that differ.The mapping projects raw point-cloud inputs into a shared feature space for cross-domain samples.
- 3.2 Local Feature Alignment: SA nodes represent local regions containing a node and its k nearest neighbors, whose locations determine included points.The method seeks nodes covering common geometric structures rather than parts unique to particular objects.
- 3.2 Local Feature Alignment: Geometric-guided shift learning predicts node offsets from weighted neighboring edge vectors, producing self-adaptive receptive fields for local alignment.The edge directions are weighted using mid-level features, and the shifted node is used to find a new neighborhood.
- 3.3 Global Feature Alignment: Global features concatenate raw point features with interpolated SA-node features before a generator and global max-pooling produce object representations.Two classifiers operate on these representations, and their discrepancy is minimized alongside source classification and local MMD losses during end-to-end training.
- 3.2 Local Feature Alignment: Node attention weights the contributions of SA nodes, while MMD aligns their cross-domain features because local offset optimization is sensitive to gradient disturbance.The attention module uses a bottleneck residual structure, and the model applies an RBF kernel for MMD.
- 3.5 Theoretical Analysis: The theoretical analysis connects alternating classifier-discrepancy optimization with reducing H∆H-distance between source and target domains.Classifiers maximize target discrepancy while the generator minimizes it, while source classification and MMD losses remain part of the method.
4 PointDA-10 Dataset
PointDA-10 is a benchmark for 3D point-cloud domain adaptation, built from three ten-class subsets of ModelNet40, ShapeNet, and ScanNet. It defines six cross-domain adaptation scenarios across CAD and scanned indoor-object domains.
- PointDA-10 comprises ModelNet-10, ShapeNet-10, and ScanNet-10, each extracted from 10 classes shared across the source datasets.The source datasets are ModelNet40, ShapeNet, and ScanNet.
- The benchmark organizes six adaptation scenarios: M → S, M → S*, S → M, S → S*, S* → M, and S* → S.
- ModelNet-10 contains sampled surfaces from clean CAD models, while ShapeNet-10 contains more structurally varied online-repository models.ModelNet40’s nightstand class is treated as cabinet because their structures are considered similar; ShapeNet points are uniformly sampled on object surfaces.
- ScanNet-10 contains isolated objects from annotated indoor-scene bounding boxes, where missing parts and occlusion make the domain challenging but realistic.
5 Experiments
Experiments evaluate unsupervised point-cloud classification adaptation on PointDA-10 using shared training policies and repeated target-domain testing. Results show strong overall performance, with local alignment improving convergence and SA nodes supporting geometric matching, while imbalance and pseudo-labeling remain important caveats.
- 5.1 Experiments Setup: Experiments evaluate unsupervised domain adaptation for point-cloud classification on PointDA-10, training with labeled source and unlabeled target samples.Models are evaluated on the target-domain test set, with average top-1 accuracy reported over three runs.
- 5.1 Experiments Setup: The study compares PointDAN with MMD, ADDA, DANN, and MCD under the same loss and training policy.The setup also reports source-only and fully supervised references.
- 5.2 Classification Results on PointDA-10 Dataset: PointDAN outperforms all general-purpose baselines across every PointDA-10 adaptation scenario, including scenarios with the largest domain gaps.The results also report a substantial remaining gap between unsupervised DA and supervised performance.
- 5.2 Classification Results on PointDA-10 Dataset: Class-wise results show local alignment helps most classes, especially Monitor and Chair, but Sofa and Bed remain vulnerable to negative transfer.Chair, Table, and Sofa account for more than 60% of M-to-S samples, contributing to declines for Bed and Sofa.
- 5.3 Quantitative Analysis: Ablations show local alignment is most effective when combined with SA nodes, whereas fixed-node local alignment yields limited improvement.The full method can underperform G+L+A in class-wise accuracy because imbalanced pseudo-label inference may accumulate errors.
- 5.3 Quantitative Analysis: Local alignment accelerates and stabilizes convergence on ModelNet-to-ShapeNet compared with baseline methods, while conv3 provides the best mid-level feature for local alignment.Visualization further shows that matched SA nodes representing similar structures, such as legs and planes, contribute most to alignment across domains.
6 Conclusion
PointDAN is a multi-scale 3D unsupervised domain adaptation framework that aligns local geometric structures and global features, evaluated on a new benchmark.
- PointDAN aligns point-cloud features across domains at multiple scales, combining local geometric structures with global feature alignment.
- Self-Adaptive nodes represent common geometric structures across domains for local feature alignment.
- A GAN-based method aligns features globally within the PointDAN framework.
- The authors build a new 3D domain adaptation benchmark and report superiority over state-of-the-art domain adaptation methods.