PaperlayerView exact revision on arXiv

Source-linked AI summary

Variational Relational Point Completion Network

Liang Pan, Xinyi Chen, Zhongang Cai, Junzhe Zhang, Haiyu Zhao, Shuai Yi, Ziwei Liu

arXiv:2104.10154v1cs.CVcs.LG

TL;DR

Incomplete scans and deterministic completion mappings make fine, structurally plausible reconstruction difficult. VRCNet combines dual-path probabilistic modeling with relational enhancement and contributes the MVP dataset. It outperforms state-of-the-art methods on evaluated benchmarks and shows robustness and generalizability on real-world scans.

  • Problem

    Existing methods often miss fine local details, use deterministic partial-to-complete mappings, and overlook structural relations in man-made objects.

  • Method

    VRCNet uses dual-path probabilistic modeling for partial and complete clouds, then relational modules to refine coarse completions, alongside the MVP dataset.

  • Results

    VRCNet outperforms previous state-of-the-art methods across evaluated benchmark datasets and produces richer, plausible completions for real-world scans.

  • Takeaways & Limitations

    The framework provides fine-detail completion with learned structural relations and supports evaluation using a large multi-view partial-cloud dataset.

  • Takeaways & Limitations

    The probabilistic formulation uses a predefined Gaussian conditional prior p(zg) = N(0, I) and a weighting parameter λ.

Abstract

from arXiv · show

Real-scanned point clouds are often incomplete due to viewpoint, occlusion, and noise. Existing point cloud completion methods tend to generate global shape skeletons and hence lack fine local details. Furthermore, they mostly learn a deterministic partial-to-complete mapping, but overlook structural relations in man-made objects. To tackle these challenges, this paper proposes a variational framework, Variational Relational point Completion network (VRCNet) with two appealing properties: 1) Probabilistic Modeling. In particular, we propose a dual-path architecture to enable principled probabilistic modeling across partial and complete clouds. One path consumes complete point clouds for reconstruction by learning a point VAE. The other path generates complete shapes for partial point clouds, whose embedded distribution is guided by distribution obtained from the reconstruction path during training. 2) Relational Enhancement. Specifically, we carefully design point self-attention kernel and point selective kernel module to exploit relational point features, which refines local shape details conditioned on the coarse completion. In addition, we contribute a multi-view partial point cloud dataset (MVP dataset) containing over 100,000 high-quality scans, which renders partial 3D shapes from 26 uniformly distributed camera poses for each 3D CAD model. Extensive experiments demonstrate that VRCNet outperforms state-of-theart methods on all standard point cloud completion benchmarks. Notably, VRCNet shows great generalizability and robustness on real-world point cloud scans.

1. Introduction

VRCNet addresses incomplete point-cloud completion by combining probabilistic modeling with relational enhancement, and introduces the MVP dataset for broader evaluation. Experiments report stronger benchmark performance, richer details, and robustness on real-world scans.

  • Motivation: Existing completion methods often use deterministic partial-to-complete mappings and struggle to recover fine geometric details and structural relations.The paper highlights relations such as symmetry, regular arrangements, and surface smoothness as important for plausible fine-grained shapes.
  • Approach: VRCNet uses PMNet for probabilistic coarse completion and RENet for relational enhancement of fine point-cloud details.PMNet uses a dual-path design, while RENet exploits multi-scale point relations conditioned on coarse completions.
  • Results: VRCNet outperforms previous state-of-the-art methods across evaluated benchmark datasets.The reported evaluations include the MVP and Completion3D benchmarks.
  • Results: VRCNet reconstructs richer details, predicts different plausible completions from different observations, and performs robustly on KITTI and ScanNet scans.The introduction specifically describes symmetry-based detail recovery and impressive completions for incomplete real-world scans.

2. Related Works

The paper situates point-cloud feature learning around pooling, flexible convolution, and self-attention, while its PMNet figure separates training-time reconstruction from completion and refinement.

  • Multi-scale Features Exploitation: Point-cloud networks commonly exploit local features through local pooling and flexible convolution because regular image convolutions do not directly apply.The absence of regular grids motivates specialized point-cloud operations.
  • Multi-scale Features Exploitation: Self-attention uses linear layers such as fully connected and shared MLP layers, which are suitable for point-cloud processing.
  • Probabilistic Modeling: PMNet separates a training-only reconstruction path from a completion path that predicts coarse shapes before RENet refines local details.The completion path uses inferred distributions and global features, followed by relational enhancement.

3. Our Approach

VRCNet completes point clouds through coarse probabilistic modeling followed by relational enhancement. Its dual-path training links partial and complete shapes, while multi-scale relational modules refine local details.

  • Probabilistic Modeling: VRCNet predicts coarse complete structures from partial clouds using global features and estimated latent distributions, then refines them into fine completions.The coarse completion serves as an adaptive 3D anchor for subsequent point-relation modeling.
  • Probabilistic Modeling: PMNet uses parallel reconstruction and completion paths, with a VAE-like complete-cloud path and a partial-cloud path regularized toward the complete-cloud latent distribution.The reconstruction path is used only during training, so it does not affect inference efficiency.
  • Relational Enhancement: R-PSK adds a residual path to the selective-kernel design, and RENet uses hierarchical encoding and decoding to exploit multi-scale structural relations.Edge-preserved pooling and unpooling support high-resolution complete point clouds with predicted fine local details.
  • Relational Enhancement: PSA adaptively aggregates neighboring point features using learned relations, while PSK adjusts receptive fields and fuses structural relations across scales.PSK uses branches with different K-NN sizes to model relational structures at different scales.
  • Loss Functions: VRCNet is trained end-to-end with reconstruction-path, completion-path, and relational-enhancement losses; the first two include KL and reconstruction terms, while the last uses reconstruction only.The reconstruction loss is the symmetric Chamfer Distance, with weighting parameters for the joint objective.

4. Multi-View Partial Point Cloud Dataset

The MVP dataset provides diverse, high-quality multi-view partial scans by rendering each CAD model from 26 uniformly distributed camera poses. Its scale and coverage support completion and other partial point-cloud tasks.

  • Diversity & Uniform Views: MVP renders 26 partial point clouds per CAD model from camera poses uniformly distributed on a unit sphere.Randomly selecting the first pose is equivalent to applying a random rotation to all 26 fixed relative poses.
  • Diversity & Uniform Views: Uniform camera views reduce similar renderings and cover most parts of each complete 3D shape.
  • Large-Scale & High-Resolution: The dataset contains over 100,000 high-quality incomplete and complete point clouds.
  • MVP contains high-quality synthetic partial scans for CAD models that imitate real-scanned incompleteness caused by self-occlusion.
  • Besides shape completion, MVP supports partial point-cloud tasks including classification, registration, and keypoint extraction.

5. Experiments

Experiments evaluate VRCNet on MVP, Completion3D, and real-world scans using geometric metrics, qualitative comparisons, ablations, and a user study. VRCNet consistently outperforms competing methods while recovering fine structural details and plausible real-scan completions.

  • Shape Completion on Our MVP Dataset: VRCNet outperforms all evaluated methods on MVP shape completion, including methods supporting multiple output resolutions.
  • Shape Completion on Our MVP Dataset: VRCNet recovers fine details by learning structural relations such as geometric symmetries, regular arrangements, and surface smoothness.Examples include reconstructing missing chair legs, a lamp bowl, and a motorbike front wheel from observed structure.
  • Ablation Study: Ablation studies report better completion results after adding PSA, the dual-path architecture, and the two-branch PSK module.
  • Shape Completion on Completion3D: VRCNet ranks first on Completion3D and significantly outperforms state-of-the-art methods using 2,048-point completions.
  • Shape Completion on Real-world Partial Scans: On real scans, VRCNet completes KITTI cars and ScanNet chairs and tables, with fine-tuning required for sparse LiDAR cars but not for chairs and tables.
  • User Study: A user study found VRCNet most preferred overall among PCN, NSFA, and VRCNet for real-scanned point-cloud completion.

6. Conclusion

The paper concludes that VRCNet exploits 3D structural relations for complete-shape prediction while contributing reusable point-feature modules and the MVP dataset.

  • VRCNet exploits 3D structural relations to predict complete shapes.
  • Its PSA and PSK self-attention modules adaptively learn point-cloud features and can be used in other point-cloud tasks.
  • The MVP dataset contains over 100,000 high-quality 3D point clouds for future partial-point-cloud studies.

Supplementary Material

The supplementary material provides detailed analyses, inference information, dataset comparisons, ablations, resource measurements, and a real-scan user study.

  • The supplement includes method analysis, inference details, dataset comparisons, ablation studies, resource usages, and a user study on real scans.

B. Analysis

VRCNet combines probabilistic modeling with relational enhancement: a dual-path variational design produces coarse shape skeletons, while relation operations refine local details and support multiple plausible completions.

  • Probabilistic Modeling: The reconstruction and completion paths use similar variational autoencoder structures, with complete-cloud distributions guiding incomplete-cloud distributions during training.The reconstruction path models complete clouds, while the completion path generates complete shapes from partial clouds.
  • Local Point Relation Learning: Relation operations adapt aggregation weights from local-element relations instead of using fixed convolution weights.This adaptive weighting provides the basis for exploiting structural relations in point clouds.
  • Conditional Completion: VRCNet generates different complete chairs from different observed parts, using learned symmetry and coarse shape skeletons to infer missing structures.Large missing regions are refined from coarse completions, while partial observations influence the predicted complete shape.
  • Relational Enhancement: Self-attention enables RENet to recover fine-grained missing shapes through structural relations such as symmetry, regular arrangements, and surface smoothness.The method produces pluralistic completions for real-scanned incomplete clouds and recovers symmetric structures such as lamp cylinder bulbs.
  • Two-Stage Architecture: The pipeline uses PMNet for probabilistic coarse completion and RENet for multi-scale relational refinement; inference feeds the 1024-point coarse cloud with the 2048-point incomplete cloud.The combined 3072-point input is processed by RENet for fine completion.

D. User Study on Real Scans

The user study evaluates completion quality on real scans of cars, chairs, and tables using blinded volunteer rankings, with VRCNet reported as the most preferred method overall.

  • Study Design: Fifteen volunteers ranked PCN, NSFA, and VRCNet on real scans from the car, chair, and table categories.Each category presented anonymous method outputs for comparative ranking.
  • Study Design: The double-blind evaluation assigned 3, 2, and 1 points to the best, middle, and worst results, respectively, with shuffled presentation order.Scores were averaged across volunteers for each method and category.
  • Results: VRCNet was the most preferred method overall in the real-scan completion user study.The reported comparison covers PCN, NSFA, and VRCNet across cars, chairs, and tables.

E. Ablation Studies

The ablations examine proposed modules and resource usage, reporting accuracy gains from the modules and significant quality improvements for VRCNet at an acceptable additional computational cost.

  • Module Ablations: The ablation study evaluates combinations of the Point Self-Attention Kernel, Dual-Path Architecture, and Point Selective Kernel Module.The study reports results using 2,048-point inputs.
  • Module Ablations: Using the proposed modules improves point cloud completion accuracy, with PSA and the dual-path architecture showing straightforward effectiveness during training and evaluation.The Point Selective Kernel can produce fluctuating evaluation results during training while improving completion performance.
  • Resource Usage: The resource comparison measures PCN, NSFA, and VRCNet under the same batch size and NVIDIA V100 workstation conditions.NSFA has many non-trainable operations, including ball query, grouping, and sampling.
  • Resource Usage: VRCNet achieves significant completion-quality improvements with an acceptable increment in computational cost.This conclusion is reported in the resource-usage comparison against PCN and NSFA.

G. Dataset Comparisons

The MVP dataset addresses inconsistent and unrealistic prior dataset settings by providing a unified, high-quality multi-view benchmark with diverse sampling and shape coverage.

  • Motivation: Earlier incomplete-point-cloud datasets lacked a unified standardized setting, making direct performance comparisons difficult.The discussion also notes that generated incomplete and complete shapes were low quality and unrealistic.
  • Sampling and Views: MVP uses Poisson Disk Sampling rather than uniform sampling to generate complete point clouds that better describe underlying 3D shape surfaces.The comparison notes that PCN and Completion3D randomly render only one incomplete point cloud per CAD model for their testing sets.
  • MVP Dataset: MVP is designed as a high-quality unified benchmark for partial point clouds, with more shape categories than the compared datasets.The dataset comparison identifies diversity of uniform views, large scale, high quality, and rich categories as key properties.
Loading 2104.10154v1…