PaperlayerView exact revision on arXiv

Source-linked AI summary

EC-Net: an Edge-aware Point set Consolidation Network

Lequan Yu, Xianzhi Li, Chi-Wing Fu, Daniel Cohen-Or, Pheng-Ann Heng

arXiv:1807.06010v1cs.CV

TL;DR

Point-cloud consolidation must handle sparse, irregular, noisy scans, while prior and neural methods have limitations around complex geometry and sharp features. EC-Net learns local geometry and edges from point patches, regresses upsampled point coordinates and point-to-edge distances, and trains with an edge-aware joint loss. Across virtual and real scans, it improves consolidation and reconstruction quality, particularly in preserving sharp features, but it is not designed to complete large holes or missing parts.

  • Problem

    Sparse, irregular, and noisy scanned point clouds require consolidation, while existing approaches are limited by simplified priors or inadequate handling of sharp features.

  • Method

    EC-Net processes local point patches, expands learned features, regresses point coordinates and point-to-edge distances, and applies an edge-aware joint loss against meshes and edges.

  • Results

    EC-Net improves point-cloud consolidation and 3D reconstruction quality on virtual and real scans, especially at sharp features, compared with state-of-the-art methods.

  • Takeaways & Limitations

    Edge-aware consolidation can preserve sharp features while improving reconstructed surfaces on both sharp and smooth objects.

  • Takeaways & Limitations

    EC-Net is not designed for completion, so filling large holes and missing parts remains a separate problem.

Abstract

from arXiv · show

Point clouds obtained from 3D scans are typically sparse, irregular, and noisy, and required to be consolidated. In this paper, we present the first deep learning based edge-aware technique to facilitate the consolidation of point clouds. We design our network to process points grouped in local patches, and train it to learn and help consolidate points, deliberately for edges. To achieve this, we formulate a regression component to simultaneously recover 3D point coordinates and point-to-edge distances from upsampled features, and an edge-aware joint loss function to directly minimize distances from output points to 3D meshes and to edges. Compared with previous neural network based works, our consolidation is edge-aware. During the synthesis, our network can attend to the detected sharp edges and enable more accurate 3D reconstructions. Also, we trained our network on virtual scanned point clouds, demonstrated the performance of our method on both synthetic and real point clouds, presented various surface reconstruction results, and showed how our method outperforms the state-of-the-arts.

1 Introduction

EC-Net addresses the limitations of prior-based and neural point-cloud consolidation by learning local geometry and sharp edges directly from point patches. Its edge-aware synthesis and joint loss are designed to improve consolidation and surface reconstruction, especially around sharp features.

  • Prior consolidation methods rely on simplified geometric assumptions, so they tend to work well only for specific model classes.Common priors include piecewise smoothness.
  • Neural point-cloud methods are often oblivious to sharp features, where undersampling is more severe and accurate reconstruction is challenging.
  • EC-Net is presented as the first edge-aware consolidation network, learning edge geometry and generating additional points along edges and surfaces.
  • The network processes irregular 3D points in local patches, using PointNet-inspired features and a points-only patch extraction scheme shared across training and testing.
  • EC-Net regresses point-to-edge distances and point coordinates with an edge-aware joint loss that encourages proximity to surfaces and edges and even surface distribution.
  • Trained on synthesized virtual scans and evaluated on virtual and real scans, EC-Net improves consolidation and surface reconstruction quality over state-of-the-art methods, especially at sharp features.

2 Method

EC-Net trains on synthetic scans and local point patches, embedding and expanding features to regress output coordinates and edge distances. Its joint loss encourages surface adherence, edge proximity, and even point distribution.

  • 2.1 Training data preparation: Training uses meshes with manually annotated sharp edges and virtual scans that reproduce variable density and synthetic noise.The virtual scans are composed from camera-based 3D point samples, while edge annotations provide training supervision.
  • 2.1 Training data preparation: Patches are extracted as geodesically close local point groups using a k-nearest-neighbor graph and shortest-path distances rather than Euclidean proximity.This avoids selecting points on opposite sides of thin surfaces and supports consistent training and inference without meshes at inference time.
  • 2.2 Edge-aware Point set Consolidation Network: PointNet++ embeds irregular patch neighborhoods, retains centroid-near features, and expands them in feature space to produce upsampled features.The feature expansion maps N × D features to rN × D2 features, with D=256 and D2=128 in the reported implementation.
  • 2.2 Edge-aware Point set Consolidation Network: Expanded features regress point-to-edge distances and residual 3D coordinates, with the distance feature also supplied to coordinate regression.The regressed distances identify edge points as outputs whose distances fall below a threshold.
  • 2.3 Edge-aware joint loss function: The edge-aware joint loss combines surface, edge, repulsion, and truncated regression terms to shape accurate, edge-aware, and evenly distributed outputs.Surface and edge terms use shortest distances to associated mesh triangles and annotated edge segments, while repulsion penalizes overly close neighboring outputs.

3 Experiments

Experiments evaluate EC-Net on synthetic and real scans, comparing consolidation and reconstruction against established methods. Results show improved surface proximity, edge preservation, and reconstruction quality, including under quantization noise.

  • Experimental setup: The test set uses unseen CAD models and virtual scans, with comparisons against EAR, CLOP, GPF, and PU-Net.Training models are excluded from testing, and virtual scans are generated using the same procedure described for training inputs.
  • Surface reconstruction: EC-Net yields the lowest mean Hausdorff errors across the tested models and achieves the best visual reconstruction quality, particularly near edges.Surface reconstruction uses the ball pivoting algorithm, with comparisons against ground-truth meshes.
  • Method comparison: Compared with PU-Net and EAR, EC-Net better preserves sharp features, while competing methods can smooth or over-sharpen details under noise or undersampling.The comparison also reports improved Poisson reconstruction quality on real scans relative to CLOP and GPF.
  • Quantitative consolidation: EC-Net produces consolidated points closer to the ground-truth meshes than PU-Net and EAR according to mean and RMS point-to-surface distances.The comparison directly evaluates distances from generated points to the associated original mesh.
  • Noise robustness: Under low, medium, and high quantization noise, consolidation moves points closer to the ground-truth surface and shows the same pattern for point-to-edge distances.The experiment uses distinct quantization levels to mimic depth imprecision in real scans.
  • Real scans: On real scans, EC-Net generates more points near edges and surfaces while preserving sharp features, although severely undersampled tiny regions remain imperfect.The real scans contain noise and inhomogeneous point distributions.

4 Discussion and future works

The discussion presents EC-Net as an edge-aware consolidation network whose reconstruction improvements are strongest at sharp features. It also identifies completion, severe undersampling, and scale variation as boundaries for the current method.

  • Discussion: EC-Net is presented as the first edge-aware network for point-cloud consolidation, trained on synthetic data and tested on virtual and real data.Its joint loss encourages points to approach surfaces and edges while becoming more evenly distributed.
  • Discussion: Experiments show improved 3D reconstruction quality, especially at sharp features, compared with state-of-the-art consolidation methods.The conclusion attributes edge awareness to identifying edge points and encouraging outputs near edges and surfaces.
  • Limitations and future work: The method is not designed for completion, so filling large holes and missing parts remains a separate problem.Future work includes enhancing the network and training process for point-cloud completion.
  • Limitations and future work: Severely undersampled tiny structures may lack enough points for reconstructing their sharp edges accurately.The paper notes that insufficient points can make a patch too large relative to the tiny structure.
  • Limitations and future work: The fixed number of points per patch prevents the current implementation from adapting to structures at varying scales.This is identified as a limitation of the current implementation.

A Our Training Data set

EC-Net’s training set combines annotated everyday objects and CAD models, providing sharp-edge supervision for learning edge-aware point-cloud consolidation.

  • Training data: The training data contain 12 everyday objects and 24 CAD models.These models form the dataset used to train EC-Net.
  • Training data: Sharp edges are manually annotated on the training models for edge-aware learning.Figures 10 and 11 visualize the annotated edges in red.

B Network Architecture of EC-Net

EC-Net uses multilevel local feature extraction and feature expansion, then regresses edge distances and residual coordinates to produce consolidated 3D points.

  • Feature embedding: The feature embedding component uses four PointNet++ levels with grouping radii 0.1, 0.2, 0.4, and 0.6.The corresponding sample counts are N, N/2, N/4, and N/8.
  • Feature aggregation: Interpolated multilevel features are reduced to 64 dimensions per level and concatenated into a D = 256 feature.This aggregated feature is denoted as f.
  • Feature expansion: Feature expansion creates r copies of f, processes them with separated convolutions, and concatenates the outputs into f′.The separated convolutions produce 256 and 128 output channels.
  • Regression components: The edge-distance component regresses fdist from f′ and then predicts the point-to-edge distance d.The two fully connected layers have widths 64 and 1, respectively.
  • Regression components: The coordinate component concatenates fdist and f′ into a 192-dimensional feature and regresses residual coordinates before restoring the original coordinates.Two fully connected layers with widths 64 and 3 produce the residual point coordinates.
  • Network details: All convolutional and fully connected layers use ReLU except the final distance and residual-coordinate regression layers.The exceptions leave the final outputs unactivated.

C.1 Comparison with other reconstruction methods on benchmark models

On noisy benchmark models, EC-Net’s consolidation produces visually higher-quality reconstructions that better preserve sharp edges than the compared methods.

  • C.1 Comparison with other reconstruction methods on benchmark models: EC-Net better preserves sharp edges and achieves better visual quality on two noisy benchmark objects.The benchmark models contain both random and systematic error.
  • C.1 Comparison with other reconstruction methods on benchmark models: The comparison includes benchmark results associated with the website’s three best visual methods and EC-Net.
  • C.1 Comparison with other reconstruction methods on benchmark models: Figure 13 compares different reconstruction methods on benchmark models.

C.2 Additional point consolidation and surface reconstruction results

Figures 14–16 provide additional examples of point consolidation and surface reconstruction produced by EC-Net.

  • C.2 Additional point consolidation and surface reconstruction results: Figures 14–16 present additional point consolidation results produced with EC-Net.The figures also show corresponding surface reconstruction results.
  • C.2 Additional point consolidation and surface reconstruction results: The additional figures include surface reconstruction results produced with the proposed method.

C.3 Visual comparison of point-to-surface distances

EC-Net yields lower visual point-to-surface distances than EAR and PU-Net across three models, based on colors representing proximity to the ground-truth mesh.

  • C.3 Visual comparison of point-to-surface distances: EC-Net points are visibly darker than EAR and PU-Net points, indicating lower point-to-surface distances on three models.Each point is colored by its minimum distance to the original ground-truth mesh surface.
  • C.3 Visual comparison of point-to-surface distances: Figure 17 compares point-to-surface distances for EAR, PU-Net, and EC-Net across three different models.

D Surface Reconstruction Results from Point Clouds of Varying Number of Points

The study applies EC-Net to four scans of the same model containing approximately 8k, 16k, 32k, and 64k points, with corresponding reconstructions shown.

  • D Surface Reconstruction Results from Point Clouds of Varying Number of Points: Four virtual scans contain approximately 8k, 16k, 32k, and 64k point samples before surface reconstruction.The top row of Figure 18 shows the input point clouds, and the bottom row shows the resulting reconstructions.
  • D Surface Reconstruction Results from Point Clouds of Varying Number of Points: EC-Net is applied separately to each point cloud to produce a corresponding surface reconstruction.
Loading 1807.06010v1…