PaperlayerView exact revision on arXiv

Source-linked AI summary

PC2WF: 3D Wireframe Reconstruction from Raw Point Clouds

Yujia Liu, Stefano D'Aronco, Konrad Schindler, Jan Dirk Wegner

arXiv:2103.02766v1cs.CV

TL;DR

Practical 3D sensing often produces point clouds, while direct wireframe reconstruction from noisy points is challenging. PC2WF learns a pipeline from point clouds to wireframes; it reports promising results for man-made polyhedral objects and better isolated vertex and edge detection than low-level detectors, but does not handle strongly curved edges.

  • Problem

    Directly inferring a wireframe from noisy point clouds is challenging because corner finding, localization, and edge linking are interdependent.

    Source: §1 · p. 1

  • Method

    PC2WF is an end-to-end trainable architecture that converts point clouds into wireframes by detecting and localizing corners and pruning candidate edges.

    Source: §1 · p. 2

  • Results

    PC2WF achieves promising results for man-made polyhedral objects and outperforms low-level corner or edge detectors on isolated vertex and edge detection tasks.

    Source: §5 · pp. 8–9

  • Takeaways & Limitations

    The method moves from low-level detection toward compact, editable 3D geometry for man-made polyhedral objects.

    Source: §5 · pp. 8–9

    The current method cannot handle wireframes with strongly curved edges.

    Source: §5 · p. 9

Abstract

from arXiv · show

We introduce PC2WF, the first end-to-end trainable deep network architecture to convert a 3D point cloud into a wireframe model. The network takes as input an unordered set of 3D points sampled from the surface of some object, and outputs a wireframe of that object, i.e., a sparse set of corner points linked by line segments. Recovering the wireframe is a challenging task, where the numbers of both vertices and edges are different for every instance, and a-priori unknown. Our architecture gradually builds up the model: It starts by encoding the points into feature vectors. Based on those features, it identifies a pool of candidate vertices, then prunes those candidates to a final set of corner vertices and refines their locations. Next, the corners are linked with an exhaustive set of candidate edges, which is again pruned to obtain the final wireframe. All steps are trainable, and errors can be backpropagated through the entire sequence. We validate the proposed model on a publicly available synthetic dataset, for which the ground truth wireframes are accessible, as well as on a new real-world dataset. Our model produces wireframe abstractions of good quality and outperforms several baselines.

1 INTRODUCTION

Point clouds are a minimally committed output of practical 3D sensing, while many man-made objects can be represented more compactly and intuitively as polyhedral geometry. PC2WF learns to convert point clouds into wireframes by jointly detecting and localizing corners and linking them with edges.

1 INTRODUCTION

  • Point clouds are a minimally committed common format from practical 3D sensors, while many man-made objects can be abstracted as corners, straight edges, and planar surfaces.Prior abstraction approaches either detect surfaces and intersect them or directly detect salient corners and edges.Source: §1 · p. 1
  • Wireframes represent shape as corner vertices linked by edges, making salient contours and their connectivity explicit for editing and manipulation.The paper identifies CAD, manufacturing, metrology, inspection, visualization, animation, and rendering as potential uses.Source: §1 · p. 1
  • Wireframe inference from noisy points is challenging because corner detection, accurate localization, and edge linking are interdependent.A plausible corner should also be a meeting point of multiple non-collinear edges.Source: §1 · pp. 1–2
  • PC2WF is an end-to-end trainable architecture that encodes points, finds and localizes corner candidates, then prunes candidate edges into a wireframe.All trainable stages support back-propagation through the sequence.Source: §1 · p. 2

2 RELATED WORK

Prior work addresses corner and edge-point detection, structured 3D models, and wireframe reconstruction, but direct 3D wireframe reconstruction remains limited. PC2WF targets an end-to-end learnable mapping from point clouds to vectorized wireframes.

Corner and edge detection

  • Corner detectors commonly use local second-moment matrices, while 3D edge-point methods use curvature and related geometric features, sometimes with learned classification.EC-Net is described as the first deep-learning method known to the authors for detecting 3D edge points in point clouds.Source: §2 · p. 2

Learning structured models

  • Structured-model methods include meshes, polygonal surfaces, parametric surfaces, and CAD reconstruction; image wireframe methods benefit from the regular pixel grid, unlike irregular 3D points.Prior 3D wireframe work includes Manhattan-world building-interior reconstruction with heuristic edge detection and geometric constraints.Source: §2 · p. 2
  • Primitive-fitting approaches can use coarse occupancy grids, while direct point-cloud primitive fitting is limited by a hard-coded maximum number of instances.Polyfit selects candidate surfaces with RANSAC and works best on objects with straight edges and planar surfaces.Source: §2 · pp. 2–3

Related work

  • Most 3D methods are handcrafted for restricted schematic settings or label individual edge points; PC2WF instead aims to learn a direct point-cloud-to-wireframe mapping.The paper describes direct 3D wireframe reconstruction as a relatively sparse area of work.Source: §2 · pp. 2–3

3 METHOD

PC2WF maps a point cloud to a wireframe through feature extraction, corner detection and localization, and edge detection. Candidate edges are verified using features sampled along their segments, and the trainable stages are optimized jointly.

Overall pipeline

  • The four-block pipeline extracts per-point features, detects corner-containing patches, localizes their vertices, and instantiates and prunes candidate edges.The wireframe is an undirected graph with 3D vertices and edges connecting vertex pairs.Source: §3 · p. 3

Vertex detection

  • Separating patch-level corner detection from localization reduces computation by discarding unlikely patches early and empirically reduces false positives while improving localization accuracy.Patches are formed using geodesic neighborhoods to avoid shortcuts around thin structures or high-curvature regions.Source: §3 · pp. 3–4

Vertex localisation

  • The localizer predicts a vertex as a weighted mean of patch points, using point coordinates and backbone features; predicting weights safeguards against extrapolation errors.The localizer also permits farthest-point sampling instead of testing every input point.Source: §3 · p. 4

Edge detector

  • The edge detector samples points along each candidate segment, pools their backbone features, and scores the candidate as an edge or non-edge.At inference, candidate pairs form a fully connected graph, with candidates too distant from input points discarded before verification.Source: §3 · pp. 4–5
  • Training balances positive and negative edge examples drawn from ground-truth and predicted vertices, including spurious, inaccurate, wrong-link, and near-miss negatives.The examples are sampled from the positive and negative sets rather than evaluating every pair, which the authors found caused strong class imbalance.Source: §3 · pp. 4–5

Training and post-processing

  • The complete network minimizes a joint loss combining patch, vertex, and edge losses, with non-maximum suppression pruning redundant nearby vertices and edges.The joint loss is L = Lpat + αLvert + βLedge.Source: §3 · p. 5

4 EXPERIMENTS

The experiments evaluate PC2WF on synthetic point-cloud datasets using vertex, edge, and whole-wireframe measures. Results show strong vertex detection relative to Harris3D and indicate that long inserted or deleted edges contribute substantially to WED.

4 EXPERIMENTS

  • The ABC subset contains 3,000 straight-edged CAD models split into 2,000 training, 500 validation, and 500 test samples; the furniture dataset contains 250 models.Point clouds are generated with 14 virtual cameras and Gaussian noise; scanned clouds contain approximately 100k–200k points.Source: §4 · p. 5

4.1 EVALUATION MEASURES

  • Evaluation separates vertex and edge geometry from wireframe structure, reporting mAPv, edge-point mAPe, sAP/msAP, and wireframe edit distance.sAP checks whether both endpoints of a predicted edge match a ground-truth edge; WED compares graphs using ordered vertex and edge edit operations.Source: §4.1 · pp. 5–6
  • WED matches vertices, translates matched vertices, inserts missing vertices, then deletes and inserts edges, with costs based on displacement and edge length.The fixed operation order avoids arbitrary edit sequences, which the authors note make graph edit distance NP-hard.Source: §4.1 · p. 6

4.2 RESULTS AND COMPARISONS

  • Patch sizes from 20 to 50 give the best vertex-detection performance, maintaining high precision across a broad recall range and outperforming Harris3D.Patch size 1 has low precision, though still better than Harris3D; the authors attribute this to noise sensitivity in per-point features.Source: §4.2 · p. 6
  • 1.4142: the average increase in WEDe is much larger, and inserted or deleted edges dominate WED.The reported increase is attributed to the significant length of inserted or deleted edges.Source: §4.2 · p. 8

5 CONCLUSION

PC2WF produces promising wireframe reconstructions for man-made, polyhedral objects, while strongly curved edges remain a limitation.

5 CONCLUSION

  • PC2WF achieves promising wireframe reconstruction results for man-made, polyhedral objects.Source: §5 · p. 8
  • Figure 10 compares raw point clouds, predicted wireframes, and ground truth for ABC and furniture examples.Source: Figure 10 · p. 9
  • PC2WF goes beyond low-level corner or edge detectors and outperforms them on isolated vertex and edge detection tasks.Source: §5 · p. 9
  • Strongly curved edges remain the stated limitation of the method.Source: §5 · p. 9

A.1 NETWORK ARCHITECTURE

The appendix describes the FCGF feature-extraction backbone and post-processing steps that remove redundant edges and straighten predictions.

A.1 NETWORK ARCHITECTURE

  • The FCGF backbone uses a U-Net with skip connections and ResNet blocks; its 3D convolution layers generally include batch normalization and ReLU.The last convolution layer is excluded from the batch-normalization and ReLU specification.Source: §A.1 · p. 13, Figure 11 · p. 13

A.2 POST-PROCESSING

  • Non-maximum suppression removes nearby duplicate predicted edges, retaining the edge with the maximum probability among qualifying edges.The appendix illustrates this post-processing in Figure 12.Source: §A.2 · p. 13, Figure 12 · p. 14
  • Straightening removes vertices on or near detected edges to connect their other endpoints, and both straightening and NMS improve performance.The reported ablation is in Table 5; without NMS, results are much worse because sAP penalizes double-predicted lines.Source: §A.2 · p. 13, Table 5 · p. 13, Figure 12 · p. 14

A.3 TRAINING DETAILS

PC2WF is fine-tuned from pre-trained FCGF with ADAM for 10 epochs, followed by a learning-rate reduction.

A.3 TRAINING DETAILS

  • Training fine-tunes pre-trained FCGF with ADAM at an initial learning rate of 0.001 for 10 epochs, then halves the rate.Inputs are normalized to [0, 1], and the loss weights are α = 10 and β = 1.Source: §A.3 · p. 14

B DATASETS

The furniture dataset is collected from web-sourced objects and annotated with wireframe edges and intersections in Google SketchUp.

B DATASETS

  • The furniture dataset is collected from the web, with vertex and edge annotations made in Google SketchUp using wireframe face style.Dataset statistics are presented in Table 6.Source: B DATASETS · p. 14, Table 6 · p. 14

C CHOICE OF POSITIVE AND NEGATIVE EDGE SAMPLES

The ablation results show that both ground-truth and predicted-vertex edge samples contribute to edge-detector performance, and that selected negative examples are useful. The full set combination reaches msAP 0.891, compared with lower scores for the tested reduced sets.

C CHOICE OF POSITIVE AND NEGATIVE EDGE SAMPLES

  • msAP is 0.891 with the full edge-sample combination, versus 0.760 with ground-truth samples only and 0.740 with predicted-vertex samples only.Removing predicted-vertex samples creates a train–inference input-vertex distribution shift; ground-truth samples help provide enough positive vertex examples early in training.Source: C CHOICE OF POSITIVE AND NEGATIVE EDGE… · p. 14, Table 7 · p. 15
  • msAP is 0.586 when training uses only positive edge samples, indicating that selected negative samples are helpful.The comparison is between training on the positive sets alone and the full positive/negative sampling setup.Source: C CHOICE OF POSITIVE AND NEGATIVE EDGE… · p. 15, Table 7 · p. 15

D.1 QUANTITATIVE RESULTS ON FURNITURE DATASET

On the furniture dataset, PC2WF performs similarly to its performance on ABC, with moderate patch sizes giving the best results on both datasets.

D.1 QUANTITATIVE RESULTS ON FURNITURE DATASET

  • Moderate patch sizes of 20–50 achieve the best performance on both the furniture and ABC datasets.The reported furniture results cover vertex and edge detection across patch sizes.Source: §D.1 · p. 15

D.2 POINT CLOUD NOISE LEVELS

The noise experiments find no substantial performance effect across the tested Gaussian-noise levels, though higher noise reduces visual fidelity, especially for small details. Additional examples show reconstructions on noisy ABC and furniture point clouds with small vertex-position deviations from ground truth.

D.2 POINT CLOUD NOISE LEVELS

  • Across σnoise values of 0, 0.01, 0.02, and 0.05, noise does not substantially affect performance, but the higher-noise model is more sensitive to the sAP threshold ηw.At σnoise = 0.05, ηw = 0.03 leads to a less accurate result; visual quality tends to decrease as noise rises, particularly for small details.Source: §D.2 · p. 16, Table 8 · p. 16, Figure 16 · p. 16

D.3 VISUALISATION

  • Additional ABC and furniture examples show successful wireframe reconstruction from noisy point clouds, with small deviations between predicted and ground-truth vertex positions.The examples are shown in Figures 17–20, with vertices marked in orange.Source: §D.3 · p. 17, Figure 17 · p. 17, Figure 18 · p. 18

Related papers

Loading 2103.02766v1…