PaperlayerView exact revision on arXiv

Source-linked AI summary

Snowflake Point Deconvolution for Point Cloud Completion and Generation with Skip-Transformer

Peng Xiang, Xin Wen, Yu-Shen Liu, Yan-Pei Cao, Pengfei Wan, Wen Zheng, Zhizhong Han

arXiv:2202.09367v3cs.CV

TL;DR

Point-cloud completion methods often struggle to preserve local structure needed to reveal fine geometric detail. SnowflakeNet uses progressive point splitting with a skip-transformer to generate locally structured point clouds. Experiments report superior completion performance on sparse and dense benchmarks, with SPD also evaluated across several other generative tasks.

  • Problem

    Most point-cloud completion methods struggle with unstructured local predictions, making it difficult to preserve local geometric details such as smooth regions, sharp edges, and corners.

    Source: §1 · p. 1

  • Method

    SnowflakeNet progressively splits parent points into child points with SPD, while a skip-transformer uses attention across SPD layers to learn local splitting patterns.

    Source: §1 · p. 1

  • Results

    SnowflakeNet outperforms current state-of-the-art methods on sparse Completion3D and dense PCN completion benchmarks, and SPD is demonstrated across additional generation tasks.

    Source: §5 · p. 15

  • Takeaways & Limitations

    SPD is applied beyond completion to point-cloud auto-encoding, novel point-cloud generation, single-image reconstruction, and upsampling.

    Source: §5 · p. 15

    The skip-transformer's k-nearest-neighbor aggregation limits long-range shape context, and noisy inputs can prevent SPD from arranging point splitting properly.

    Source: §5 · p. 15

Abstract

from arXiv · show

Most existing point cloud completion methods suffer from the discrete nature of point clouds and the unstructured prediction of points in local regions, which makes it difficult to reveal fine local geometric details. To resolve this issue, we propose SnowflakeNet with snowflake point deconvolution (SPD) to generate complete point clouds. SPD models the generation of point clouds as the snowflake-like growth of points, where child points are generated progressively by splitting their parent points after each SPD. Our insight into the detailed geometry is to introduce a skip-transformer in the SPD to learn the point splitting patterns that can best fit the local regions. The skip-transformer leverages attention mechanism to summarize the splitting patterns used in the previous SPD layer to produce the splitting in the current layer. The locally compact and structured point clouds generated by SPD precisely reveal the structural characteristics of the 3D shape in local patches, which enables us to predict highly detailed geometries. Moreover, since SPD is a general operation that is not limited to completion, we explore its applications in other generative tasks, including point cloud auto-encoding, generation, single image reconstruction, and upsampling. Our experimental results outperform state-of-the-art methods under widely used benchmarks.

1 INTRODUCTION

SnowflakeNet addresses the difficulty of preserving local geometric detail in point-cloud completion by generating points through structured, progressive splitting. The introduction presents SPD and its skip-transformer as central to this approach, then positions SPD for other generative tasks and transformer-based decoding.

1 INTRODUCTION

  • Existing completion methods struggle to preserve local point structure, making smooth regions, sharp edges, and corners difficult to reveal.Prior approaches constrain generation through rooted trees or assumed topology, but often leave local point predictions unstructured.Source: §1 · p. 1
  • SnowflakeNet progressively splits parent points into child points through stacked SPD layers, inheriting shape characteristics during point-wise generation.The paper describes this as snowflake-like growth of points in 3D space.Source: §1 · pp. 1–2
  • A skip-transformer uses attention across SPD layers to learn local splitting patterns and produce compact, structured point arrangements.It summarizes patterns from a previous layer to inform splitting in the current layer, capturing local patch structure.Source: §1 · pp. 1–2
  • SnowflakeNet combines hierarchical rooted-tree generation with an explicit, explainable process and locally structured point splitting.The introduction contrasts its local pattern focus with prior methods that expand point counts or reconstruct global shape without preserving local structure.Source: §2.1 · pp. 2–3

2.2 Point Cloud Generation

  • SPD is extended beyond completion to point-cloud auto-encoding, novel point-cloud generation, single-image reconstruction, and upsampling.The paper presents these tasks as applications of a general point-cloud generation operation.Source: §1 · pp. 1–2, §2.2 · p. 3

2.3 Relation to Transformer

  • The paper applies transformer-based structures to point-cloud decoding, extending their use beyond representation learning in the encoder.The proposed skip-transformer is presented as a way to generate high-quality 3D shapes.Source: §1 · p. 1, §2.3 · p. 3

3 SNOWFLAKENET

SnowflakeNet combines feature extraction, seed generation, and progressive point generation to complete point clouds. Its SPD layers grow point trees, while skip-transformer connections coordinate splitting across layers and the losses supervise completion and input-shape preservation.

3.1 Overview

  • The feature extractor builds a global shape code, and the seed generator combines predicted coarse points with the input before downsampling to form the point-generation seeds.The method uses three set-abstraction layers with point transformers; it typically sets N_c = 256 and N_0 = 512.Source: §3.1 · pp. 3–4

3.2 Snowflake Point Deconvolution (SPD)

  • SPD progressively splits parent points into children, inheriting parent shape features and producing local tree-structured patches through point-wise splitting.A parent is duplicated r_i times and the children are displaced into its neighborhood; r_i = 1 can reposition points, while r_i > 1 expands their number.Source: §3.2 · pp. 4–5

3.3 Skip-Transformer

  • The skip-transformer links consecutive SPD layers by attending to k-nearest neighbors’ displacement features, refining context used to coordinate local splitting patterns.Its skip connection links the previous layer’s displacement feature with the current layer’s point feature.Source: §3.3 · p. 5, §3.2 · p. 5

3.4.1 Completion loss

  • Completion supervision sums four Chamfer-distance losses on the coarse seed cloud and successive split outputs, each matched to a ground-truth cloud at corresponding sampling density.The paper also uses Earth Mover’s distance as a training loss by replacing the Chamfer-distance terms.Source: §3.4.1 · pp. 5–6

3.4.2 Preservation loss

  • The preservation loss is a one-way partial-matching constraint, requiring the output to match the incomplete input without imposing the reverse constraint.The authors use the partial matching loss as L_preservation.Source: §3.4.2 · p. 6

4 EXPERIMENTS

The experiments cover completion benchmarks and additional point-cloud generative tasks. The PCN experiments include EMD-based evaluation and visual comparison.

4 EXPERIMENTS

  • Experiments span PCN and Completion3D completion benchmarks, novel-shape completion, auto-encoding, generation, single-image reconstruction, and upsampling.The PCN and Completion3D benchmarks are described as widely used.Source: §4 · p. 6
  • PCN completion is evaluated in terms of EMD ×10^2, with lower values better.Source: Table 2 · p. 6
  • The PCN visual comparison follows MSN and SpareNet settings, with SnowflakeNet trained using EMD.Source: Figure 7 · p. 6

4.1 Evaluation on the PCN Dataset

On PCN, SnowflakeNet is evaluated using both Chamfer distance and EMD, and it achieves the best overall results on both metrics. Visual comparisons report smoother point distributions and clearer local structures.

4.1.1 Dataset briefs and evaluation metric

  • The PCN evaluation uses L1 CD and EMD; EMD is also used because CD may be insensitive to global point distribution.Source: §4.1.1 · p. 7

4.1.2 Quantitative comparison

  • SnowflakeNet reduces average EMD by 0.232 (13.2%) versus PoinTr’s 1.764 and average CD by 0.35 (4.6%) versus PMP-Net++’s 7.56.Source: §4.1.2 · p. 7

4.1.3 Visual comparison

  • Visual comparisons report smoother car boundaries and clearer chair-back structure, while EMD-guided outputs tend to maintain consistent distribution across shapes.Source: §4.1.3 · p. 7

4.2 Evaluation on the Completion3D Dataset

On Completion3D, SnowflakeNet achieves the best leaderboard results and is reported to produce more detailed completions on sparse inputs. The evaluation uses L2 Chamfer distance.

4.2.1 Dataset briefs and evaluation metric

  • Completion3D has 2,048-point partial and ground-truth clouds, and evaluation uses L2 Chamfer distance on the test set.Source: §4.2.1 · p. 7

4.2.2 Quantitative comparison

  • SnowflakeNet reduces average CD by 0.50 (6.1%) versus VE-PCN’s 8.10 and leads the Completion3D leaderboard.Source: §4.2.2 · p. 7

4.2.3 Visual comparison

  • Visual comparisons report more complete plane details and finer boat sails and table legs than competing methods.The figure caption also describes smoother surfaces and more detailed structures.Source: §4.2.3 · p. 8

4.3 Evaluation on the ShapeNet-34/21 Dataset

On ShapeNet-34/21, SnowflakeNet is evaluated across three levels of incompleteness, including unseen categories. It outperforms the compared methods in CD and produces visually uniform completions for unseen objects.

4.3.1 Dataset briefs and evaluation metric

  • The dataset uses 34 observed categories for training and 21 unseen categories for novel-shape completion, evaluated with L2 CD and F-Score.Difficulty levels correspond to removing 2,048, 4,096, or 6,144 points.Source: §4.3.1 · p. 8

4.3.2 Quantitative comparison

  • For unseen categories, SnowflakeNet lowers CD-Avg by 5.4 (26.3%) versus PoinTr and has competitive F1 scores.It also outperforms the other counterparts in CD across Simple, Moderate, and Hard levels.Source: §4.3.2 · p. 8

4.3.3 Visual comparison

  • Visual results on unseen bowl and copy-machine instances show recovered complete shapes with visually uniform point distributions.Source: §4.3.3 · pp. 8–9

4.4 Extension to Real-World Scenarios

SnowflakeNet is evaluated on two real-world settings: KITTI after car-data fine-tuning and ScanNet chairs without fine-tuning. The passages describe quantitative and visual comparisons but do not provide their numerical outcomes.

4.4 Extension to Real-World Scenarios

  • The real-world evaluation covers KITTI and ScanNet chair instances.The experiments are presented as evaluations of SnowflakeNet's generalization to real-world scenarios.Source: §4.4 · p. 9

4.4.1 Extension to the KITTI benchmark

  • SnowflakeNet is tested on KITTI after fine-tuning its trained model on ShapeNet cars, with fidelity and MMD used for quantitative comparison.The visual comparison is shown in Fig. 10.Source: §4.4.1 · p. 9, Table 5 · p. 9, Figure 10 · p. 9

4.4.2 Extension to ScanNet chairs

  • On ScanNet chairs, the Completion3D-pretrained SnowflakeNet is tested without fine-tuning against four methods using their pretrained models.The comparison uses fidelity and MMD, with visual results in Fig. 11.Source: §4.4.2 · p. 9, Table 6 · p. 9, Figure 11 · p. 9

4.5 Ablation Studies

The ablations examine the skip-transformer, SnowflakeNet components, splitting strategy, neighborhood size, and noise robustness; a visualization also illustrates SPD's local point-generation patterns.

4.5.1 Effect of the skip-transformer

  • The Full model performs best among skip-transformer variations; comparisons support the skip-transformer connection and transformer attention, while attention also contributes relative to No-att.The variations remove the transformer, replace it with current-layer self-attention, or remove the cross-layer connection.Source: §4.5.1 · pp. 9–10, Table 7 · p. 9

4.5.2 Effect of each part in SnowflakeNet

  • The Full model outperforms folding-based expansion; SPD with the PCN encoder improves over PCN-baseline, while preservation loss slightly improves average performance with category-dependent effects.These comparisons assess point-wise splitting, decoder use with another encoder, and partial matching loss.Source: §4.5.2 · p. 10, Table 8 · p. 10

4.5.3 The effect of the splitting strategy

  • Multiple splitting steps significantly improve average CD over one-step splitting; two steps suffice for sparse 2048-point output, while extra SPDs add computation without necessarily improving performance.An additional SPD with upsampling factor 1 can adjust initial point positions and facilitate splitting.Source: §4.5.3 · p. 10, Table 9 · p. 10

4.5.4 Effect of neighbor number k

  • The Full model outperforms Self-att across tested neighbor counts; its performance improves steadily as k rises from 4 to 32, then changes little with larger neighborhoods.The authors set k to 16 to balance performance and inference efficiency; oversized neighborhoods may introduce noise and reduce focus on local context.Source: §4.5.4 · p. 10, Table 10 · p. 10

4.5.5 Robustness to random noise

  • Training with noise hampers noise-free performance but improves robustness at 0.5%–1.5% noise; performance degrades at 2.0%.At 2.0% noise, the input may lose its underlying structure, although the model can still produce a plausible overall shape.Source: §4.5.5 · pp. 10–11, Table 11 · p. 11, Figure 12 · p. 11

4.5.6 Visualization of point generation process of SPD

  • SPD places child points smoothly around parents on planes and captures geometries in thin tubes and sharp edges.The visualization shows two successive splitting layers and their paths from P1 to P2 and P2 to P3.Source: §4.5.6 · p. 11

4.6 Point Cloud Auto-Encoding

The auto-encoding experiments test SnowflakeNet's decoder as a point-cloud generator on ShapeNet. The method outperforms compared counterparts quantitatively and produces detailed, uniform reconstructions and smooth latent transitions.

4.6.1 Dataset and evaluation metric

  • The experiment evaluates decoder generation through point-cloud reconstruction from a reduced, encoded representation on ShapeNet, including three separate categories and the full dataset.The dataset has 51,127 shapes from 55 categories; CD and EMD are the evaluation metrics.Source: §4.6 · p. 11, §4.6.1 · p. 11

4.6.2 Network arrangement and training loss

  • The decoder uses a PointNet encoder and a seed generator followed by two stacked SPDs, each with upsampling factor 2, so generation relies on the encoder's latent code.Training uses L2 Chamfer distance and Earth Mover's distance as reconstruction losses.Source: §4.6.2 · p. 11

4.6.3 Quantitative comparison

  • Average CD falls by 1.05, 20% below DPM's 5.25 average CD on the entire ShapeNet dataset; SnowflakeNet also leads across categories in per-category CD and EMD.The comparison reports the best performance among counterparts for both L2 CD and EMD, using a decoder with two SPDs.Source: §4.6.3 · p. 11, Table 12 · p. 12

4.6.4 Visual comparison

  • SnowflakeNet produces more detailed geometric structures and more uniform, evenly surface-covering points than DPM in visual comparisons.Latent interpolation and extrapolation transition smoothly, including between categories, while maintaining visually uniform point distributions.Source: §4.6.4 · p. 12, Figure 14 · p. 12, Figure 15 · p. 12

4.7 Novel Shape Generation

The paper applies its decoder to novel point-cloud generation on ShapeNet using a learned latent prior. Results are competitive with recent generative methods, and generated shapes show uniformity, diversity, and enriched local geometry.

4.7.1 Dataset and evaluation metric

  • The ShapeNet evaluation compares airplane and chair generation using COV, MMD, 1-NNA, and JSD to assess coverage, fidelity, distribution similarity, and marginal point-distribution similarity.The comparison follows DPM's experimental settings and uses results for other models cited from DPM.Source: §4.7.1 · p. 12, §4.7.3 · p. 13

4.7.2 Network arrangement and training loss

  • The network arrangement matches auto-encoding and uses normalizing flows to parameterize the latent prior; generation samples Gaussian w, maps it through the flow, then passes latent f to the decoder.The flow is a trainable bijection from an isotropic Gaussian to a complex distribution.Source: §4.7.2 · pp. 12–13

4.7.3 Quantitative and qualitative results

  • SnowflakeNet is highly competitive with ShapeGF and DPM and significantly better than other compared models; it achieves the best airplane-category MMD score.The comparison includes DPM, ShapeGF, PointFlow, TreeGAN, GCN-GAN, and PC-GAN.Source: §4.7.3 · p. 13, Table 13 · p. 13
  • Generated samples show good uniformity and diversity while retaining enriched local geometric structures despite latent codes sampled randomly from a Gaussian distribution.The paper presents examples of novel generated point clouds in Fig. 16.Source: §4.7.3 · p. 13, Figure 16 · p. 13

4.8 Single Image Reconstruction

SnowflakeNet is extended to single-image reconstruction, with evaluation on ShapeNet showing the strongest reconstruction performance among the compared methods and visual results with smoother, less noisy surfaces.

4.8 Single Image Reconstruction

  • Single-image reconstruction aims to reconstruct a point cloud from an image of the underlying object.Source: §4.8 · p. 13

4.8.1 Dataset and evaluation metric

  • Evaluation uses ShapeNet’s 43,783 shapes across 13 categories, L1 Chamfer distance, 2,048 output points, and 30,000 sampled ground-truth points.The dataset split is 70% training, 20% testing, and 10% validation.Source: §4.8.1 · p. 13

4.8.2 Network arrangement and training loss

  • The reconstruction network uses a ResNet-18 encoder, maps its latent code to 128 dimensions, and uses the auto-encoding decoder arrangement.Chamfer-L1 distance is the training loss.Source: §4.8.2 · pp. 13–14

4.8.3 Quantitative comparison

  • SnowflakeNet has the best reconstruction performance among the compared methods and the best per-category CD across ShapeNet categories.Table 14 reports per-point L1 Chamfer distance; the passage does not give specific table values.Source: Table 14 · p. 14, §4.8.3 · p. 14

4.8.4 Visual comparison

  • Visual results show smoother surfaces and less noise than competing reconstructions, while Pix2Vox++ misses detail and DPM produces noise requiring postprocessing.Compared with 3DAttriFlow, SnowflakeNet reconstructs the object while recovering smooth surfaces.Source: §4.8.4 · p. 14, Figure 17 · p. 14

4.9 Point Cloud Upsampling

The section extends SPD to upsample sparse, noisy, nonuniform point clouds. It reports strong benchmark performance, including the best FPS results on both metrics and improved uniformity in visual comparisons.

4.9 Point Cloud Upsampling

  • SPD is extended to upsampling because the task generates dense, uniform point clouds from sparse, noisy, nonuniform inputs.Source: §4.9 · p. 14

4.9.3 Quantitative comparison

  • Under FPS, SPD achieves the best L2 CD and HD; under RS, its CD is close to PU-GAN and Dis-PU while its HD is better, with significantly fewer parameters.The authors also report good point uniformity despite not training with repulsion or uniform loss.Source: Table 15 · p. 14, §4.9.3 · p. 15

4.9.1 Dataset and evaluation metric

  • Testing uses the PU-GAN benchmark with 120 training and 27 testing objects; inputs are sampled by FPS or random sampling, and evaluation uses L2 CD and HD.For RS, the authors average five test runs; each test input has 2,048 points and ground truth has 8,192.Source: §4.9.1 · p. 14

4.9.2 Network arrangement and training loss

  • Four stacked SPDs use upsampling factors 1, 2, 2, and 1, followed by an additional refinement SPD; training uses L2 CD against downsampled ground truth at matching point density.Source: §4.9.2 · pp. 14–15

4.9.4 Visual comparison

  • Visual comparisons show SPD’s outputs most similar to ground truth, preserving uniform point distribution in both complex local structures and flat regions.Source: Figure 18 · p. 15, §4.9.4 · p. 15

5 CONCLUSION, LIMITATION AND FUTURE WORK

SnowflakeNet uses layered SPD and a skip-transformer to generate locally structured, detailed point clouds, with reported gains on completion benchmarks and results across additional generation tasks. The authors identify limits in long-range context and uniform splitting.

5 CONCLUSION, LIMITATION AND FUTURE WORK

  • SnowflakeNet models completion as snowflake-like point growth through multiple SPD layers, with a skip-transformer supporting locally compact, structured outputs and detailed geometry.Source: §5 · p. 15
  • Experiments report superior performance over current methods on sparse Completion3D and dense PCN, and demonstrate SPD on auto-encoding, novel generation, image reconstruction, and upsampling.The generation results are quantitative and qualitative.Source: §5 · p. 15
  • The skip-transformer aggregates k-nearest neighbors, limiting long-range shape context; with too much input noise, point splitting may be misarranged.Source: §5 · p. 15
  • SPD assigns the same number of child points to every seed despite differing local shape complexity, such as lampshades needing more points than lampposts.Adaptive splitting may require significant network changes.Source: §5 · p. 15

Related papers

Loading 2202.09367v3…