Source-linked AI summary
SeedFormer: Patch Seeds based Point Cloud Completion with Upsample Transformer
Haoran Zhou, Yun Cao, Wenqing Chu, Junwei Zhu, Tong Lu, Ying Tai, Chengjie Wang
TL;DR
Point cloud completion seeks to recover complete shapes from sparse partial observations, where global representations can lose local details and omit unseen information. SeedFormer introduces Patch Seeds and an Upsample Transformer for coarse-to-fine generation, and evaluations report clear improvements over state-of-the-art methods on several benchmark datasets.
Problem
Point cloud completion must recover complete 3D shapes from sparse, incomplete observations, while global features can lose fine details and represent only observed information.
Method
SeedFormer represents shapes with regional Patch Seeds and uses an Upsample Transformer to aggregate neighboring features during coarse-to-fine point generation.
Results
SeedFormer achieves clear improvements over state-of-the-art completion methods on several challenging benchmark datasets.
Takeaways & Limitations
Patch Seeds preserve regional information for detail recovery, while the Upsample Transformer captures neighborhood information during point generation.
Abstract
from arXiv · showhide
Point cloud completion has become increasingly popular among generation tasks of 3D point clouds, as it is a challenging yet indispensable problem to recover the complete shape of a 3D object from its partial observation. In this paper, we propose a novel SeedFormer to improve the ability of detail preservation and recovery in point cloud completion. Unlike previous methods based on a global feature vector, we introduce a new shape representation, namely Patch Seeds, which not only captures general structures from partial inputs but also preserves regional information of local patterns. Then, by integrating seed features into the generation process, we can recover faithful details for complete point clouds in a coarse-to-fine manner. Moreover, we devise an Upsample Transformer by extending the transformer structure into basic operations of point generators, which effectively incorporates spatial and semantic relationships between neighboring points. Qualitative and quantitative evaluations demonstrate that our method outperforms state-of-the-art completion networks on several benchmark datasets. Our code is available at https://github.com/hrzhou2/seedformer.
1 Introduction
Point cloud completion must recover complete 3D shapes from sparse, incomplete observations, but global-feature encoder-decoder methods lose fine details and represent only observed information. SeedFormer addresses these limitations with Patch Seeds and an Upsample Transformer for coarse-to-fine detail recovery.
- Raw point clouds are often sparse and incomplete because of limited sensor resolution and self-occlusion, making completion important for downstream 3D applications.
- SeedFormer is reported to improve existing-structure preservation and missing-detail recovery compared with GRNet and SnowflakeNet in visual comparisons.
- Global-feature encoder-decoder architectures lose fine-grained details during pooling and encode only incomplete information from the observed part.
- Point generators must model spatial and semantic relationships among neighboring points because generated 3D coordinates are unstructured but locally related.
- SeedFormer uses Patch Seeds to preserve regional information while representing complete shape structure, then recovers details in a coarse-to-fine decoding process.
- The Upsample Transformer aggregates neighboring point features through transformer-style attention during point generation to capture local geometric features.
2 Related Work
Prior point cloud completion research progressed from voxel and global-feature representations toward direct point manipulation and increasingly local or transformer-based generators. SeedFormer extends this trajectory with Patch Seeds propagated through detail-recovery layers.
- Voxelization-based completion applies CNNs to voxel grids but suffers information loss and rapidly increasing computational cost at higher resolutions.
- Direct point-cloud methods avoid intermediate voxel representations, with PCN using an encoder-decoder architecture whose global feature represents the input shape.
- Point generators evolved from fully connected coordinate prediction to folding operations and SnowflakeNet's deconvolution-based point splitting.
- Compared with PoinTr's local proxies for feature translation, SeedFormer propagates Patch Seeds into subsequent upsample layers for recovering fine details.
3 Method
SeedFormer represents incomplete shapes with Patch Seeds and progressively completes them through seed generation and coarse-to-fine upsampling. Its Upsample Transformer aggregates local spatial and semantic information to generate and refine points.
- Seed generator: The encoder extracts local patch features and centers from the incomplete point cloud using point transformer and set abstraction layers.These features represent local structures before seed generation.
- Seed generator: The seed generator predicts a coarse yet complete shape, producing seed points and features that are grouped into the Patch Seeds representation.The generated seeds are intended to cover the complete shape, including unseen regions.
- Coarse-to-fine generation: Several upsample layers progressively densify the point cloud, while interpolated seed features are propagated into subsequent layers.Each input point is expanded into r_l points through coarse-to-fine generation.
- Patch Seeds: Patch Seeds combine seed coordinates and features, with each seed covering a local region and encoding semantic clues for detail recovery.This representation preserves regional information that can support recovering ambiguous missing details.
- Upsample Transformer: The Upsample Transformer uses channel-wise local attention, kernel-specific geometric patterns, and positional encoding to aggregate neighboring point features.Its attention combines query, key, and positional information, while generated features support skip connections and displacement-based point refinement.
- Seed generator: In the seed generator, softmax normalization can be removed so attention weights are not restricted to (0, 1), supporting seed placement beyond local neighborhoods.This variant differs from the upsample layers, which use skip connections and seed features.
4 Evaluation
SeedFormer is evaluated on synthetic and real-scanned completion benchmarks, with comparisons spanning standard metrics, qualitative detail recovery, and architecture ablations. It achieves strong benchmark performance while Patch Seeds and Upsample Transformer designs improve completion quality in controlled studies.
- PCN: On PCN, SeedFormer achieves the best scores across all categories using per-point L1 Chamfer Distance ×1000.The dataset uses 2,048-point partial inputs and 16,384-point complete ground truth sampled from ShapeNet surfaces.
- Qualitative results: Qualitative PCN results show preserved chair-arm and chair-back details without undesired components.Additional visual comparisons report SeedFormer performing clearly better than PCN, GRNet, and SnowflakeNet.
- ShapeNet-55/34: SeedFormer achieves the best scores across all ShapeNet-55 categories and evaluation metrics.The evaluation includes L2 Chamfer Distance at three difficulty levels, overall CD, category results, and F-Score@1%.
- ShapeNet-55/34: 15.6% lower overall CD and 16.8% lower average CD on hard difficulty are reported against PoinTr on ShapeNet-55.On ShapeNet-34 unseen objects, SeedFormer reaches an average CD of 1.34, 34.6% lower than PoinTr.
- ShapeNet-55/34: SeedFormer again achieves the best scores on ShapeNet-34 test sets, including unseen categories.The unseen-category result is an average CD of 1.34, reported as better generalization ability than PoinTr.
- KITTI: KITTI evaluation uses Fidelity Distance and Minimal Matching Distance because real LiDAR scans are sparse and lack complete ground truth.The model is fine-tuned on ShapeNetCars before comparison on 2,401 extracted car point clouds.
- Ablation studies: Patch Seeds improve the ablation score from 6.97 to 6.74, and 256 seed points are reported as suitable for covering an input object.The ablation compares the Patch Seeds representation with a global-feature design under identical training settings.
- Ablation studies: Local aggregation improves point generation over folding and deconvolution, with Upsample Transformer performing best among similar designs.The study also reports that standard softmax is not optimal in the seed generator, where the modified design scores 6.83 without softmax.
5 Visualization of Patch Seeds
Patch Seeds are visualized as 256 generated seed points derived from 128 input patches, with seed assignments revealing patch-to-patch generation paths. The visualization compares inputs, seeds, colored predictions, and ground truth.
- Patch Seeds: The seed generator takes 128 input patches and produces 256 seed points.Each patch center is split into two seeds through different groups of self-attention weights.
- Visualization sequence: The visualization sequence compares the partial input, extracted patch centers, generated seeds, seed-colored predictions, and ground truth.Predicted points are colored according to their nearest seed.
- Patch Seeds: Fig. 6 colors the two seed groups red and blue, enabling visual tracking of seed-generation paths.The groups are labeled seed0 and seed1 and arise from patch-to-patch translation rather than a single global feature.
6 Conclusion
SeedFormer introduces Patch Seeds to preserve global structure and fine-grained regional details, while its Upsample Transformer incorporates neighborhood information during point generation. Experiments report clear improvements over state-of-the-art methods on challenging benchmark datasets.
- Patch Seeds capture global shape structure and fine-grained local details through regional features stored in local seeds.The representation is designed to improve shape recovery and detail preservation in generated point clouds.
- The Upsample Transformer extends transformer structures into point generation to capture useful neighborhood information.
- SeedFormer achieves clear improvements over state-of-the-art competitors on several challenging benchmark datasets.
A Implementation Details
The implementation extracts patch centers and features from 2,048-point inputs, then uses them in the seed generator and configures dataset-specific upsampling rates. Input sizes are normalized through duplication or subsampling when necessary.
- Encoder: The encoder downsamples 2,048-point inputs and outputs 128 patch centers with corresponding 256-channel patch features.The architecture uses point transformer and set abstraction layers with FPS downsampling.
- Seed Generator: The seed generator takes patch centers and features as inputs, using projected patch features for transformer keys and queries.Positional encoding is calculated without incorporating seed features before seed features are obtained.
- Experimental settings: PCN uses upsampling rates r1 = 1, r2 = 4, r3 = 8, while ShapeNet-34/55 uses r1 = 1, r2 = 4, r3 = 4.Each dataset input contains 2,048 points; smaller inputs are duplicated and larger inputs are subsampled.
B Ablation Studies
The ablation studies examine local-aggregation generators, including graph-convolution and point-wise-attention alternatives to the default Upsample Transformer. These alternatives remain effective, with point-wise attention offering faster computation while outperforming state-of-the-art methods.
- Ablation generator designs: Two optional local-aggregation generators are more effective than previous methods and remain decent alternatives to the default Upsample Transformer.Both options are evaluated in Table 6 and are described as more efficient than the default design.
- Graph-convolution generator: Graph convolutions aggregate local neighborhood information before shared MLPs learn displacement offsets for newly generated points.
- Point-wise attention: Point-wise attention assigns one-dimensional weights to neighboring points and combines them with duplicated values to generate point features.This design differs from the Upsample Transformer by using point-wise attention for more efficient computation.
- Point-wise attention: Point-wise attention is faster than the default transformer structure while still outperforming state-of-the-art methods.
C Complexity Analysis
The complexity analysis compares model parameters, theoretical FLOPs, and reference Chamfer Distances on PCN. SeedFormer has a favorable model size, but its local aggregation raises computational cost; a faster variant offers a cost-performance trade-off.
- Complexity analysis: Table 9 evaluates models on PCN using parameter counts, theoretical FLOPs, and average Chamfer Distances.The evaluation uses 16,384-point outputs and batch size 1.
- Complexity analysis: SeedFormer has a favorable model size because its seed generator and upsample layers process each point with a shared generator.
- Complexity analysis: Upsample Transformer local-point aggregation makes SeedFormer relatively computationally costly, though its cost is comparable with GRNet and FoldingNet.
- Complexity analysis: The faster point-wise-attention version achieves a result of 6.85 with lower computational cost, providing a trade-off between cost and performance.
D Additional Experimental Results
Additional experiments report visual and quantitative comparisons for SeedFormer on PCN and ShapeNet benchmarks. SeedFormer performs clearly better than prior methods on PCN visual results and achieves the best scores across ShapeNet-34 novel categories.
- SeedFormer performs clearly better than PCN, GRNet, and SnowflakeNet in visual comparisons on the PCN dataset.The comparison uses released pretrained models for all methods.
- SeedFormer achieves the best scores on all ShapeNet-34 novel categories.The models are evaluated under simple, moderate, and hard difficulty levels.
- ShapeNet-55 detailed results are reported in Table 11 across simple, moderate, and hard difficulty levels.S., M., and H. denote the simple, moderate, and hard settings.