PaperlayerView exact revision on arXiv

Source-linked AI summary

LaS-Comp: Zero-shot 3D Completion with Latent-Spatial Consistency

Weilong Yan, Haipeng Li, Hao Xu, Nianjin Ye, Yihao Ai, Shuaicheng Liu, Jingyu Hu

arXiv:2602.18735v2cs.CVcs.RO

TL;DR

Existing 3D shape completion methods struggle with diverse partial patterns and object categories. LaS-Comp is a zero-shot, category-agnostic framework using two complementary stages and 3D foundation-model priors, and it surpasses existing approaches across diverse benchmarks, categories, and partial patterns.

  • Problem

    Existing shape completion methods have limited generalization across diverse partial patterns and broad object categories, despite the need for robust reconstruction from partial observations.

  • Method

    LaS-Comp uses a training-free two-stage design that explicitly preserves observed geometry and implicitly aligns boundaries while leveraging compatible 3D foundation models.

  • Results

    LaS-Comp surpasses existing approaches across diverse object categories, partial patterns, Omni-Comp, and standard benchmarks.

  • Takeaways & Limitations

    Omni-Comp enables more comprehensive in-the-wild evaluation of zero-shot 3D shape completion using real-world scans and synthetic shapes with diverse partial patterns.

  • Takeaways & Limitations

    Extremely noisy inputs remain challenging, causing fine details and thin structures to be over-smoothed or distorted when geometry is heavily corrupted.

Abstract

from arXiv · show

This paper introduces LaS-Comp, a zero-shot and category-agnostic approach that leverages the rich geometric priors of 3D foundation models to enable 3D shape completion across diverse types of partial observations. Our contributions are threefold: First, \ourname{} harnesses these powerful generative priors for completion through a complementary two-stage design: (i) an explicit replacement stage that preserves the partial observation geometry to ensure faithful completion; and (ii) an implicit refinement stage ensures seamless boundaries between the observed and synthesized regions. Second, our framework is training-free and compatible with different 3D foundation models. Third, we introduce Omni-Comp, a comprehensive benchmark combining real-world and synthetic data with diverse and challenging partial patterns, enabling a more thorough and realistic evaluation. Both quantitative and qualitative experiments demonstrate that our approach outperforms previous state-of-the-art approaches. Our code and data will be available at \href{https://github.com/DavidYan2001/LaS-Comp}{LaS-Comp}.

1. Introduction

LaS-Comp is a zero-shot, category-agnostic framework that uses complementary latent–spatial stages and 3D foundation-model priors for faithful completion across diverse partial patterns. The introduction also presents Omni-Comp, a benchmark designed for more comprehensive real-world evaluation.

  • Capabilities: LaS-Comp supports category-agnostic completion across random crops, single-view scans, and missing semantic parts, with unconditional or text-guided control.Text guidance uses the foundation models’ built-in classifier-free guidance mechanism.
  • LaS-Comp: LaS-Comp bridges latent and spatial domains through a two-stage design for category-agnostic 3D shape completion.The Explicit Replacement Stage (ERS) injects partial geometry into the latent representation, while the Implicit Alignment Stage (IAS) preserves boundary coherence.
  • LaS-Comp: The framework is zero-shot, training-free, and compatible with different latent-generative-based 3D foundation models.It exploits their geometric priors and supports diverse partial patterns, including single-view scans and irregularly missing regions.
  • Omni-Comp: Omni-Comp combines real-world scans and synthetic data with diverse, challenging partial patterns for more comprehensive in-the-wild evaluation.Existing benchmarks are described as mostly limited to single-view partial scans and insufficiently representative of real-world diversity.

2. Related Work

Related work progresses from supervised paired-data reconstruction to unsupervised and self-supervised alternatives, then to category-agnostic completion using pretrained generative priors. However, existing learning-based methods remain limited by restricted training categories and weaker real-world or unseen-category performance.

  • Supervised Shape Completion: Supervised methods learn partial-to-complete 3D reconstruction from paired datasets, with later work using coarse-to-fine refinement to recover missing and fine geometry.Pioneering methods established paired datasets and end-to-end mappings, while subsequent approaches refined reconstructions progressively.
  • Unsupervised Shape Completion: Unsupervised methods reduce reliance on paired data through unpaired training, adversarial losses, cycle consistency, and latent-space alignment.Recent self-supervised approaches further relax the need for complete shapes.
  • Unsupervised Shape Completion: ShapeNet-trained methods can degrade on real-world data or unseen categories because ShapeNet contains limited categories.This limitation motivates category-agnostic approaches based on broader priors.
  • Shape Completion with Generative Priors: Recent category-agnostic methods leverage large-scale pretrained generative models, including Stable Diffusion-based signed-distance optimization and Zero-1-to-3-guided multi-view refinement.SDS-Complete distills 2D priors to optimize signed distance fields, while ComPC initializes partial inputs as 3D Gaussians and iteratively refines rendered views.

3. Method

LaS-Comp performs zero-shot, category-agnostic 3D completion by iteratively denoising latent features conditioned on a partial shape. Its two complementary stages explicitly preserve observed geometry and implicitly align synthesized regions for coherent completion.

  • Method Overview: LaS-Comp leverages pretrained 3D foundation-model priors for zero-shot completion through iterative denoising guided by the partial input.Classifier-free guidance supports both text-guided and unconditional shape completion.
  • Method Overview: After the final denoising iteration, the refined latent feature is decoded to produce the completed shape.The decoder D maps x0 to Sc.
  • Explicit Replacement Stage: The Explicit Replacement Stage injects the partial shape into the latent process using spatial masking, directly imposing observed geometry while generatively completing missing regions.The partial shape is voxelized into an aligned occupancy grid, replaced in the predicted complete shape, and re-encoded into a latent feature.
  • Explicit Replacement Stage: ERS decomposes generation into clean and noisy branches, combining fidelity enforcement with diversity enhancement through a partial-aware noise schedule.The schedule stabilizes observed regions with limited, time-dependent noise while allowing greater stochastic exploration in missing regions.
  • Implicit Alignment Stage: The Implicit Alignment Stage reduces replacement-induced boundary artifacts by applying a geometry-alignment BCE loss within masked regions and a single latent-feature gradient update.The refined latent is used to compute the next denoising state rather than updating model parameters.

4. Experiments

Experiments across standard and challenging completion benchmarks show that LaS-Comp achieves strong correctness, fidelity, generalization, and diversity. Ablations further show that explicit geometric conditioning is essential to preserve observed structure and obtain the best overall performance.

  • Evaluation Datasets: Omni-Comp evaluates 30 objects from distinct categories across real-world Redwood and YCB scans and synthetic shapes, using diverse partial patterns.The benchmark is designed to assess generalization across real-world and synthetic scenarios with challenging partial observations.
  • Completion Correctness: Qualitative Redwood results preserve observed geometry while completing missing regions with precise surface geometry and coherent topology.Examples include plant leaf structures and trash-bin rims and wheels; competing methods distort visible geometry or produce over-smoothed, implausible structures.
  • Completion Fidelity: LaS-Comp outperforms ComPC [31] by a large margin in almost all categories on sparse KITTI LiDAR scans and noisy ScanNet depth scans.The result confirms robustness and completion fidelity on challenging real-world partial inputs.
  • Evaluation on Omni-Comp: On Omni-Comp, LaS-Comp improves over ComPC [31] by 49.6% in CD and 39.4% in EMD on average across partial patterns.Unlike methods optimized for single scans, it maintains robust performance across categories and patterns, including challenging contours, random crops, and semantic parts.
  • Ablation Studies: Ablations show that removing the Explicit Replacement Stage causes the most significant performance drop, while the full pipeline achieves the best performance.The naive latent-only baseline performs poorly because corresponding partial and ground-truth regions have latent discrepancy; ten IAS optimization steps provide no obvious gain over the full setting.

5. Conclusion

LaS-Comp is a zero-shot, category-agnostic 3D shape completion framework using 3D foundation-model priors, combining explicit replacement and implicit alignment for faithful, globally coherent reconstruction. The paper also introduces Omni-Comp, a benchmark spanning real-world scans, synthetic shapes, and diverse partial patterns.

  • LaS-Comp leverages 3D foundation-model geometric priors for zero-shot, category-agnostic 3D shape completion.
  • Its Explicit Replacement Stage and Implicit Alignment Stage jointly target high-fidelity reconstruction and global geometric coherence.
  • Omni-Comp combines real-world scans and synthetic shapes with diverse partial patterns for comprehensive evaluation.
  • The supplementary material analyzes the latent–spatial gap, evaluation metrics, implementation details, Omni-Comp, qualitative visualizations, and limitations with future directions.
  • Observed regions can share surface geometry while exhibiting obvious differences between partial-input and ground-truth VAE latents.

6. More Discussions about the Latent-Spatial Gap

The discussion identifies a substantial latent-spatial gap between partial inputs and ground truth in observed regions, making partial latent codes unreliable for completion guidance. LaS-Comp therefore operates in geometric space, explicitly reinserting known partial points to preserve observed geometry and impose a stronger constraint.

  • Latent-Spatial Gap: Partial-input and ground-truth VAE latents differ significantly within the observed regions.The comparison uses a latent mask downsampled from the 64^3 voxel space to the 16^3 latent space.
  • Latent-Spatial Gap: Limited correlation makes direct reliance on partial latent codes fundamentally fragile for completion guidance.This finding is also supported by the ablation study in the main paper.
  • Geometric-Space Completion: The method instead projects back to geometric space and explicitly injects known partial points into the decoded space.This preserves the observed geometry and provides a stronger, more reliable constraint for completion.

7. Details of the Evaluation Metrics

The evaluation uses distance-based metrics to assess completion quality, observed-shape preservation, fidelity, and diversity. Metrics are reported with task-specific scaling factors, and EMD uses fixed computation settings.

  • Distance Metrics: Chamfer Distance (CD) uses symmetric nearest-neighbor distances between predicted and ground-truth point clouds to measure support and surface coverage, reported by ×102.Its two terms evaluate predicted-point support by the ground truth and prediction coverage of the ground-truth surface.
  • Distance Metrics: Earth Mover’s Distance (EMD) measures the minimum average transport cost between uniformly resampled predicted and ground-truth sets, reported by ×102.The computation uses bijective matching with eps = 0.005 and 50 iterations.
  • Partial-Input Consistency: Unidirectional Chamfer Distance (UCD) measures squared L2 distance from the partial input to the completion, while Unidirectional Hausdorff Distance (UHD) measures its worst-case observed-point distance.UCD is reported by ×104, whereas UHD is reported by ×102.
  • Set-Level Metrics: Minimum Matching Distance (MMD) averages, over ground-truth shapes, the closest generated completion under Chamfer Distance, measuring generated-set fidelity and reported by ×102.For each ground-truth shape, the closest generated completion is selected under CD.
  • Set-Level Metrics: Total Mutual Difference (TMD) averages pairwise Chamfer distances among multiple completions for one partial input, with larger values indicating greater diversity and reporting by ×102.The metric evaluates differences among K generated completion hypotheses.

8. More Implementation Details

LaS-Comp is compatible with 3D backbones exposing spatial-to-latent encoding, latent generation, and latent-to-spatial decoding, instantiated with Direct3D-S2 and TRELLIS. The implementation specifies common voxelization, optimization, dataset-resolution, and hardware settings.

  • Backbone compatibility: LaS-Comp requires backbones with an encoder E, generative process G, and decoder D, and is instantiated with Direct3D-S2 and TRELLIS.These components represent spatial-to-latent encoding, latent generation, and latent-to-spatial decoding.
  • Backbone compatibility: Direct3D-S2 uses a VAE encoder, conditional diffusion transformer, and VAE decoder, while TRELLIS uses its VAE encoder, first-stage rectified-flow transformer generator, and VAE decoder.Both backbones use a 643 voxel-grid resolution, and inputs are voxelized before processing.
  • Optimization settings: The experiments use 100 denoising steps, CFG scale 1.0, t rescale factor 3.0, and one latent optimization update per step with learning rate 1×10−5.The occupancy threshold is 0.5, while IAS optimizes binary cross-entropy directly on decoder logits.
  • Data preprocessing: Point-cloud resolution is 16,384 for Redwood, synthetic datasets, and Omni-Comp, versus 2,048 for KITTI and ScanNet.These settings follow prior experimental configurations.
  • Efficiency: Under FP16 with batch size 1, the method uses 10.29 GB peak VRAM and runs in 35.92s on a single RTX3090.The reported configuration demonstrates practical deployability.

9. Details of the Omni-Comp Benchmark

Omni-Comp is a comprehensive benchmark for robust 3D shape completion evaluation, combining 30 diverse objects from real-world and synthetic sources. Its construction emphasizes complex geometry, training-data separation, and standardized preprocessing and resolution.

  • Benchmark composition: Omni-Comp contains 30 objects from distinct categories: 10 Redwood scans, 10 YCB everyday objects, and 10 synthetic shapes.The sources are selected for complex geometry, downstream relevance, and rich semantic structure, respectively.
  • Data selection: Real-world samples avoid near-cuboidal geometries, while synthetic identities are excluded from Objaverse-XL metadata to prevent training-data overlap.The selection prioritizes richer structures and reduces overlap with generative-backbone training data.
  • Preprocessing: For single-scan samples without masks, SAM2 and OWL-ViT extract masks from registered RGB-D data before depth back-projection into 3D space.The preprocessing then manually aligns provided ground-truth point clouds with the backprojected partial point clouds, following SDS-Comp.
  • Preprocessing: All partial and ground-truth data are sampled at a resolution of 16,384, following prior work.Additional visualizations cover diverse objects and partial patterns.

10. More Visualization Results

Additional visualizations show that LaS-Comp produces detailed, semantically plausible completions across challenging real-world scans, diverse synthetic geometries, and Omni-Comp’s partial patterns and categories. It also generates multiple distinct hypotheses for a fixed partial input while remaining consistent with the observed geometry in unconditional and text-guided settings.

  • Redwood dataset: On challenging Redwood scans with sensor noise, self-occlusions, and depth discontinuities, LaS-Comp consistently recovers geometrically detailed and semantically plausible completions across instances and viewpoints.The examples are visualized as point clouds in Fig. 13.
  • Synthetic data: On synthetic partial point clouds, LaS-Comp accurately completes thin structures, high-curvature regions, and complex topologies with rich geometric details and coherent semantics.These examples use cleaner sampling and more diverse geometric patterns than the real-world scans.
  • Omni-Comp benchmark: Across Omni-Comp’s random-crop, single-scan, and semantic-part settings and broad category range, LaS-Comp produces detailed, semantically meaningful completions that respect the observed geometry.Additional examples are shown in Figs. 15–17.
  • Completion diversity: For a fixed partial point cloud, LaS-Comp generates multiple distinct full-shape hypotheses consistent with the observed geometry under unconditional and text-guided completion.The visualizations illustrate the inherent ambiguity of the completion task in Fig. 18.

11. Limitations and Future Work

LaS-Comp remains challenged by extremely noisy or ambiguous inputs, where it can recover coarse structure but may lose fine details and thin structures. Future work targets noise-aware preprocessing, confidence-aware refinement, and completion without ground-truth shapes for in-the-wild scans.

  • Limitations: Extremely noisy inputs can preserve coarse object structure but over-smooth or distort fine details and thin structures.Severe corruption of the observed geometry limits faithful completion, as illustrated in Fig. 10.
  • Future Work: Future work will explore confidence-aware refinement that adapts denoising strength by regional reliability, preserving confident areas while applying cautious updates elsewhere.This strategy is intended for weak or ambiguous input cues.
  • Future Work: Future work will develop stronger outlier removal to detect scattered points and locally inconsistent clusters before completion.The goal is to expose cleaner partial inputs for shape completion.
  • Future Work: The method currently adopts GT-based normalization, limiting applicability when real-world scans are available but ground-truth shapes are missing.The authors plan more robust methods for in-the-wild 3D completion.
Loading 2602.18735v2…