Source-linked AI summary
Neural Wavelet-domain Diffusion for 3D Shape Generation
Ka-Hei Hui, Ruihui Li, Jingyu Hu, Chi-Wing Fu
TL;DR
3D shape generation needs a compact way to model continuous implicit surfaces without the redundancy and indirectness of existing approaches. The paper uses paired wavelet coefficients with a diffusion generator and detail predictor, and reports diverse, high-quality shapes with complex topology, clean surfaces, and fine details. The authors position this approach as exceeding state-of-the-art 3D generation capabilities, while noting that detailed limitations are discussed separately.
Problem
Existing implicit-surface generation is hindered by non-compact dense representations and indirect latent-space or representation-conversion strategies.
Method
The method encodes TSDF-based implicit shapes with paired coarse and detail biorthogonal-wavelet volumes, then uses diffusion to generate coarse coefficients and a predictor to generate compatible details.
Results
The experiments report diverse, high-quality shapes with complex topology, clean surfaces, sharp boundaries, and fine details, exceeding state-of-the-art approaches.
Takeaways & Limitations
The approach demonstrates direct unconditional generative modeling of 3D implicit shapes in a compact wavelet representation without a pre-trained latent space or representation conversion.
Takeaways & Limitations
The paper directs readers to supplementary Section K for its limitations discussion.
Abstract
from arXiv · showhide
This paper presents a new approach for 3D shape generation, enabling direct generative modeling on a continuous implicit representation in wavelet domain. Specifically, we propose a compact wavelet representation with a pair of coarse and detail coefficient volumes to implicitly represent 3D shapes via truncated signed distance functions and multi-scale biorthogonal wavelets, and formulate a pair of neural networks: a generator based on the diffusion model to produce diverse shapes in the form of coarse coefficient volumes; and a detail predictor to further produce compatible detail coefficient volumes for enriching the generated shapes with fine structures and details. Both quantitative and qualitative experimental results manifest the superiority of our approach in generating diverse and high-quality shapes with complex topology and structures, clean surfaces, and fine details, exceeding the 3D generation capabilities of the state-of-the-art models.
1 INTRODUCTION
The paper targets direct, efficient generation of high-quality 3D implicit shapes, addressing limitations of conventional representations and indirect latent-space approaches. It introduces a compact wavelet representation with diffusion-based coarse generation and detail prediction, producing diverse shapes with complex structures and fine details.
- Motivation: Conventional voxel, point-cloud, and mesh representations struggle with high resolutions or irregular topology, limiting high-fidelity 3D generation.Implicit functions allow arbitrary-topology meshes at desired resolutions.
- Motivation: Direct implicit-function generation is computationally difficult because dense samples redundantly encode mostly constant or smoothly varying regions.This representation non-compactness hinders efficient generative learning on implicit surfaces.
- Motivation: Indirect latent-space learning and conversion to point clouds or voxels cannot ensure realistic implicit generation and often leave a visual-quality gap versus reconstruction.The cited approaches regularize generated implicit functions only indirectly.
- Approach: The method introduces a compact biorthogonal-wavelet representation using paired coarse and detail coefficient volumes to encode 3D shapes.The representation is designed for direct generative modeling of continuous implicit functions.
- Approach: A diffusion-based generator produces coarse coefficient volumes from noise, while a detail predictor generates compatible detail coefficients for fine structures.Together, the networks model shape distribution and enrich generated shapes with details.
- Results: The experiments report diverse and realistic shapes with diverse topology, clean surfaces, sharp boundaries, and fine details, exceeding state-of-the-art approaches.Examples include curved or thin beams, a small pulley, and complex cabinets.
2 RELATED WORK
Prior work addresses implicit shape generation through reconstruction-derived latent spaces, representation conversion, and alternative discrete representations, but efficient direct implicit-surface generation remains open. This paper instead combines compact wavelet modeling with diffusion to generate high-quality shapes without latent pretraining or conversion.
- Implicit-function generation: Implicit-function reconstruction supports flexible surfaces, but generation lacks fixed ground-truth supervision for each sample and remains an open efficiency problem.Prior generation methods often first learn a latent embedding through reconstruction.
- Wavelet-domain generation: The proposed wavelet representation and diffusion model directly synthesize implicit surfaces without a pre-trained latent space or representation conversion.This avoids the indirect route used by several prior approaches.
- Alternative representations: Voxel methods primarily learn coarse structures and miss fine details because of memory restrictions.This illustrates the trade-off between discrete grids and detailed shape synthesis.
- Alternative representations: Point-cloud methods use adversarial, flow-based, or diffusion models, but their discrete samples limit mesh quality and structural continuity.The related-work discussion contrasts these methods with continuous implicit surfaces.
- Multi-scale representations: Multi-scale neural implicit methods improve reconstruction quality or inference efficiency by predicting hierarchical latent codes or restricting frequency content.These works provide context for the paper’s frequency-domain representation.
3 OVERVIEW
The pipeline prepares compact wavelet coefficients, learns coarse and detail distributions with two networks, and reconstructs shapes from their generated coefficients. Diffusion generates the coarse structure, while a detail predictor supplies compatible fine-scale information before mesh extraction.
- Data preparation: Data preparation truncates each shape’s signed distance field and transforms it into multi-scale wavelet coefficient volumes.The compact representation retains a pair of coarse and detail volumes at the same scale.
- Pipeline: The overview separates preparation, learning, and generation into three procedures: compact encoding, two-network training, and coefficient-to-shape reconstruction.This organization corresponds to Figure 2(a–c).
- Shape learning: Shape learning trains a diffusion generator to iteratively refine random noise into coarse coefficient volumes.The generator learns diverse 3D shape distributions in the coarse representation.
- Shape learning: A detail predictor learns to produce detail coefficient volumes conditioned on coarse coefficient volumes.Its role is to introduce further detail into generated shapes.
- Shape generation: The generation pipeline applies the generator and detail predictor sequentially, then uses an inverse wavelet transform and marching cubes to obtain the output shape.It starts from a random Gaussian noise sample and produces associated coarse and detail coefficients.
4 METHOD
The method compresses implicit 3D shapes into multi-scale wavelet coefficients and learns to generate coarse coefficients with diffusion before predicting compatible details. Truncated signed distance fields and biorthogonal filters preserve structure while reducing redundancy and artifacts.
- Compact Wavelet Representation: Each shape is represented by a truncated signed distance field, then decomposed into coarse and detail wavelet coefficient volumes at multiple scales.The TSDF clips regions far from the surface, while wavelet decomposition separates scale-specific shape information.
- Compact Wavelet Representation: At scale J=3, reconstruction changes TSDF values by only 2.8% for ShapeNet Chair despite omitting 97% of coefficients.This motivates retaining a compact coarse-scale representation while dropping higher-scale detail volumes during representation construction.
- Compact Wavelet Representation: Biorthogonal wavelets with synthesis and analysis vanishing moments of six and eight are chosen to produce smoother decompositions than Haar wavelets.The method also computes detail coefficients as a difference rather than storing seven traditional detail channels.
- Shape Learning: The generator learns a diffusion sequence that denoises Gaussian noise volumes into coarse coefficient volumes representing generated shapes.The forward process adds Gaussian noise, while the learned backward process iteratively removes it; the generator is trained with a mean-squared noise-prediction loss.
- Shape Learning: A detail predictor conditionally maps generated coarse coefficients to compatible detail coefficients, enriching the resulting shapes with fine structures.It is trained by regression on paired coarse and detail coefficient volumes and uses additional convolution layers for the larger detail representation.
5 RESULTS AND EXPERIMENTS
Experiments show that the method generates high-quality, diverse 3D shapes across categories, outperforming state-of-the-art methods on most quantitative cases and producing detailed, clean, and novel structures.
- 5.1 Galleries of our generated shapes: The method generates diverse topologies, fine details, clean surfaces, and complex structures across multiple shape categories without obvious artifacts.These properties include small, thin, and structurally complex forms that are challenging for existing approaches.
- 5.2 Comparison with Other Methods: Table 1 evaluates Chair and Airplane generation using MMD, COV, and 1-NNA under the same setting as prior methods.The experiment generates 2,000 shapes per category, samples 2,048 points per shape, and uses publicly released comparison models.
- 5.2 Comparison with Other Methods: Visual comparisons show finer details, cleaner surfaces, and no obvious artifacts relative to state-of-the-art methods.The comparisons match generated shapes with similar structures and topology to facilitate visual assessment.
- 5.2 Comparison with Other Methods: The method surpasses competing approaches in almost all evaluation cases for both Chair and Airplane categories.It achieves the lowest MMD, largest COV, and lowest 1-NNA in almost all cases; 1-NNA is emphasized as more robust for fine generation quality.
- 5.3 Model Analysis: Novelty analysis shows that generated shapes retain realistic similarities to training examples while differing in local structures and spanning low- to high-LFD relationships.For 500 generated chairs, the LFD distribution covers shapes similar to the training set and also more novel shapes.
- 5.3 Model Analysis: Ablation results show that the detail predictor substantially improves all metrics, while replacing the diffusion generator or directly predicting TSDF degrades performance.The comparison is conducted on the Chair category using the full pipeline and multiple ablated cases.
6 CONCLUSION
The paper presents a compact wavelet-based representation and neural generation approach for diverse, high-quality 3D shapes. It identifies extensions to conditional shape tasks, animation, finer-detail objects, and 3D scenes as future directions.
- The method decomposes TSDF-based implicit shapes with biorthogonal wavelets into coarse and detail coefficient volumes.
- A diffusion generator produces coarse coefficients from noise, while a detail predictor reconstructs compatible fine details.
- Quantitative and qualitative experiments show diverse, realistic shapes with fine details, complex thin structures, and clean surfaces.
- The compact wavelet representation is presented as enabling future conditional tasks such as image- or point-cloud-based reconstruction and shape editing.
- Future directions include spatio-temporal wavelet representations for animation, objects with extremely fine details, and 3D scenes.