Source-linked AI summary
Efficient Part-level 3D Object Generation via Dual Volume Packing
Jiaxiang Tang, Ruijie Lu, Zhaoshuo Li, Zekun Hao, Xuan Li, Fangyin Wei, Shuran Song, Gang Zeng, Ming-Yu Liu, Tsung-Yi Lin
TL;DR
Fused 3D outputs and fixed-length latent codes leave part-level generation with the challenge of representing editable objects with varying numbers of parts. The paper introduces image-conditioned dual-volume generation, reporting robust, diverse, and structurally consistent results compared with existing approaches.
Problem
Existing methods often produce fused meshes, while fixed-length latent codes make objects with varying numbers of parts difficult to represent.
Source: §1 · p. 1, §2.2 · p. 3Method
The framework uses dual-volume packing and directly generates an arbitrary number of semantically meaningful parts from a single image without segmentation priors.
Source: §1 · p. 2Results
The method achieves more robust, diverse, and structurally consistent part-level generation than existing approaches, with a simpler and more efficient pipeline.
Source: §1 · p. 2Takeaways & Limitations
The framework is a step toward editable, structured 3D content for downstream applications.
Source: §5 · p. 10Part granularity is unpredictable because extraction relies on scene-graph information, and two volumes cannot represent three mutually contacting parts.
Source: §4.5 · p. 9
Abstract
from arXiv · showhide
Recent progress in 3D object generation has greatly improved both the quality and efficiency. However, most existing methods generate a single mesh with all parts fused together, which limits the ability to edit or manipulate individual parts. A key challenge is that different objects may have a varying number of parts. To address this, we propose a new end-to-end framework for part-level 3D object generation. Given a single input image, our method generates high-quality 3D objects with an arbitrary number of complete and semantically meaningful parts. We introduce a dual volume packing strategy that organizes all parts into two complementary volumes, allowing for the creation of complete and interleaved parts that assemble into the final object. Experiments show that our model achieves better quality, diversity, and generalization than previous image-based part-level generation methods.
1 Introduction
Existing methods often produce fused shapes or rely on segmentation followed by sequential part completion, while part-level generation must handle varying part counts. This work directly generates semantically meaningful parts using dual-volume packing, and reports robust, diverse, structurally consistent results.
1 Introduction
- Fused shapes limit part-level editing and manipulation, while generating complete, meaningful parts requires modeling global layout and local interactions.These applications include game development and robotics.Source: §1 · p. 1
- Segmentation-then-completion methods rely on external priors and process parts sequentially, making inference time grow linearly with part count.Segmentation errors can also negatively affect final generation quality.Source: §1 · p. 1
- Dual-volume packing converts the part-connectivity graph into a bipartite graph through heuristic edge contraction, preserving separation between contacting parts.Packing disjoint parts together supports fixed-length output and avoids sequential processing.Source: §1 · p. 2
- The method achieves more robust, diverse, and structurally consistent part-level generation than existing approaches, with a simpler and more efficient pipeline.The framework generates an arbitrary number of semantically meaningful parts from a single image without segmentation priors.Source: §1 · p. 2
2 Related Work
Related work spans 3D latent denoising models and approaches to part-level generation. Existing methods face challenges from fused outputs, variable part counts, costly mesh-token generation, or segmentation-dependent pipelines.
2.1 3D Denoising Generative Models
- Latent denoising models use compact VAE representations to support efficient 3D generation, and this work extends that line to part-level generation.Earlier uncompressed representations can generalize poorly on small or sparse datasets.Source: §2.1 · p. 3
2.2 Part-level 3D Generation
- Latent denoising models commonly output fused meshes, while their fixed-length codes make objects with varying part counts difficult to represent.Autoregressive mesh methods accommodate variable-length outputs but can be computationally expensive for complex meshes with many faces.Source: §2.2 · p. 3
- Segmentation-then-completion pipelines use 2D priors and iterative per-part processing, motivating direct generation of part-level meshes.The cited approaches include multi-view segmentation and completion, 3D part segmentation, and per-part latent completion.Source: §2.2 · p. 3
- Dual volume packing places parts into two volumes so that parts within each volume do not contact and can be separated during mesh extraction.The packing is based on converting the part-connectivity graph into a bipartite graph.Source: Figure 2 · p. 4
3 Methodology
The framework represents contacting and disjoint parts through dual-volume packing, then curates part-level training data and jointly generates two latent codes from an input image. Decoded parts are assembled into the complete shape.
3 Methodology
- The model takes a single-view image and predicts separable, complete parts for assembly rather than a single fused mesh.Its methodology covers dual-volume packing, part-level data curation, and model training.Source: §3 · p. 4
3.1 Dual Volume Packing
- Contacting parts fuse in a single SDF volume, whereas disjoint parts remain separate mesh components and can be isolated.Graph vertex coloring assigns contacting parts different colors and packs parts with the same color together.Source: §3.1 · p. 4
- Two volumes provide fixed-length output for bipartite connectivity graphs; additional volumes can reduce space utilization and complicate learning through multiple valid colorings.For non-bipartite graphs, the method contracts edges to eliminate odd cycles, greedily choosing the edge with greatest penetration depth and repeating until none remain.Source: §3.1 · pp. 4–5
3.2 Part-level Data Curation
- Part data is extracted from mesh scene graphs or connected components, then repaired by stitching boundary loops to improve SDF balance.Empirical merging rules address seams, small components, and components with high intersection-over-union.Source: §3.2 · p. 5
- Filtering removes samples with highly unbalanced occupancy across the two packed volumes because they are difficult to learn and less useful for training.The filtering criterion compares the occupancy ratios of the two volumes.Source: §3.2 · p. 5
3.3 Dual latent Generation
- A VAE encodes packed volumes, DINOv2 extracts image features for cross-attention, and a rectified flow model denoises the latent codes conditioned on those features.The VAE uses uniform and salient-edge point samples and supports multiple latent code sizes.Source: §3.3 · pp. 5–6
- The flow model jointly denoises two latent codes, adds a learnable part embedding to the second code to reduce duplicate parts, and decodes them into volumes of disjoint parts.The decoded parts are assembled into the complete shape.Source: §3.3 · p. 6, Figure 3 · p. 6
4 Experiments
The experiments assess image-to-3D quality, part-level generation, diversity, and speed, then examine training-design ablations and method limitations. The results report competitive image alignment and direct part generation, while identifying constraints in part granularity and two-volume packing.
4.1 Implementation Details
- Training uses filtered part-level data after VAE and flow-model pretraining, followed by VAE and flow-model fine-tuning for part-level generation.The dataset is filtered to around 254K meshes; flow-model fine-tuning uses a latent dimensionality of 4096 × 2 = 8192.Source: §4.1 · p. 6
4.2 Qualitative Comparisons.
- Qualitative comparisons show competitive or superior image-to-3D quality, more reasonable directly generated parts, and diverse outputs from different random seeds.The part-level comparison contrasts direct part generation with baselines requiring mesh segmentation and completion.Source: §4.2 · p. 7, Figure 5 · p. 8, Figure 6 · p. 9
4.3 Quantitative Comparisons.
- The fused-shape-pretrained model achieves competitive image alignment, while part-level fine-tuning remains comparable to recent approaches; direct inference takes about 30 seconds regardless of part count.Table 1 reports cosine-similarity comparisons; the speed comparison contrasts direct generation with HoloPart’s preprocessing and part-wise completion pipeline.Source: Table 1 · p. 8, §4.3 · p. 8
4.4 Ablation Study
- Part-level VAE fine-tuning improves robustness to spatial variation, filtering improves segmentation quality and diversity, and the part embedding helps distinguish the dual volumes.Without filtering, outputs tend toward two-part segmentations; without the embedding, the model can generate overlapping or identical parts.Source: §4.4 · p. 9, Figure 7 · p. 9
4.5 Limitations and Future Work
- Scene-graph-based extraction makes part granularity unpredictable, and two-volume packing cannot represent three mutually contacting parts.The authors suggest additional input such as a segmentation mask for control, or using more volumes to address packing constraints.Source: §4.5 · p. 9
5 Conclusion
The paper presents an end-to-end framework for generating part-level 3D objects from one image. Dual volume packing uses part connectivity to handle arbitrary numbers of parts, and experiments report high-quality, diverse, efficient generation.
5 Conclusion
- The framework generates part-level 3D objects from a single image and handles an arbitrary number of parts without relying on 2D segmentation priors.It leverages 3D part connectivity and dual volume packing.Source: §5 · p. 10
- Experiments show high-quality, diverse, and efficient part-level mesh generation that outperforms existing baselines.Source: §5 · p. 10
- The work offers a step toward editable and structured 3D content for downstream applications.Source: §5 · p. 10
A.1 Bipartite Contraction
The appendix describes a greedy contraction procedure for making the part-connectivity graph bipartite and reports the processed dataset’s part-count distribution. It limits cycle enumeration on complex graphs to control processing time.
A.1 Bipartite Contraction
- The algorithm finds graph cycles by depth-first search, then repeatedly contracts the highest-weight edge in each odd cycle until none remain.Source: §A.1 · p. 14
- For graphs with 100 or more edges, it skips cycle enumeration and uses two-coloring, contracting conflicting same-color edges instead.The passage limits cycle enumeration to graphs with fewer than 100 edges because enumerating all simple cycles can be time-consuming for dense graphs.Source: §A.1 · p. 14
- Approximately 60% of processed samples have fewer than 10 parts, while 5% have more than 200 parts.Source: §A.1 · p. 14, Figure 8 · p. 14
A.2 Surface Sampling
This section describes converting packed sub-meshes into signed distance fields and preparing surface and edge samples for VAE training. It also specifies the mesh extraction and sampling procedures.
A.2 Surface Sampling
- Each of the two packed sub-meshes is represented by an unsigned distance field on a 512^3 grid; flood filling from an outside corner identifies empty space, whose complement is marked occupied to form a signed distance field.Source: §A.2 · p. 14
- Marching Cubes extracts a watertight surface from the resulting signed distance field.Source: §A.2 · p. 15
- The VAE input uses 32,768 uniform surface points and 16,384 salient edge points, with random surface-point subsampling if the mesh has no salient edges.Training iterations sample 16,384 uniform, 8,192 near-surface, and 8,192 near-salient-edge points for SDF supervision.Source: §A.2 · p. 15
B.1 Qualitative Comparisons
Qualitative comparisons report that the method outperforms prior approaches in most cases. The reported advantages include finer geometric detail, complete parts, smoother surfaces, and stronger part separation and overall fidelity.
B.1 Qualitative Comparisons
- Our method outperforms prior approaches in most qualitative comparisons, particularly in preserving fine-grained geometric details and producing complete parts.Source: §B.1 · pp. 15–16
- The comparisons highlight preservation of fine-grained geometry and completeness of generated parts.Source: §B.1 · p. 15
- Generated shapes exhibit better surface smoothness, part separation, and overall fidelity than baseline methods.Source: §B.1 · p. 16
B.2 Failure Cases
The model fails on intricate internal structures and can produce inconsistent part segmentation across random seeds.
B.2 Failure Cases
- Highly entangled connectivity in intricate internal structures can defeat greedy odd-cycle contraction, producing incomplete or incorrect reconstructions.Drawer-like objects may have inner compartments disconnected from their outer casings, leaving internal parts broken or missing.Source: §B.2 · p. 16
- Inconsistent part annotations and no explicit granularity control can yield varying segmentations across random seeds, including fused wheels or erroneous splits.Some outputs remain satisfactory and semantically aligned, while others do not.Source: §B.2 · p. 16
- The authors identify more reliable part annotations and stronger structural priors as needed for consistent decomposition.Source: §B.2 · p. 16