Point2Part: Unified 3D Partitioning from Point Prompts

1 INTRODUCTION

• Existing part generators can assign the same region to multiple parts or leave regions uncovered, while point-prompt segmentation often predicts independent masks that need overlap resolution.Existing generation methods typically control parts through counts, text, or 2D masks, whereas segmentation prompts provide direct 3D control but do not ensure a complete, non-overlapping partition.Source: §1 · p. 2
• The method treats decomposition as a joint partition of the entire shape, making parts non-overlapping and their union exhaustive by construction.A point prompt specifies each desired part; a prompt encoder maps prompts to part tokens, and a decoder jointly assigns every query point to exactly one part with coarse-to-fine boundary refinement.Source: §1 · p. 2
• Point2Part outperforms state-of-the-art generation and segmentation methods on all part-quality metrics across image-to-part generation, mesh-to-part generation, and mesh-surface segmentation, while improving geometric compatibility by an order of magnitude.The shared shape latent supports all three tasks, and users can adjust the number and locations of prompts to control the decomposition.Source: §1 · p. 2
• A shared shape latent lets the same framework process image or mesh inputs and produce prompted parts or mesh-surface segmentations.The model builds on a pretrained 3D generation model and includes a shape-latent backbone, prompt encoder, and part decoder.Source: §1 · p. 2

Promptable part segmentation

• 3D promptable segmentation methods answer prompts independently, so masks can overlap or leave gaps; Point2Part instead answers all prompts jointly and generates closed parts as well as segmenting the surface.Prior methods include Point-SAM, P3-SAM, PartSAM, and S2AM3D.Source: §2 · p. 3

Part-level generation

• Part-level generators control parts through counts, boxes, image masks, part names, or segmentations, but do not constrain parts to stay out of one another or report whether they do.Point2Part decodes exclusive and exhaustive parts from user point prompts and a single whole-shape field.Source: §2 · p. 3

Primitive-based decompositions

• Primitive-based methods approximate shapes with a small set of geometric primitives rather than recovering part decompositions, so their outputs are not directly comparable under part-level metrics.Examples include cuboids, superquadrics, learned convexes, and deformed spheres.Source: §2 · p. 3

3.1 PROBLEM FORMULATION

• Each object is decomposed into one closed solid per 3D point prompt, with parts that have no volume overlap and whose union recovers the object.For image input, the object is the solid enclosed by the mesh generated from the image.Source: §3.1 · p. 3

3.2 MODEL

• A shared shape latent from an image or mesh feeds a prompt encoder that attends to the latent and inter-part relationships to produce one token per prompted part.The backbone uses a mesh encoder for mesh input and an image-conditioned flow model for image input.Source: §3.2 · pp. 4–5
• The part decoder refines query features using shape latents and all part tokens, jointly scoring each interior query against the prompted parts.Each part token acts as a dynamic linear classifier; one pass over queries decodes all parts, with cost set by query count and growing only slightly with part count.Source: §3.2 · p. 5
• At each interior location, assigning the highest-scoring part makes the partition exclusive and exhaustive regardless of predicted scores, without suppression, flood filling, or an overlap penalty.Part-specific signed-distance fields retain the whole-shape field only within the assigned part; marching cubes then extracts closed meshes whose cut surfaces close where parts meet.Source: §3.2 · pp. 5–6
• Training samples 1–4 surface points per part and concentrates queries near part boundaries; inference uses one point per part unless specified otherwise.The objective combines classification and Dice losses for the partition with a consistency loss across prompt sets for the same part.Source: §3.2 · p. 6

3.3 INFERENCE

• For image part generation, prompts are placed on the generated mesh; segmentation instead queries each input face centroid and assigns its predicted part label, leaving mesh geometry unchanged.Coarse-to-fine extraction refines the whole-shape SDF near its surface and part SDFs where the two highest assignment scores are close.Source: §3.3 · pp. 6–7

4.1 SETUP

• Baselines use official checkpoints and inference settings; evaluation uses 200 PartObjaverse-Tiny meshes, with the same assets used across all three tasks.Training uses HY3D-Bench with 240K labeled assets; baselines use their own datasets, almost all unreleased. Metrics cover part quality, whole-shape geometry, compatibility, and inference time.Source: §4.1 · p. 7

4.2 MAIN RESULTS

• The method achieves the best part quality for mesh- and image-conditioned generation and reduces inter-part penetration by over an order of magnitude.Whole-shape fidelity is bounded by backbone reconstruction; the method is fastest from a mesh and second fastest from an image.Source: §4.2 · p. 7, Table 1 · p. 8
• The model leads on every segmentation metric, with its largest gain in tight-threshold pF1, and is fastest, tied with S2AM3D.The tight-threshold pF1 result indicates boundaries close to ground truth rather than merely in the right region.Source: §4.2 · p. 8, Table 2 · p. 8

4.3 ABLATIONS

• Removing the prompt encoder causes the largest part-quality drop, while independent binary assignments cause substantial penetration despite similar part quality.Removing consistency loss causes a smaller but consistent drop; removing coarse-to-fine refinement mainly affects tight F1.Source: §4.3 · p. 8, Table 3 · p. 8
• Across both backbones, every metric improves as more prompt points are added, with a single point indicating location and additional points revealing part extent.The evaluated segmentation ablation varies the number of prompt points K.Source: §4.3 · p. 8, Table 4 · p. 8
• The qualitative comparison covers image generation, mesh generation, and segmentation; red marks volume claimed by multiple parts, where prior methods overlap, drop, split, or merge parts while ours matches ground truth.The figure places image generation at the top, mesh generation in the middle, and segmentation at the bottom.Source: Figure 3 · p. 9
• Extremely thin structures and large open surfaces are challenging because they may not enclose well-defined volumes, and the pretrained backbone struggles to represent and reconstruct them.The paper suggests that improving backbone representation of these geometries could extend the method to a broader range of shapes.Source: §4.3 · p. 9

• Jointly assigning the whole shape to prompted parts produces an exclusive and exhaustive partition by construction.Source: §5 · p. 9
• A single model supports image-to-part generation, mesh-to-part generation, and mesh part segmentation.Source: §5 · p. 9
• The model achieves state-of-the-art part quality while reducing part geometric incompatibility by over an order of magnitude.Source: §5 · p. 9

• Selected added weights start at zero so training begins from the backbone’s behavior, with equal initial logits and mean-token pooling.The zero-initialized weights include residual-branch output projections, the fs projection, and the first encoder layer’s attention-pooling query.Source: A IMPLEMENTATION AND TRAINING DETAILS · p. 13, Table 5 · p. 13
• Training queries come mostly from the surface and inter-part seams, with a sparse uniform pool elsewhere in the bounding box.Queries are sampled rather than evaluated on a grid because most grid cells lie far from surfaces or seams, where the loss is already zero.Source: A IMPLEMENTATION AND TRAINING DETAILS · p. 13
• Each part receives one to four surface prompts per prompt set, and a consistency loss links the partitions from two sets.Figure 4 distinguishes sampled queries by part and depicts two prompt sets with balls and cubes.Source: A IMPLEMENTATION AND TRAINING DETAILS · p. 13, Figure 4 · p. 14
• Training runs for three days on four A100 80GB GPUs, with warm-up, hold, and decay learning-rate phases and averaged weights used at inference.The exponential moving average is used to remove step-to-step variation of the raw weights without changing the model.Source: A IMPLEMENTATION AND TRAINING DETAILS · p. 13

• Baselines use released checkpoints and default settings, except PartField selects among 20 cluster counts while this method uses the annotated part count.FullPart and UniPart are not compared because their inference code has not been fully released.Source: B METRIC DETAILS · p. 14
• Interpenetration probes are sampled on each generated part’s surface and counted only if they remain inside their own part before testing against other parts.The protocol samples 2,000 points per part and uses the fast-winding-number sign test; it avoids a grid or resolution parameter.Source: B METRIC DETAILS · p. 14
• The reported residual for this method reflects marching-cubes interpolation across a seam, while independently decoded fields report interpenetration proper.For open-shell methods, the fractional winding number makes the value approximate.Source: B METRIC DETAILS · p. 14
• Watertightness is the benchmark-average share of generated parts that are closed, with every edge shared by exactly two faces.An open part has no interior and therefore cannot be moved or replaced as a solid.Source: B METRIC DETAILS · p. 14
• Image-input shapes are aligned to ground truth with rigid ICP from 24 axis-aligned initial poses, while mesh inputs use identity alignment.Meshes are normalized by their bounding boxes; the selected transform is applied to the whole shape and its parts.Source: B METRIC DETAILS · p. 14
• Generated parts are matched one-to-one to ground-truth parts by Hungarian assignment on part Chamfer distance before pCD and pF1 scoring.For mIoU, each ground-truth face takes the label of the generated part projected onto it.Source: B METRIC DETAILS · p. 14

• As ground-truth part counts rise, our mesh-input accuracy stays close to its simplest-object level, and its margin is largest above 20 parts.For image input, the ordering is the same, while OmniPart falls below PartPacker on objects with the most parts.Source: C ADDITIONAL RESULTS · p. 15, Figure 5 · p. 15
• Across five prompt-sampling runs, standard deviations are 0.08 for pCD, 0.4 for pF1@.01 and pF1@.05, and 0.3 for segmentation mIoU.Source: C ADDITIONAL RESULTS · p. 15
• Three prompt points per part capture most of the observed mIoU and pCD improvement; both metrics change little beyond three points for either backbone.The figure plots segmentation quality against prompt points per part K, with mIoU and pCD shown separately.Source: C ADDITIONAL RESULTS · p. 15, Figure 6 · p. 16
• Under a second per asset, our model is the most accurate in the segmentation quality–inference-time comparison.Increasing K improves quality without changing time appreciably, since prompts are pooled into one token and decoder cost is independent of K.Source: C ADDITIONAL RESULTS · p. 15, Figure 7 · p. 16
• On PartNeXt, our method leads every segmentation column across face mIoU and part-level pCD and pF1, with gains transferring to a benchmark from a different source.Source: C ADDITIONAL RESULTS · p. 15, Table 6 · p. 16

D ADDITIONAL VISUALIZATION

• Generated parts are closed meshes with cut faces that separate without repair and reassemble into the whole; segmentation instead labels faces on open input-mesh surfaces.Exploded views show image-based generation, mesh-based generation, and mesh segmentation.Source: D ADDITIONAL VISUALIZATION · p. 17, Figure 9 · p. 17
• On both segmentation benchmarks, one prompt per part keeps small repeated parts separate, while baselines merge them or fragment larger parts.Examples include cactus spikes and keyboard keys being merged, and roofs and piano bodies being fragmented.Source: D ADDITIONAL VISUALIZATION · p. 17, Figure 10 · p. 18
• Across further image and mesh examples, baseline parts overlap at joints or become fragmentary, while ours has negligible overlap and follows the ground-truth decomposition.The mesh examples cover twelve assets and the image examples ten.Source: D ADDITIONAL VISUALIZATION · p. 18, Figure 12 · p. 20, Figure 13 · p. 21
• Thin shells and large open surfaces are poorly reconstructed by the pretrained VAE, and the decomposition inherits those geometric errors.The visual compares the input, VAE reconstruction, decomposition, and ground truth.Source: Figure 11 · p. 19
• Adding point prompts moves the decomposition from coarse to fine by splitting off the part under each new point while leaving other parts in place.The comparison contrasts point prompts with a part count, part names, and image masks.Source: D ADDITIONAL VISUALIZATION · p. 19, Figure 14 · p. 22
• A prompt at the contact between two parts can give both prompts the same neighborhood and merge the parts; body-placed prompts avoid this, and more points can recover from ambiguity.The figure compares one prompt near a contact, one on each part body, and three points per part.Source: Figure 8 · p. 17, D ADDITIONAL VISUALIZATION · p. 19