PARTFIELD: Learning 3D Feature Fields for Part Segmentation and Beyond

• PARTFIELD avoids predefined part templates and text prompts, which can impede scale through inconsistent labels, ambiguous names, and geometric parts without obvious descriptions.It aims to cover varied part concepts and multi-level decompositions so applications can choose the desired granularity and semantics.Source: §1 · p. 2
• PARTFIELD predicts a continuous 3D feature field whose distances group points by part, and clustering its features yields hierarchical shape decompositions.Features can be queried continuously at any location and used in downstream applications.Source: §1 · p. 2
• PARTFIELD uses triplet-based contrastive learning on image-mask proposals and explicit 3D supervision, accommodating diverse part definitions and granularity.The objective encourages samples from the same part to be more similar than samples from distinct parts, supporting training on large-scale open-world data.Source: §1 · p. 2
• The learned field is consistent across shapes despite no explicit cross-shape supervision, enabling co-segmentation, selection, and correspondence across modalities.The model is demonstrated on in-the-wild meshes, point clouds, and 3D Gaussian splats.Source: §1 · p. 2
• More than 20% performance improvements and an order of magnitude faster inference are reported against the latest baselines on class-agnostic part segmentation.The approach also enables high-quality hierarchical part decomposition.Source: §1 · p. 2

Background

• Traditional data-driven part segmentation [17] [58] often uses predefined templates and supervised learning, constrained by the limited scale and diversity of annotated 3D datasets [38].Source: §2 · p. 3

Point-Prompt 3D Segmentation

• Point-prompt methods [27] [79] and SAM2POINT [9] enable interactive 3D part selection, whereas PartField learns general part features for automatic hierarchical decomposition.Source: §2 · p. 3

Text-Prompt 3D Segmentation

• Text prompts may miss parts without inherent semantic meaning, so PartField learns class-agnostic parts from large-scale data, including semantic and purely geometric parts.Source: §2 · p. 3

Class-Agnostic 3D Segmentation

• Prior class-agnostic methods lift SAM labels or features into 3D, while closed-domain part models generalize poorly; PartField is among the first feedforward models to learn an open-world part-decomposition feature field.Source: §2 · p. 3

3. Method

• PartField maps any 3D point on an input shape to a feature vector, with nearby features representing points in the same part and encoding part hierarchy.The continuous feature field can be clustered to produce parts at different scales.Source: §3 · p. 3

3.1. Training Part Proposals and Point Triplets

• Training proposals come from class-agnostic 2D masks projected from rendered views and available 3D annotations, which complement open-world surface supervision with interior structure and human semantics.Triplets draw positive points from within a proposal and negatives from outside it; 2D proposals supervise visible surfaces, while 3D proposals can include shape interiors.Source: §3.1 · pp. 3–4

3.2. Contrastive Learning with Negative Sampling

• A relative triplet objective avoids explicit scale conditioning for ambiguous, multi-scale proposals, while mixed uniform, 3D-hard, and feature-hard negatives improve accuracy, especially near part boundaries.Positive points are encouraged to be more similar to each other than to a negative point; hard negatives are drawn near part boundaries in Euclidean or feature space.Source: §3.2 · p. 4

3.3. Feedforward Model

• A feedforward 3D model predicts a consistent, complete feature field that extends into shape interiors, resists noisy proposals, and correlates features across shapes.Its triplane field is built from point-cloud features projected onto three planes, processed by 2D CNNs and a transformer, and queried at arbitrary 3D locations.Source: §3.3 · pp. 4–5

3.4. Inference and Clustering

• At inference, one network pass produces triplanes that can be sampled on mesh faces or shape interiors, then clustered into a hierarchical part decomposition.The method supports other 3D modalities, and uses face-adjacency agglomerative clustering by default for mesh decompositions.Source: §3.4 · p. 5

4.1. Implementation Details

• Training uses approximately 340k filtered Objaverse shapes with SAM2-generated 2D proposals and around 30k PartNet shapes with hierarchical 3D part labels.Objaverse shapes are rendered as RGB and mesh-normal images for proposal generation; PartNet labels from all hierarchy levels serve as 3D proposals.Source: §4.1 · p. 5

4.2. Comparison of Class-Agnostic Segmentation

• On PartObjaverse-Tiny, PARTFIELD improves by 22.3% over the second-best method; its feature-field prediction takes less than one second, and PartNetE results show a similar outperforming trend.Clustering to obtain the segmentation takes a few seconds. The reported metric is instance-level mean IoU on both datasets.Source: §4.2 · pp. 6–7, Table 1 · p. 6, Table 2 · p. 6
• Text-based methods generally perform worse in open-world settings, while noisy multi-view predictions can undermine per-shape optimization; SAMesh can over-segment and offers less flexible granularity control.PARTFIELD produces more consistent, robust segmentations and supports adaptive multi-scale decomposition.Source: §4.2 · p. 7

4.3. Applications

• Agglomerative clustering extracts meaningful hierarchical part decompositions, while feature similarity reveals cross-shape consistency even without explicit cross-shape supervision.Users can explore the part hierarchy and choose which parts to subdivide further; consistency is demonstrated across shapes with different topologies, geometries, and functions.Source: §4.3 · p. 7, Figure 6 · p. 7, Figure 7 · p. 7
• PARTFIELD features enable co-segmentation across differing geometries and point-to-point correspondences across substantial shape variations; its co-segmentation results outperform SAMPart3D.Co-segmentation initializes target clustering from source-part mean features; correspondence uses feature-space nearest neighbors followed by smooth discrete optimization.Source: §4.3 · pp. 7–8, Figure 8 · p. 8, Figure 9 · p. 8

4.4. Ablations and Analysis

• 2D proposals from large-scale unlabeled Objaverse already yield decent results, 3D triplets add a modest open-world gain, and hard-negative mining sharpens boundaries; combining both gives the best performance.The additional 3D data are smaller and less diverse, and their potential benefit for interior structures is not reflected by the surface-only metric.Source: §4.4 · p. 8, Table 3 · p. 8, Figure 10 · p. 8
• Despite training mainly on point clouds from human-created meshes, PARTFIELD generalizes to AI-generated assets, 3D Gaussian splatting reconstructions, and mechanical-part CAD models.The evaluated AI-generated assets include outputs from Trellis and Edify3D; the CAD models come from ABC.Source: §4.4 · p. 8

5. Limitations and Future Work

• Although PVCNN and triplane enable fast inference, their extrinsic feature space is weakly correlated with 3D position, and cross-shape applications require consistently oriented shapes.Part segmentation is agnostic to this position correlation.Source: §5 · p. 8
• Current investigations explore PartField only at object scale, with extending it to large scenes left for future work.Source: §5 · p. 8
• The small-scale cross-shape applications investigate an emergent property of the method and are intended to motivate research into part contrastive learning for foundational 3D shape-analysis features.Source: §5 · pp. 8–9

Details of Hard Negative Mining

• For each sampled mask, the method draws 64 positive pairs, 256 uniform random negatives, and 256 3D-hard negatives.Masks are sampled per batch of shapes, and pair labels come from the mask labels.Source: §1 · p. 13
• 3D-hard negatives are sampled with probability weighted by the negative Euclidean distance from candidate pc to query pa.Source: §1 · p. 13
• Feature-hard mining follows the 3D-hard strategy but computes candidate distances in feature space.Source: §1 · p. 13

SAMPart3D

• SAMPart3D [69] is evaluated directly on its mesh-face labels, selecting the best metric per shape across scales from 0.0 to 2.0 in 0.125 increments.The evaluation uses the official codebase and released checkpoint without output alignment or label transfer.Source: §2 · p. 13

Find3D

• Find3D receives 5,000 dummy-white points per shape and ground-truth text prompts; its point labels are transferred to mesh faces by nearest neighbor.Nearest neighbors are found for the centers of input mesh faces.Source: §2 · p. 13

SAMesh

• SAMesh uses default settings on PartObjaverseTiny, but refined hyperparameters and code modifications on PartNetE to address low-poly meshes and poor topology.The authors confirmed the default-setting issue for PartNet meshes, and the adjusted setup performed better than the default on PartNetE.Source: §2 · p. 13

PartSLIP

• PartSLIP [31] uses its zero-shot instance-segmentation predictions from rendered multi-view RGB-D fused into a dense colored point cloud, conditioned on ground-truth part names.Predicted point labels are transferred to mesh faces by nearest neighbor to each face center.Source: §2 · p. 13

Ultrametric Feature Field

• Ultrametric Feature Field uses rendered multi-view images, SAM hierarchies, default NeRF optimization, depth-fused point features, and clustering, selecting the best result per shape across 20 scales.Cluster labels are transferred to the input mesh by nearest neighbor to each face center before evaluation.Source: §2 · p. 13

3. Multiclass Regression Interactive Cosegmentation

• A few user clicks on one COSEG guitar shape extend in real time to segmentations of all guitar shapes using their precomputed feature fields.This is an additional demonstration in the supplemental video, alongside clustering-based cosegmentation.Source: §3 · p. 14
• Instead of clustering, annotated points form a small training set for one-vs-rest multiclass logistic regression from PartField features to segmentation labels.The models are easily fit and evaluated on all shapes in real time on the GPU, providing interactive feedback.Source: §3 · p. 14

4. PartNetE Class Grouping

• Electronics & Computing Devices groups seven classes.Keyboard, Mouse, Laptop, Phone, Camera, USB, Display (monitor), Remote, Printer, and Switch (if treated as a network or power switch).Source: §4 · p. 14
• Large Home Appliances groups five classes.WashingMachine, Dishwasher, Refrigerator, Oven, and Microwave.Source: §4 · p. 14
• Kitchen & Food-Related Items groups nine classes.KitchenPot, Kettle, Toaster, CoffeeMachine, Faucet, Dispenser, Knife, Bottle, and Bucket (often used in kitchen/cleaning contexts).Source: §4 · p. 14
• Furniture & Household Infrastructure groups nine classes.Table, Chair, FoldingChair, StorageFurniture, Door, Window, Lamp, Trash-Can, and Safe (often a household or office fixture).Source: §4 · p. 14
• Tools, Office Supplies, & Miscellaneous groups eleven classes.Stapler, Scissors, Pen, Pliers, Lighter, Box, Cart (e.g., utility cart), Globe (decorative/educational), Suitcase (travel/personal), Eyeglasses (personal), and Clock.Source: §4 · p. 14