PaperlayerView exact revision on arXiv

Source-linked AI summary

PartNeXt: A Next-Generation Dataset for Fine-Grained and Hierarchical 3D Part Understanding

Penghao Wang, Yiyang He, Xin Lv, Yukai Zhou, Lan Xu, Jingyi Yu, Jiayuan Gu

arXiv:2510.20155v3cs.CV

TL;DR

Existing 3D part datasets such as PartNet are limited by untextured geometries and expert-dependent annotation, restricting scalability and usability. PartNeXt combines 23,519 high-quality textured meshes with detailed hierarchical part masks across 50 categories and a fully web-based interface for scalable crowdsourcing. Current models struggle with fine-grained leaf-level segmentation and open-vocabulary part-centric queries, while Point-SAM trained on PartNeXt substantially outperforms its PartNet-trained counterpart.

  • Problem

    Existing 3D part datasets such as PartNet are limited by untextured geometries and expert-dependent annotation, restricting scalability and usability.

  • Method

    PartNeXt combines 23,519 high-quality textured meshes with detailed hierarchical part masks across 50 categories and a fully web-based interface for scalable crowdsourcing.

  • Results

    Current models struggle with fine-grained leaf-level segmentation and open-vocabulary part-centric queries, while Point-SAM trained on PartNeXt substantially outperforms its PartNet-trained counterpart.

  • Takeaways & Limitations

    PartNeXt provides a foundation for evaluating and improving fine-grained, hierarchical, texture-aware 3D part understanding.

  • Takeaways & Limitations

    PartNeXt currently has 23,519 models, requires predefined fine-grained hierarchies, and provides only plain part-name annotations.

Abstract

from arXiv · show

Understanding objects at the level of their constituent parts is fundamental to advancing computer vision, graphics, and robotics. While datasets like PartNet have driven progress in 3D part understanding, their reliance on untextured geometries and expert-dependent annotation limits scalability and usability. We introduce PartNeXt, a next-generation dataset addressing these gaps with over 23,000 high-quality, textured 3D models annotated with fine-grained, hierarchical part labels across 50 categories. We benchmark PartNeXt on two tasks: (1) class-agnostic part segmentation, where state-of-the-art methods (e.g., PartField, SAMPart3D) struggle with fine-grained and leaf-level parts, and (2) 3D part-centric question answering, a new benchmark for 3D-LLMs that reveals significant gaps in open-vocabulary part grounding. Additionally, training Point-SAM on PartNeXt yields substantial gains over PartNet, underscoring the dataset's superior quality and diversity. By combining scalable annotation, texture-aware labels, and multi-task evaluation, PartNeXt opens new avenues for research in structured 3D understanding.

1 Introduction

PartNeXt addresses limitations in existing 3D part datasets by providing textured, fine-grained hierarchical annotations at scale and evaluating models on segmentation and part-centric reasoning.

  • Motivation: Existing PartNet annotations can lose textures and deform geometry through remeshing, limiting visual cues needed for annotation and learning.PartNet provides large-scale hierarchical annotations, but its data quality and annotation tooling constrain broader usability and scalability.
  • Dataset contribution: PartNeXt contains 23,519 high-quality textured meshes with detailed hierarchical part masks across 50 object categories.The dataset combines models from Objaverse, ABO, and 3D-FUTURE with a web-based crowdsourcing workflow.
  • Annotation challenge: PartNeXt’s dual-panel interface supports sequential labeling of visible and interior parts, addressing occlusion challenges in complex 3D objects.The workflow displays unannotated regions alongside already segmented parts and is designed to reduce reliance on specialized 3D expertise.
  • Evaluation: State-of-the-art segmentation methods perform noticeably worse on PartNeXt, especially for fine-grained leaf-level parts.The reported methods include PartField, SAMPart3D, and SAMesh.
  • Evaluation: PartNeXt introduces 3D part-centric question answering to test open-vocabulary part grounding, detection, and reasoning in 3D LLMs.The benchmark targets capabilities that current 3D LLMs struggle to perform reliably.

2 Related Work

PartNeXt extends prior 3D datasets and part benchmarks with broader category coverage, textured direct annotations, and evaluation of fine-grained spatial reasoning.

  • 3D datasets: Large-scale repositories such as ShapeNet, Objaverse, ABO, and 3D-FUTURE provide complementary sources of textured and diverse 3D models.Their differences in scale, domain, metadata, and rendering quality motivate dataset construction that combines multiple sources.
  • 3D part annotation: PartNet established large-scale hierarchical part annotation with detailed masks for approximately 26K shapes across 24 categories.It enabled semantic, hierarchical, and instance-level part segmentation benchmarks.
  • 3D part annotation: PartNeXt covers 50 categories, exceeding PartNet’s 24-category scope while retaining fine-grained part annotations.This broader category range is intended to increase diversity beyond existing part-level datasets.
  • 3D part annotation: Unlike PartNet’s separate untextured part meshes, PartNeXt annotates parts directly on textured meshes.Direct annotation avoids extra alignment when textures are required and preserves native appearance information.
  • 3D part understanding: Part-level annotations support downstream tasks including segmentation, assembly, generative modeling, and articulated reconstruction.Recent 3D language models additionally align point clouds with language for classification, captioning, and functionality understanding.

3 Data Annotation

PartNeXt is constructed through curated multi-source collection, formalized hierarchical labels, visual annotation aids, and a web-based system for textured-mesh crowdsourcing.

  • 3.1 Data collection and preprocessing: The dataset combines curated models from Objaverse, ABO, and 3D-FUTURE after metadata filtering and CLIP-based category assignment.Objaverse filtering removes animations, overly complex meshes, scans, and architectural objects; objects below a 0.75 similarity threshold are discarded.
  • 3.2 Hierarchy Definition and Example Generation: Category-specific hierarchies are generated with GPT-4o using functionality, hierarchy, variant coverage, atomicity, and consistency principles, then reviewed by experts.Rendered category images help refine diverse part variants before human validation.
  • 3.2 Hierarchy Definition and Example Generation: Visual examples are generated and manually reviewed to reduce ambiguity from specialized part terminology during annotation.Inaccurate or missing examples can be supplemented with curated online images.
  • 3.3 Annotation System Design: The web-based annotation system combines hierarchical workflows, dual-panel visualization, and multiple face-selection tools for scalable labeling.Annotators progressively expand predefined hierarchies, select faces on the original textured mesh, and transfer confirmed regions into the segmented view.
  • 3.3 Annotation System Design: Flexible combinations of selection modes eliminate remeshing while preserving original texture information during face-level annotation.The system supports coarse connected-component selection followed by precise face removal or refinement.
  • 3.4 Statistic: PartNeXt contains 350,187 annotated instances across 23,519 objects and 50 categories, with hierarchy depths ranging from 4 to 10.Each annotation received at least one review, and 5,211 corrections were made.

4 BenchMark and Experiments

PartNeXt is evaluated through class-agnostic part segmentation, part-centric 3D question answering, and promptable segmentation training. The experiments show persistent fine-grained segmentation and structural-reasoning challenges, while PartNeXt training improves Point-SAM performance.

  • 4.1 Class-Agnostic 3D Part Instance Segmentation: The segmentation benchmark evaluates 250 objects using leaf nodes as labels and reports mean IoU from maximum predicted-part overlap for each ground-truth part.The benchmark includes five objects per category and follows the PartField evaluation protocol.
  • 4.1 Class-Agnostic 3D Part Instance Segmentation: Current 3D part segmentation methods struggle with fine-grained segmentation, balancing granularity with semantic consistency.SAMesh performs relatively strongly but over-segments; PartField can merge adjacent regions, and SAMPart3D loses continuity in weak textures with inconsistent granularity.
  • 4.2 Part-Centric 3D Question Answering: The part-centric question-answering benchmark tests part counting, classification, and grounding to probe semantic granularity, structural awareness, and spatial correspondence.Grounding requires predicting bounding-box corners, while counting asks for the number of target-part instances and classification identifies a highlighted region.
  • 4.2 Part-Centric 3D Question Answering: Across the three question-answering tasks, current 3D vision-language models remain limited on part counting and grounding, indicating insufficient fine-grained structural understanding.The benchmark compares PointLLM, ShapeLLM, and 3DLLM under category-aware and category-agnostic prompting.
  • 4.3 Analysis on 3D Promptable Segmentation: Training Point-SAM on the PartNeXt–PartNet mixture improves performance, while PartNeXt alone also yields significant gains over PartNet.These results support the quality of PartNeXt annotations and the benefit of broader category diversity for generalization.

5 Limitations and Conclusion

PartNeXt combines large-scale textured, hierarchical 3D annotations with benchmarks showing persistent limits in fine-grained part understanding, while retaining important scope boundaries.

  • Limitations: The dataset currently provides plain part-name annotations, while caption or physical-attribute labels remain a proposed enrichment.Adding such annotations could increase the information available for both category and part nodes.
  • Limitations: PartNeXt is limited to 23,519 models because annotation quality is prioritized, although expansion with additional ObjaverseXL data is planned.
  • Limitations: Predefined fine-grained hierarchies constrain open-vocabulary annotation, motivating exploration of deeper integration with vision-language models.
  • Conclusion: PartNeXt contains 23,519 models and 350,187 parts across 50 categories, with native texture and benchmarks for class-agnostic segmentation and part-centric question answering.The dataset is intended as a foundational resource for fine-grained, hierarchically structured 3D part understanding.
  • Annotation: The annotation workflow uses a web-based system guided by connectivity-aware part prompts to support efficient large-scale labeling.

B Dataset Statistic

The dataset statistics section presents PartNeXt’s object, part, and hierarchy statistics and reports that annotators may use an “Other” label for uncovered parts.

  • Dataset Statistics: Table 5 reports PartNeXt statistics using annotated objects, total parts, median parts, median hierarchy depth, and maximum hierarchy depth.
  • Other Parts: Annotators can label parts as “Other” when the predefined hierarchy does not cover all possible parts.The distribution of these instances is reported in Table 6.
  • Other Parts: Table 6 summarizes the percentage of occurrences for each count category of “Other” instances.

C Comparison with Recent 3D Part Dataset

Compared with prior 3D part datasets, PartNeXt combines broader category coverage with textured raw geometry and hierarchical annotations, avoiding key limitations of PartNet and PartObjaverseTiny.

  • PartNet: PartNet annotations lack texture and original geometry because remeshing omits color and geometric details and requires manually drawn cutting lines.
  • PartObjaverseTiny: PartObjaverseTiny retains original geometry but lacks hierarchical structure and contains only 200 models, limiting multi-granularity and large-scale applications.
  • Scale and Diversity: PartNeXt includes more than twice PartNet’s category count while offering scale comparable to PartNet and far exceeding PartObjaverseTiny’s 200 models.
  • Dataset Properties: PartNeXt preserves complete texture and original geometric structure while retaining hierarchical annotations for nuanced multi-granularity understanding.

D Detail of Segmentation BenchMark

The segmentation benchmark reports per-category metrics under standardized evaluation conditions, with each method run using its public implementation and original settings.

  • Evaluation Protocol: Experiments use two Intel Xeon Platinum 8581C processors and four NVIDIA L40 GPUs to maintain fairness across evaluations.
  • Evaluation: The benchmark reports per-category metrics for every included method.
  • Evaluation Protocol: Methods are evaluated with publicly available codebases, original settings, and unified predicted-mask formats before metric computation.

E Detail of LLM BenchMark

The benchmark standardizes evaluation of 3D LLMs across part counting and classification, using prompt-based outputs, postprocessing, and task-specific metrics.

  • E Detail of LLM BenchMark: The benchmark uses uniformly sampled point clouds and a fixed hardware setup across experiments.Meshes are sampled to 8192 points using area-based sampling; experiments use two Xeon Platinum 8581C processors and four NVIDIA L40 GPUs.
  • E Detail of LLM BenchMark: Outputs are postprocessed because current 3D LLMs often struggle with instruction following and response formatting.A Qwen3-14B model converts outputs into usable formats.
  • E Detail of LLM BenchMark: Part classification verifies whether a predicted label matches the ground-truth concept, accepting semantic equivalence rather than exact wording.The output is restricted to a True-or-False JSON object, with nonessential attributes ignored.
  • E.1 Detail of Part Count Task: Part counting converts responses into a single numerical value using rules that extract explicitly stated quantities and handle absent counts as zero.Examples cover quantities written as words or digits, single items, and no additional parts.
  • E.1 Detail of Part Count Task: Part counting selects repeated, semantically meaningful parts and evaluates predicted quantities with mean absolute error.The benchmark averages object-level count errors across objects.

E.2 Detail of Part Classification Task

The part classification task uses segmentation masks to highlight a queried part within the point cloud, providing a visual 3D prompt for semantic prediction.

  • E.2 Detail of Part Classification Task: The queried part is highlighted in red within the point cloud so the LLM can classify the prompted segment.Objects with large pre-existing red regions are excluded to reduce ambiguity.

E.3 Detail of Part Grounding Task

The part grounding task evaluates whether 3D LLMs can localize queried parts with bounding boxes, and examples show that precise coordinate generation remains difficult.

  • E.3 Detail of Part Grounding Task: ShapeLLM grounds parts by outputting eight bounding-box vertices, whose coordinate-wise extrema are used to compute IoU.The figure compares the predicted box with the ground-truth box.
  • E.3 Detail of Part Grounding Task: 3D LLMs struggle to generate precise bounding-box coordinates, indicating weak part-grounding ability.The examples compare outputs from 3DLLM and PointLLM.
  • E.3 Detail of Part Grounding Task: The grounding prompts ask for the screen’s bounding-box vertices in an explicit eight-coordinate format.Example responses place the screen at the center or origin of a normalized box.
  • E.3 Detail of Part Grounding Task: The benchmark includes visualized annotation examples and predefined hierarchies alongside grounding examples.The supplied figures are labeled as annotation-result or hierarchy visualizations.
Loading 2510.20155v3…