PaperlayerView exact revision on arXiv

Source-linked AI summary

Hunyuan3D 2.1: From Images to High-Fidelity 3D Assets with Production-Ready PBR Material

Team Hunyuan3D, Shuhui Yang, Mingxin Yang, Yifei Feng, Xin Huang, Sheng Zhang, Zebin He, Di Luo, Haolin Liu, Yunfei Zhao, Qingxiang Lin, Zeqiang Lai, Xianghui Yang, Huiwen Shi, Zibo Zhao, Bowen Zhang, Hongyu Yan, Lifu Wang, Sicong Liu, Jihong Zhang, Meng Chen, Liang Dong, Yiwen Jia, Yulin Cai, Jiaao Yu, Yixuan Tang, Dongyuan Guo, Junlin Yu, Hao Zhang, Zheng Ye, Peng He, Runzhou Wu, Shida Wei, Chao Zhang, Yonghao Tan, Yifu Sun, Lin Niu, Shirui Huang, Bojian Zheng, Shu Liu, Shilin Chen, Xiang Yuan, Xiaofeng Yang, Kai Liu, Jianchen Zhu, Peng Chen, Tian Liu, Di Wang, Yuhong Liu, Linus, Jie Jiang, Jingwei Huang, Chunchao Guo

arXiv:2506.15442v1cs.CVcs.AI

TL;DR

3D generative modeling remains fragmented and lacks a robust, scalable foundation model. The paper presents Hunyuan3D 2.1, a system for generating textured meshes from single images, whose evaluations report superiority in geometric detail preservation, texture-photo consistency, and human preference.

  • Problem

    3D generative modeling remains fragmented and lacks a robust, scalable foundation model.

    Source: §1 · p. 2

  • Method

    Hunyuan3D 2.1 generates textured meshes from single images using separate models for shape generation and PBR texture synthesis.

    Source: §1 · p. 2

  • Results

    Evaluations against leading commercial and recent open-source models report superiority in geometric detail preservation, texture-photo consistency, and human preference.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Separating shape and texture generation lets users generate untextured meshes or texture custom assets, enhancing flexibility for industrial applications.

    Source: §1 · p. 2

Abstract

from arXiv · show

3D AI-generated content (AIGC) is a passionate field that has significantly accelerated the creation of 3D models in gaming, film, and design. Despite the development of several groundbreaking models that have revolutionized 3D generation, the field remains largely accessible only to researchers, developers, and designers due to the complexities involved in collecting, processing, and training 3D models. To address these challenges, we introduce Hunyuan3D 2.1 as a case study in this tutorial. This tutorial offers a comprehensive, step-by-step guide on processing 3D data, training a 3D generative model, and evaluating its performance using Hunyuan3D 2.1, an advanced system for producing high-resolution, textured 3D assets. The system comprises two core components: the Hunyuan3D-DiT for shape generation and the Hunyuan3D-Paint for texture synthesis. We will explore the entire workflow, including data preparation, model architecture, training strategies, evaluation metrics, and deployment. By the conclusion of this tutorial, you will have the knowledge to finetune or develop a robust 3D generative model suitable for applications in gaming, virtual reality, and industrial design.

1 Introduction

Although 2D image and video generation has advanced rapidly, 3D generative modeling remains fragmented and lacks a robust, scalable foundation model. Hunyuan3D 2.1 addresses this gap with an open-source system that generates textured meshes from a single image.

Introduction

  • Hunyuan3D 2.1 combines Hunyuan3D-DiT for shape generation with Hunyuan3D-Paint for mesh-conditioned, multi-view PBR material generation, producing aligned and view-consistent textures.The system generates a textured mesh from a single image input.Source: §1 · p. 2
  • Hunyuan3D-ShapeVAE uses mesh-surface importance sampling to sharpen edges and variational token length to capture intricate geometry, while Hunyuan3D-DiT uses scalable, flexible flow-matching diffusion [15] [3].Together, they target high-quality, high-fidelity shape generation.Source: §1 · p. 2
  • Hunyuan3D-Paint generates albedo, metallic, and roughness maps using spatial-aligned multi-attention, 3D-aware RoPE, and illumination-invariant training for aligned maps, cross-view consistency, and light-free albedo robust to changing lighting.Source: §1 · p. 2
  • Separating shape and texture generation into stages enables mesh-only generation or texturing custom assets, providing flexibility for industrial applications.The paper describes this modular strategy as building on prior large reconstruction models [16] [17] [18] [19] [20] [21] [22] [23].Source: §1 · p. 2
  • Quantitative metrics and visual comparisons report superior geometric detail preservation, texture-photo consistency, and human preference over Michelangelo [8], Craftsman 1.5 [24], Trellis [25], TripoSG [9], Step1X-3D [26], and Direct3D-S2 [27].The introduction reports these results without specifying metric values.Source: §1 · p. 2

2 Data Processing

This section outlines data processing for training the shape-generation and texture models, covering dataset preparation and the acquisition of their training and testing data.

2 Data Processing

  • Data processing covers dataset preparation for training the shape-generation and texture models.Source: §2 · p. 2
  • The section describes how to obtain training and testing data for both models.Source: §2 · p. 2

2.1 Dataset collection

The dataset collection combines broad shape-generation data from public and custom sources with curated, human-annotated data for texture synthesis.

2.1 Dataset collection

  • Shape generation uses 100K+ textured and untextured models from public and custom datasets, while texture synthesis uses 70K+ strictly curated, human-annotated high-quality Objaverse-XL assets [32].Public shape data comes mainly from ShapeNet [28], ModelNet40, Thingi10K [30], and Objaverse [31] [32].Source: §2.1 · p. 2

2.2 Data preprocessing for shape generation

The preprocessing pipeline standardizes geometry, converts meshes into watertight surfaces, samples signed-distance and surface points, and renders multi-view condition images.

2.2.1 Normalization

  • Objects are scaled uniformly into a unit cube centered at the origin; point clouds are centered and scaled by their maximum distance from the centroid.This preserves aspect ratios and geometric relationships while reducing dataset-wide size variation.Source: §2.2.1 · p. 3

2.2.2 Watertight

  • IGL computes signed distances on a 3D query grid, classifies interior points with winding number ω > 0.5, and extracts a closed mesh using zero-level marching cubes.The winding number resolves ambiguous signs near self-intersections.Source: §2.2.2 · p. 3

2.2.3 SDF Sampling

  • SDF supervision uses randomly selected points near the surface and uniformly throughout [−1, 1]3 to capture fine surface details and overall shape.The two sampling regions provide complementary local and global geometric information.Source: §2.2.3 · p. 3

2.2.4 Surface Sampling

  • Surface sampling allocates approximately 50% of points to uniform coverage and 50% to high-curvature features, with density adapting to geometric complexity.Importance sampling emphasizes sharp edges and corners while keeping planar regions sparser.Source: §2.2.4 · p. 3

2.2.5 Condition Render

  • Condition images are rendered from 150 sphere-distributed Hammersley cameras with randomized offsets, FoVs sampled from 10°–70°, and radii adjusted from 1.51–9.94 for consistent framing.The camera sequence uses a randomized offset δ ∈ [0, 1)2.Source: §2.2.5 · pp. 3–4

2.3 Data preprocessing for texture synthesis

Texture synthesis training uses a curated dataset of 70k+ human-annotated assets filtered from Objaverse [31] and Objaverse-XL [32]. Each asset is rendered across multiple views with material maps and varied lighting.

2.3 Data preprocessing for texture synthesis

  • The training set contains 70k+ human-annotated high-quality assets curated under strict protocols from Objaverse [31] and Objaverse-XL [32].Source: §2.3 · p. 4
  • For each object, renders cover elevations of −20°, 0°, 20°, and a random angle, with 24 uniformly azimuth-distributed views per elevation.Each view includes 512 × 512 albedo, metallic, and roughness maps, plus HDR/Point-light images.Source: §2.3 · p. 4
  • Reference images use randomly sampled viewpoints with elevations from [-30°, 70°] and stochastic illumination from point lights (p=0.3) or HDR maps (p=0.7).Source: §2.3 · p. 4

3 Training

Hunyuan3D’s shape pipeline combines a mesh-compressing ShapeVAE with an image-conditioned flow-based DiT that predicts latent tokens for mesh reconstruction. Hunyuan3D-Paint generates physically based material maps and adds mechanisms for spatial alignment, cross-view consistency, and illumination invariance.

3.1.1 Hunyuan3D-ShapeVAE

  • ShapeVAE encodes sampled mesh points into variational latent tokens and decodes them into an SDF that marching cubes converts to a triangle mesh.Training combines SDF reconstruction MSE with KL-divergence to compact the latent space and uses dynamic multi-resolution token lengths up to 3072.Source: §3.1.1 · p. 5

3.1.2 Hunyuan3D-DiT

  • Hunyuan3D-DiT conditions on DINOv2 Giant image features and uses 21 transformer layers, latent-code skip connections, cross-attention, and an MoE layer.It trains with flow matching on an affine conditional optimal-transport path and uses a first-order Euler ODE solver at inference.Source: §3.1.2 · pp. 5–6

3.2 Hunyuan3D-Paint

  • Hunyuan3D-Paint generates albedo, roughness, and metallic maps under the Disney Principled BRDF using reference-image features, geometry normals, and canonical coordinate maps.A dual-branch UNet applies self-, multiview-, and reference-attention to albedo and metallicroughness maps, propagating albedo reference-attention outputs to the MR branch for spatial alignment.Source: §3.2 · p. 6
  • 3D-Aware RoPE adds multiresolution 3D coordinate encodings to multiview attention to reduce cross-view texture seams and ghosting.Illumination-invariant training enforces consistency using reference images of the same object rendered under different lighting conditions.Source: §3.2 · p. 7

4 Evaluation

Evaluation covers untextured shape generation, texture synthesis, and complete textured 3D asset creation.

4.1 3D Shape Generation

  • Hunyuan3D-DiT delivers the most accurate shape-generation results and follows image prompts closely, capturing intricate details and producing watertight meshes.It is compared with Direct3D-S2 [27], Step1X-3D [26], and TripoSG [9].Source: §4.1 · p. 7, Figure 5 · p. 8

4.2 Texture Map Synthesis

  • Hunyuan3D-Paint achieves the best texture-generation performance against SyncMVD-IPA [13], TexGen [46], and Hunyuan3D-2.0 [36] on FID, CLIP-FID, and LPIPS.The comparison uses an untextured shape and a single image, with quantitative and qualitative evaluations.Source: §4.2 · p. 8, Table 2 · p. 9
  • Hunyuan3D produces the strongest end-to-end results, generating the highest-fidelity PBR maps while mitigating low-quality geometry shortcomings against image-to-3D alternatives.The visual comparison covers Step1X-3D [26], 3DTopia-XL [47], and commercial Model 1 and Model 2.Source: §4.2 · p. 8

5 Conclusion

Hunyuan3D 2.1 combines high-fidelity geometry generation with PBR material synthesis in an open-source framework for production-ready 3D asset creation. The conclusion presents it as a way to make advanced 3D AIGC more accessible and scalable.

5 Conclusion

  • Hunyuan3D 2.1 pairs a DiT for shape generation with a multi-view-conditioned painter for PBR material synthesis, enabling rapid creation of studio-quality assets.The conclusion describes the assets as having exceptional visual fidelity.Source: §5 · p. 9
  • By open-sourcing its data-processing and training pipelines and model weights, Hunyuan3D 2.1 makes advanced 3D AIGC more accessible.The paper describes it as the first fully open-sourced solution for PBR-textured 3D asset generation.Source: §5 · p. 9
  • The conclusion reports that quantitative metrics demonstrate Hunyuan3D 2.1's superiority in geometric accuracy and material quality.It frames the system as bridging academic research and scalable content creation for gaming, virtual reality, and industrial design.Source: §5 · p. 9

Related papers

Loading 2506.15442v1…