Source-linked AI summary
Hunyuan3D 2.1: From Images to High-Fidelity 3D Assets with Production-Ready PBR Material
Team Hunyuan3D, Shuhui Yang, Mingxin Yang, Yifei Feng, Xin Huang, Sheng Zhang, Zebin He, Di Luo, Haolin Liu, Yunfei Zhao, Qingxiang Lin, Zeqiang Lai, Xianghui Yang, Huiwen Shi, Zibo Zhao, Bowen Zhang, Hongyu Yan, Lifu Wang, Sicong Liu, Jihong Zhang, Meng Chen, Liang Dong, Yiwen Jia, Yulin Cai, Jiaao Yu, Yixuan Tang, Dongyuan Guo, Junlin Yu, Hao Zhang, Zheng Ye, Peng He, Runzhou Wu, Shida Wei, Chao Zhang, Yonghao Tan, Yifu Sun, Lin Niu, Shirui Huang, Bojian Zheng, Shu Liu, Shilin Chen, Xiang Yuan, Xiaofeng Yang, Kai Liu, Jianchen Zhu, Peng Chen, Tian Liu, Di Wang, Yuhong Liu, Linus, Jie Jiang, Jingwei Huang, Chunchao Guo
TL;DR
Hunyuan3D 2.1 addresses the lack of accessible, scalable 3D generative modeling by providing an open-source system and tutorial for producing textured meshes from single images. It combines a flow-based shape generator with a multi-view PBR texture model, and evaluations report superiority over leading commercial and open-source approaches in geometry, texture consistency, and human preference.
Problem
3D generative modeling remains fragmented and lacks a robust, scalable foundation model, limiting broader access to advanced 3D asset synthesis.
Method
Hunyuan3D 2.1 combines Hunyuan3D-DiT for high-resolution shape generation with Hunyuan3D-Paint for multi-view PBR material synthesis from a single image.
Results
Quantitative metrics and visual comparisons confirm superiority over leading commercial and open-source models in geometric detail preservation, texture-photo consistency, and human preference.
Takeaways & Limitations
The open-source system and released processing, training pipelines, and model weights make advanced PBR-textured 3D asset generation more accessible.
Abstract
from arXiv · showhide
3D AI-generated content (AIGC) is a passionate field that has significantly accelerated the creation of 3D models in gaming, film, and design. Despite the development of several groundbreaking models that have revolutionized 3D generation, the field remains largely accessible only to researchers, developers, and designers due to the complexities involved in collecting, processing, and training 3D models. To address these challenges, we introduce Hunyuan3D 2.1 as a case study in this tutorial. This tutorial offers a comprehensive, step-by-step guide on processing 3D data, training a 3D generative model, and evaluating its performance using Hunyuan3D 2.1, an advanced system for producing high-resolution, textured 3D assets. The system comprises two core components: the Hunyuan3D-DiT for shape generation and the Hunyuan3D-Paint for texture synthesis. We will explore the entire workflow, including data preparation, model architecture, training strategies, evaluation metrics, and deployment. By the conclusion of this tutorial, you will have the knowledge to finetune or develop a robust 3D generative model suitable for applications in gaming, virtual reality, and industrial design.
1 Introduction
Hunyuan3D 2.1 addresses the lack of a robust, scalable 3D foundation model with an open-source system for generating textured meshes from single images. It combines dedicated shape and PBR texture models, supports modular use, and outperforms compared methods in reported evaluations.
- The 3D generative modeling field lacks a robust, scalable foundation model despite progress in latent representations, geometry, and texture synthesis.
- Hunyuan3D 2.1 generates a textured mesh from a single image using Hunyuan3D-DiT for shape generation and Hunyuan3D-Paint for PBR material generation.The shape model combines flow-based diffusion with Hunyuan3D-ShapeVAE, while the paint model uses mesh-conditioned multi-view diffusion.
- Hunyuan3D-Paint generates aligned albedo, metallic, and roughness maps while improving cross-view consistency and robustness to varying lighting conditions.Its design uses spatial-aligned multi-attention, 3D-aware RoPE, and illumination-invariant training.
- Separate shape and texture stages let users generate untextured meshes or texture custom assets, increasing flexibility for industrial applications.
- Quantitative metrics and visual comparisons confirm superiority in geometric detail preservation, texture-photo consistency, and human preference against leading commercial and open-source models.
- The tutorial covers architecture, data processing, training, evaluation, and deployment to help practitioners use or develop 3D generative models.
2 Data Processing
The data-processing section describes how training data are prepared for the shape and texture models. It first introduces dataset preparation, then explains how training and testing data are obtained.
- The section explains data processing for training the shape generation and texture models.
- It begins with dataset preparation before presenting procedures for obtaining training and testing data.
- The processing discussion covers both shape-generation data and texture-model data.
2.1 Dataset collection
The system uses large collections of 3D assets assembled from public and custom sources for shape generation, plus a curated human-annotated subset for texture synthesis.
- 100K+ textured and untextured 3D examples are collected from public and custom datasets for shape generation.Public sources mainly include ShapeNet, ModelNet40, Thingi10K, and Objaverse.
- 70K+ human-annotated, high-quality Objaverse-XL examples are filtered under strict curation protocols for texture synthesis.
- Dataset collection distinguishes shape-generation data from texture-synthesis data by using broader asset collection for shape and curated annotations for texture.
2.2 Data preprocessing for shape generation
Shape preprocessing standardizes meshes, repairs defective geometry into watertight surfaces, samples signed-distance and surface points, and renders varied image conditions.
- Meshes are normalized into a standardized coordinate space by fitting each object within a unit cube centered at the origin while preserving aspect ratios.The process uses each object's axis-aligned bounding box and uniform scaling.
- IGL constructs signed distance fields from defective geometry, and marching cubes extracts a topologically closed zero-level surface.Winding-number thresholding resolves ambiguous inside/outside signs near self-intersections.
- SDF queries combine near-surface and space-wide sampling to capture both intricate local geometry and broader shape information.
- The hybrid surface strategy allocates approximately 50% of points to uniform coverage and 50% near high-curvature features through importance sampling.Sampling density adapts to geometric complexity.
- Shape-generation condition images use 150 cameras uniformly distributed on a sphere with Hammersley sampling and randomized field-of-view augmentation.Augmented fields of view are sampled from U(10°, 70°).
- Camera radius is varied between 1.51 and 9.94 to maintain consistent object framing.
- The preprocessing pipeline orders normalization, watertight processing, SDF sampling, surface sampling, and condition rendering.
2.3 Data preprocessing for texture synthesis
Texture synthesis training uses a rigorously curated dataset of over 70,000 human-annotated 3D assets, rendered across multiple elevations and azimuths with PBR and lighting data.
- The training dataset contains 70k+ human-annotated, high-quality assets filtered from Objaverse and Objaverse-XL under strict curation protocols.
- Each object is rendered at four elevation angles: −20°, 0°, 20°, and one random angle.
- At each elevation, 24 uniformly azimuth-distributed views produce 512 × 512 albedo, metallic, roughness, and HDR/Point-light images.
3 Training
Hunyuan3D 2.1 combines latent shape modeling, image-conditioned flow-based diffusion, and PBR material synthesis to generate high-resolution 3D assets. Its training uses reconstruction and KL losses for shape compression, flow matching for shape generation, and multiview strategies for coherent material maps.
- Shape generation: The shape model uses Hunyuan3D-ShapeVAE to compress 3D asset geometry into a latent representation and reconstruct a neural field.The encoder processes uniformly and importance-sampled surface points through Fourier positional features, cross-attention, and self-attention; the decoder predicts an SDF.
- Shape generation: ShapeVAE training combines SDF reconstruction MSE with KL-divergence regularization, while multi-resolution training caps latent sequences at 3072 tokens.The KL term encourages a compact, continuous latent space.
- Shape generation: Hunyuan3D-DiT is a flow-based diffusion model that generates detailed, high-resolution 3D shapes from image conditions.It uses DINOv2 Giant image features, cross-attention, 21 Transformer layers, latent-code skip connections, and an MOE layer.
- Shape generation: Flow matching trains the model to predict a velocity field along an affine path from Gaussian noise toward data, then uses Euler ODE integration during inference.The path is defined by xt = (1 − t) × x0 + t × x1 and ut = x1 − x0.
- Material generation: The material framework generates albedo, roughness, and metallic maps under the Disney Principled BRDF model instead of relying only on RGB textures.It conditions generation on reference-image features, geometry-rendered normals, and canonical coordinate maps.
- Material generation: 3D-Aware RoPE improves cross-view coherence, while illumination-invariant training enforces consistent intrinsic material properties across lighting conditions.The method targets texture seams, ghosting, light-free albedo, and accurate metallic-roughness maps.
4 Evaluation
Hunyuan3D 2.1 is evaluated across untextured shape generation, texture synthesis, and complete image-to-3D asset creation. The reported comparisons show strong performance in shape fidelity, texture quality, and end-to-end generation.
- Evaluation scope: Evaluation covers 3D shape generation, texture synthesis, and complete textured 3D asset creation.The shape evaluation focuses on untextured meshes, while the other settings assess textures and end-to-end assets.
- 3D Shape Generation: Hunyuan3D-DiT delivers the most accurate shape-generation results among the compared methods.Visual comparisons also show prompt adherence, intricate-detail capture, and watertight meshes suitable for downstream applications.
- Texture Synthesis: Hunyuan3D-Paint outperforms SyncMVD-IPA, TexGen, and Hunyuan3D-2.0 in quantitative and qualitative texture comparisons.The evaluation uses FID, CLIP-FID, and LPIPS on textures generated from an untextured shape and a single image.
- Complete 3D Asset Creation: End-to-end comparisons show the model produces high-fidelity PBR material maps and mitigates shortcomings from lower-quality geometries.The comparison includes Step1X-3D, 3DTopia-XL, and two commercial models using a single image input.
5 Conclusion
Hunyuan3D 2.1 combines high-fidelity geometry generation and PBR material synthesis in an open-source framework for production-ready 3D content creation. Its end-to-end image-to-3D results are presented as having high visual fidelity.
- 5 Conclusion: Hunyuan3D 2.1 unifies high-fidelity geometry generation and PBR material synthesis within an open-source framework.The system combines a DiT for shape generation with a multi-view conditioned painter for PBR material synthesis.
- 5 Conclusion: The system is designed to support rapid creation of studio-quality 3D assets with exceptional visual fidelity.The conclusion frames this capability for production-ready 3D content creation.
6 Contributors •
The project credits sponsors, leaders, and contributors across data, shape generation, texture synthesis, and infrastructure.
- Contributors: Project sponsors are Jie Jiang, Linus, Yuhong Liu, Di Wang, Tian Liu, and Peng Chen.
- Contributors: Project leaders are Chunchao Guo and Jingwei Huang.
- Contributors: Core contributors are credited across data, shape generation, texture synthesis, and infrastructure.The listed roles span the contributor groups in the project credits.