Source-linked AI summary
Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation
Zibo Zhao, Zeqiang Lai, Qingxiang Lin, Yunfei Zhao, Haolin Liu, Shuhui Yang, Yifei Feng, Mingxin Yang, Sheng Zhang, Xianghui Yang, Huiwen Shi, Sicong Liu, Junta Wu, Yihang Lian, Fan Yang, Ruining Tang, Zebin He, Xinzhou Wang, Jian Liu, Xuhui Zuo, Zhuo Chen, Biwen Lei, Haohan Weng, Jing Xu, Yiling Zhu, Xinhai Liu, Lixin Xu, Changrong Hu, Shaoxiong Yang, Song Zhang, Yang Liu, Tianyu Huang, Lifu Wang, Jihong Zhang, Meng Chen, Liang Dong, Yiwen Jia, Yulin Cai, Jiaao Yu, Yixuan Tang, Hao Zhang, Zheng Ye, Peng He, Runzhou Wu, Chao Zhang, Yonghao Tan, Jie Xiao, Yangyu Tao, Jianchen Zhu, Jinbao Xue, Kai Liu, Chongqing Zhao, Xinming Wu, Zhichao Hu, Lei Qin, Jianbing Peng, Zhan Li, Minghui Chen, Xipeng Zhang, Lin Niu, Paige Wang, Yingkai Wang, Haozhao Kuang, Zhongyi Fan, Xu Zheng, Weihao Zhuang, YingPing He, Tian Liu, Yong Yang, Di Wang, Yuhong Liu, Jie Jiang, Jingwei Huang, Chunchao Guo
TL;DR
Hunyuan3D 2.0 addresses limited progress in large-scale 3D generation by combining open-source shape and texture foundation models. Its staged, geometry-conditioned pipeline generates high-resolution textured assets, and evaluations report superiority in alignment, fine-grained detail, texture quality, and user preference. The reported limitation evidence identifies oversaturated colors and geometric misalignment in earlier score-distillation texture methods.
Problem
Large-scale 3D generation has made limited progress, despite rapid diffusion-model advances in image and video generation.
Method
The system separates image-conditioned bare-mesh generation with Hunyuan3D-DiT from geometry-conditioned texture synthesis with Hunyuan3D-Paint.
Results
Hunyuan3D 2.0 surpasses comparative methods in condition alignment, fine-grained details, textured-asset quality, and human preference ratings.
Takeaways & Limitations
The released system provides open-source foundation models for generating high-resolution textured 3D assets and texturing both generated and handcrafted meshes.
Takeaways & Limitations
Earlier score-distillation texture methods are limited by over-saturated colors and misalignment with geometry.
Abstract
from arXiv · showhide
We present Hunyuan3D 2.0, an advanced large-scale 3D synthesis system for generating high-resolution textured 3D assets. This system includes two foundation components: a large-scale shape generation model -- Hunyuan3D-DiT, and a large-scale texture synthesis model -- Hunyuan3D-Paint. The shape generative model, built on a scalable flow-based diffusion transformer, aims to create geometry that properly aligns with a given condition image, laying a solid foundation for downstream applications. The texture synthesis model, benefiting from strong geometric and diffusion priors, produces high-resolution and vibrant texture maps for either generated or hand-crafted meshes. Furthermore, we build Hunyuan3D-Studio -- a versatile, user-friendly production platform that simplifies the re-creation process of 3D assets. It allows both professional and amateur users to manipulate or even animate their meshes efficiently. We systematically evaluate our models, showing that Hunyuan3D 2.0 outperforms previous state-of-the-art models, including the open-source models and closed-source models in geometry details, condition alignment, texture quality, and etc. Hunyuan3D 2.0 is publicly released in order to fill the gaps in the open-source 3D community for large-scale foundation generative models. The code and pre-trained weights of our models are available at: https://github.com/Tencent/Hunyuan3D-2
1 Introduction
Hunyuan3D 2.0 addresses limited progress in large-scale 3D generation by introducing open-source foundation models for shape creation and texture synthesis. Its evaluations report stronger alignment, fine-grained detail, and user preference than leading alternatives.
- Motivation: Automated high-resolution 3D asset creation is difficult, time-consuming, costly, and dependent on substantial digital-content expertise.The production pipeline can include sketch design, digital modeling, and texture mapping.
- Motivation: 3D generation has progressed relatively slowly compared with image and video generation in the era of large models and big data.The passage characterizes the field as making only gradual progress through a handful of works.
- Contribution: Hunyuan3D 2.0 introduces two open-source foundation models: Hunyuan3D-DiT for generative shape creation and Hunyuan3D-Paint for generative texture synthesis.The system is presented as a response to the need for strong open-source foundational models in 3D.
- Approach: The system separates bare-mesh creation from texture-map synthesis, enabling texture generation for both generated and handcrafted meshes.The two-stage design decouples shape and texture generation difficulties.
- Evaluation: Hunyuan3D 2.0 is evaluated against commercial and open-source systems across textured meshes, bare meshes, texture maps, and 300 user-study cases.The study involved 50 participants and reports advantages in condition alignment, fine-grained detail, and human preference.
2 Hunyuan3D 2.0 Architecture
Hunyuan3D 2.0 uses a two-component pipeline: Hunyuan3D-DiT generates a bare mesh from an input image, and Hunyuan3D-Paint synthesizes textures from geometry-conditioned multi-view outputs.
- Shape generation: Hunyuan3D-DiT first generates a high-fidelity bare mesh from the input image.The shape-generation model is paired with Hunyuan3D-ShapeVAE.
- Texture generation: Hunyuan3D-Paint then uses the generated mesh and strong geometric priors to produce self-consistent multi-view outputs for high-definition texture baking.The geometry conditions include normal maps and position maps.
3 Generative 3D Shape Generation
The shape-generation stack compresses polygon meshes into latent tokens, emphasizing high-frequency surface details, then uses a flow-based diffusion transformer conditioned on images to generate high-resolution shapes.
- Hunyuan3D-ShapeVAE: Hunyuan3D-ShapeVAE compresses polygon-mesh shapes into continuous latent token sequences and decodes them through a neural field into geometry.The decoder predicts a Signed Distance Function that can be converted into a triangle mesh with marching cubes.
- Hunyuan3D-ShapeVAE: Importance sampling adds surface points from edges and corners alongside uniformly sampled points to better capture complex-object details.The point-query construction applies Farthest Point Sampling separately to the uniform and importance-sampled point clouds.
- Hunyuan3D-ShapeVAE: The encoder uses cross-attention and self-attention over encoded point clouds and point queries to obtain a variational latent shape representation.The latent representation predicts a mean and variance for the shape embedding.
- Hunyuan3D-ShapeVAE: ShapeVAE training combines SDF reconstruction and KL-divergence losses, with reconstruction evaluated on randomly sampled spatial and surface points.A multi-resolution token-length strategy trades computation cost against reconstruction quality, with a released maximum sequence length of 3072.
- Hunyuan3D-DiT: Hunyuan3D-DiT is a flow-based diffusion model that generates high-fidelity, high-resolution 3D shapes from image prompts.Its dual- and single-stream transformer blocks process shape and condition tokens separately or jointly to support modality interaction.
- Hunyuan3D-DiT: The model injects image conditions through a pretrained image encoder using large 518 × 518 inputs and object-background preprocessing.The preprocessing removes the background, standardizes object size, centers the object, and fills the background white.
- Hunyuan3D-DiT: Flow matching trains the transformer to predict a velocity field along an affine path from Gaussian noise toward the data distribution.At inference, a Gaussian start point is evolved with a first-order Euler ODE solver under the conditioned diffusion model.
4 Generative Texture Map Synthesis
Hunyuan3D-Paint synthesizes high-resolution texture maps through preprocessing, geometry-conditioned multi-view generation, and dense-view texture baking. Its design combines image alignment, geometric following, multi-view consistency, and coverage-aware viewpoint selection.
- Pipeline: The texture pipeline has three stages: preprocessing, multi-view image synthesis, and texture baking based on dense multi-view inference.The method also describes training and text- and image-to-texture details separately.
- Pre-processing: Image delighting converts illuminated references to an unlit state, reducing the risk that lighting and shadows become baked into texture maps.The procedure is applied before multi-view generation.
- Pre-processing: Geometry-aware viewpoint selection starts from four orthogonal views and greedily adds viewpoints until selecting 8 to 12 views.The strategy seeks broad geometric-surface coverage while limiting texture-generation cost.
- Multi-view generation: Hunyuan3D-Paint uses reference conditioning, multi-task attention, geometry conditions, and camera embeddings to align generated views with images and meshes.Reference attention integrates the input image, while multi-view attention promotes consistency across generated views.
- Texture baking: Dense-view inference supports arbitrary specified viewpoints by training with randomly selected subsets of six views from 44 preset viewpoints.This reduces self-occlusion gaps and alleviates later inpainting demands.
- Texture baking: Generated multi-view images are enhanced with single-image super-resolution, unwrapped into UV textures, and filled with an inpainting procedure for uncovered patches.The super-resolution approach is reported to maintain multiview consistency.
- Applications: Hunyuan3D-Paint supports high-quality texture generation for generated meshes and arbitrary geometric models guided by text or image inputs.This extends the system beyond texturing only its own generated meshes.
5 Evaluations
The evaluation examines shape generation, texture synthesis, end-to-end textured asset generation, and user preferences. Across these settings, Hunyuan3D 2.0 is reported to produce detailed, condition-following geometry and high-quality, flexible textures.
- The evaluation covers 3D shape generation, texture map synthesis, and end-to-end textured 3D asset generation.
- 3D Shape Generation: Hunyuan3D-ShapeVAE outperforms reconstruction baselines while recovering fine-grained details and neat space without floaters.
- 3D Shape Generation: Hunyuan3D-DiT produces the most condition-following shape-generation results, including detailed surface features and holeless bare meshes.
- Texture Map Synthesis: Hunyuan3D-Paint achieves the best reported texture generative quality and semantic following, producing clear, condition-conforming, and intricate texture maps.
- Texture Map Synthesis: Hunyuan3D-Paint generates seamless, lighting-invariant textures for generated or handcrafted meshes and supports texture reskinning.
- Textured 3D Assets Generation: End-to-end comparisons report that Hunyuan3D 2.0 surpasses open- and closed-source baselines in textured asset quality and condition following.
- Textured 3D Assets Generation: A user study with 50 volunteers evaluating 300 results found that Hunyuan3D 2.0 outperforms comparison methods, particularly in adherence to image conditions.
6 Hunyuan3D-Studio
Hunyuan3D-Studio extends Hunyuan3D 2.0 into an accessible production platform with sketch conversion, mesh simplification, and character animation tools.
- Platform overview: Hunyuan3D-Studio provides experts and novices with tools for 3D generation production and research, including Sketch-to-3D, Low-polygon Stylization, and Autonomous Character Animator.These features aim to streamline 3D creation and broaden access.
- Sketch-to-3D: The Sketch-to-3D module converts sketches into detailed images while preserving contours, then synthesizes high-resolution, high-fidelity textured 3D assets.This workflow reduces the barrier to content creation.
- Sketch-to-3D: Sketch-to-3D generates detailed, realistic assets that remain closely consistent with the original sketches.The module supports both professional creation and low-barrier use by ordinary users.
- Low-polygon Stylization: Low-polygon Stylization converts dense generated meshes into low-polygon meshes through geometric editing and texture preserving.Reducing face count lowers computational costs in graphics pipelines.
- Low-polygon Stylization: Geometric editing merges mesh vertices under an optimization criterion, producing models represented by only dozens of triangles.The resulting low-polygon mesh can differ substantially in vertices and faces from the dense mesh.
- Autonomous Character Animator: Hunyuan3D-Studio adds character animation by predicting skeleton skinning and using motion retargeting with motion templates to drive generated characters.The function extends static asset generation toward animation production.
7 Related Work
Related work spans 3D representations, generative models, datasets, and texture synthesis, while Hunyuan3D 2.0 combines large-scale data with coordinated shape and texture generation.
- Representations: 3D shape-generation research uses diverse representations, including voxels, point clouds, polygon meshes, and implicit functions, because 3D lacks a universal storage format.This representational diversity distinguishes 3D generation from other modalities.
- Shape Generative models: Earlier shape-generation systems used variational autoencoders, GANs, normalizing flows, and autoregressive modeling, while diffusion models later expanded 3D generation capabilities.The passage situates diffusion models within the evolution of generative paradigms.
- Large-scale Dataset: Large-scale 3D datasets remain smaller than datasets in language and image generation, although Objaverse and Objaverse-XL helped enable scaling for shape generation.Hunyuan3D 2.0 leverages these open-source datasets.
- Texture Synthesis: Texture synthesis research includes categorical generation, geometry-conditioned multi-view generation, score distillation, and optimization-free approaches based on image diffusion models.These approaches target text or image guidance for texturing meshes.
- Texture Synthesis: Earlier categorical texture methods failed to generalize across object categories, limiting their applicability in production environments.This limitation motivated broader texture-generation approaches.
- Texture Synthesis: Score-distillation methods often produce over-saturated colors and geometry misalignment, while training-free methods inherit limitations from forward-facing viewpoint bias.Training multi-view diffusion models improves texture consistency over training-free approaches.
- Texture Synthesis: Hunyuan3D-Paint uses multi-task attention to achieve multi-view consistency and image alignment simultaneously, producing consistent textures aligned with the reference image.The model is trained on a large 3D rendering dataset.
8 Conclusion
Hunyuan3D 2.0 is an open-source system for generating high-resolution textured meshes from images through coordinated shape and texture models.
- Conclusion: Hunyuan3D 2.0 introduces Hunyuan3D-ShapeVAE, Hunyuan3D-DiT, and Hunyuan3D-Paint for image-to-textured-mesh generation.ShapeVAE compresses 3D objects into latent tokens, DiT generates image-aligned shapes, and Paint textures generated or user-crafted meshes.
- Conclusion: The combined shape and texture models produce high-resolution, high-fidelity textured 3D assets from a single image.The authors position the released system as a baseline for large-scale open-source 3D foundation models.
9 Contributors
The report credits sponsors, project leaders, and contributors across data, shape generation, texture synthesis, downstream tasks, and studio development.
- Sponsors: Project sponsors are Jie Jiang, Yuhong Liu, Di Wang, Yong Yang, and Tian Liu.
- Project Leaders: Project leaders are Chunchao Guo, Jingwei Huang, and Zibo Zhao.
- Core Contributors: Core contributors are listed for data, shape generation, texture synthesis, downstream tasks, and Studio development.The listed teams include contributors such as Lifu Wang, Zeqiang Lai, Shuhui Yang, Xianghui Yang, and Zhuo Chen.