Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation

1 Introduction

• Automated high-resolution 3D asset generation addresses a complex, costly, and time-consuming process requiring expertise in design, modeling, and texture mapping.3D assets are used in gaming and film as well as physical simulation and embodied AI.Source: §1 · p. 3
• Hunyuan3D 2.0 introduces open-source Hunyuan3D-DiT for shape generation and Hunyuan3D-Paint for texture synthesis to address limited progress in 3D generation.The introduction frames a strong open-source foundation model as a driver of progress in large-model domains.Source: §1 · p. 3
• Its two-stage pipeline first creates a bare mesh and then synthesizes its texture, enabling texturing of generated or handcrafted meshes.Separating shape and texture generation decouples their respective challenges.Source: §1 · p. 3
• Hunyuan3D-ShapeVAE captures mesh detail, while Hunyuan3D-DiT applies flow matching in the VAE latent space and Hunyuan3D-Paint bakes mesh-conditioned multi-view images into high-resolution texture maps.ShapeVAE uses mesh-surface importance sampling and variational token length; DiT uses a dual-single stream transformer.Source: §1 · p. 3
• Hunyuan3D 2.0 outperformed evaluated alternatives in conditional-image alignment, fine-grained mesh details, and human preference ratings.The comparison covered three commercial closed-source end-to-end products, Trellis [101], and separate shape and texture models, using visual and quantitative evaluations plus a 300-case study with 50 participants.Source: §1 · p. 3

2 Hunyuan3D 2.0 Architecture

• The shape-generation model comprises Hunyuan3D-ShapeVAE and Hunyuan3D-DiT, which generates a high-fidelity bare mesh from the input image.The paper discusses these components in Section 3.Source: §2 · p. 3
• The pipeline first generates a bare mesh from an input image with Hunyuan3D-DiT, then uses Hunyuan3D-Paint to create a textured asset.Figure 2 presents the overall architecture and shows the handoff from bare-mesh generation to texture generation.Source: §2 · p. 3, Figure 2 · p. 4
• Hunyuan3D-Paint uses mesh normal and position maps as geometry conditions to generate multi-view images for texture baking.Leveraging geometric priors and the input image, it produces self-consistent views used to bake high-definition texture maps.Source: §2 · p. 3, Figure 2 · p. 4

3 Generative 3D Shape Generation

• Hunyuan3D-DiT predicts ShapeVAE latent token sequences from a user-provided image, which the VAE decoder converts into a polygon mesh within a latent-diffusion pipeline [74].The system comprises a mesh-compressing Hunyuan3D-ShapeVAE and a flow-based Hunyuan3D-DiT trained on its latent space.Source: §3 · p. 4

3.1 Hunyuan3D-ShapeVAE

• ShapeVAE uses vector-set tokens and a variational transformer to encode surface points and decode an SDF that marching cubes converts into a triangle mesh.The encoder takes sampled surface-point coordinates and normals; the decoder predicts the shape’s signed distance function.Source: §3.1 · pp. 4–5
• Importance sampling adds points at mesh edges and corners to uniform surface samples, helping represent complex regions that uniform-only encoding may fail to reconstruct.The encoder combines both point sets and uses cross-attention with point queries followed by self-attention to form the latent shape representation.Source: §3.1 · pp. 4–5, Figure 3 · p. 5
• ShapeVAE training combines sampled-point SDF reconstruction with KL regularization, and randomizes token-sequence length up to 3072 to balance computation and reconstruction quality.Shorter sequences reduce computation, while longer sequences support higher-quality reconstruction; the released model’s longest sequence supports fine-grained shape generation.Source: §3.1 · p. 5

3.2 Hunyuan3D-DiT

• DiT uses dual-stream blocks that process latent and image-condition tokens separately before attention interaction, followed by single-stream blocks that concatenate tokens for spatial and channel attention.It omits latent positional embeddings because the ShapeVAE tokens themselves represent position and occupancy in the 3D grid.Source: §3.2 · pp. 5–6, Figure 4 · p. 6
• DiT conditions generation on DINOv2 Giant image tokens from 518 × 518 inputs, after removing the background and centering and resizing the object against white.The preprocessing is intended to reduce background impact and increase the effective resolution of the input image.Source: §3.2 · p. 6
• DiT is trained with flow matching and generates shapes by integrating from Gaussian noise with a first-order Euler ODE solver.The model uses an affine path with a conditional optimal-transport schedule during flow-matching training.Source: §3.2 · p. 6

4.1 Pre-processing

• Preprocessing removes illumination and shadows from the reference image and selects 8 to 12 geometry-aware viewpoints, starting with four orthogonal views and greedily adding views that cover unseen regions.The delighting model is trained on paired renders under random HDRI and even white lighting, enabling illumination-invariant synthesis and reducing the need for texture inpainting.Source: §4.1 · pp. 7–8

4.2 Hunyuan3D-Paint

• Hunyuan3D-Paint combines reference-image conditioning, multiview attention, and geometry and camera conditioning to generate image-aligned, geometry-following, multiview-consistent images.Its reference branch uses noiseless VAE image features at timestep 0 and frozen SD2.1 weights; separate reference and multiview attention modules handle prompt fidelity and view consistency. Canonical normal and coordinate maps are concatenated with latent noise, while a learnable camera embedding supplies viewpoint information.Source: §4.2 · pp. 8–9

4.3 Texture Baking

• Dense-view inference reduces later inpainting by training on six randomly selected viewpoints from 44 and generating images for any specified viewpoints.A pretrained single-image super-resolution model enhances each view while maintaining multiview consistency; uncovered UV texels are filled using distance-weighted textures from connected textured vertices.Source: §4.3 · p. 9

4.4 Implementation Details

• Hunyuan3D-Paint supports text- or image-guided texturing of arbitrary geometric models, including re-skinning the same geometry with matched or mismatched images.The multiview model is trained from a Stable Diffusion 2 v-model checkpoint on white-light renders at 512 × 512 resolution for 80,000 steps, with batch size 48 and learning rate 5 × 10−5.Source: §4.4 · pp. 9–10

5.1 3D Shape Generation

• Hunyuan3D-ShapeVAE outperforms all reconstruction baselines and recovers fine-grained detail in clean meshes without floaters.Comparisons with 3DShape2VecSet and Michelangelo demonstrate the effectiveness of importance sampling.Source: §5.1 · p. 10, Figure 6 · p. 11
• Hunyuan3D-DiT produces the most condition-following shape-generation results, with detailed, complete meshes that follow image prompts.Visual examples highlight human faces, surface bumps, logo text, layouts, and holeless meshes.Source: §5.1 · p. 10, Table 2 · p. 12, Figure 7 · p. 12

5.2 Texture Map Synthesis

• Hunyuan3D-Paint achieves the best texture-map generative quality and semantic following, with examples of clear, intricate, condition-conforming textures.Examples include fish, rabbit, football, castle, and bear textures.Source: Table 3 · p. 13, §5.2 · p. 13, Figure 8 · p. 14
• Hunyuan3D-Paint produces seamless, lighting-invariant textures for bare or hand-crafted meshes and can reskin meshes with different prompts.Figure 9 shows different texture maps generated for two meshes.Source: §5.2 · p. 13, Figure 9 · p. 15

5.3 Textured 3D Assets Generation

• Hunyuan3D 2.0 surpasses the compared models in textured 3D asset quality and condition following, with visual examples showing detailed geometry and accurate textures.The examples also include high-resolution, high-fidelity outputs for complex actions or scenes.Source: §5.3 · p. 13, Table 4 · p. 15, Figure 11 · p. 16
• In a study of 300 results rated by 50 volunteers, Hunyuan3D 2.0 outperformed comparison methods, particularly in adherence to image conditions.Ratings covered visual quality, adherence to image conditions, and overall satisfaction.Source: §5.3 · p. 15, Figure 10 · p. 15

6.1 Sketch-to-3D

• Sketch-to-3D converts sketches into detailed images while preserving their contours, then generates realistic, textured 3D assets that remain consistent with the sketches.The module lowers barriers to content creation; earlier approaches [1] [32] relied on small generative or reconstruction models trained on limited datasets.Source: §6.1 · p. 17

6.2 Low-polygon Stylization

• Low-polygon stylization reduces dense meshes to production-ready low-polygon assets through geometric editing and texture preservation.It merges vertices using a traditional method, then uses a KD-tree to transfer nearby texture colors to the simplified mesh and bakes its texture map.Source: §6.2 · p. 17

6.3 3D Character Animation

• Character animation extracts mesh features, uses a GNN to predict skeleton key points and skinning weights, then retargets motion templates to drive generated characters.The module extends static Hunyuan3D 2.0 assets for applications such as game development and animation production [57].Source: §6.3 · pp. 17–18

7.1 3D Shape Generation

• Hunyuan3D 2.0 represents implicit shapes with vector sets, which improve efficiency over structured triplanes or sparse volumes while preserving spatial priors less directly.Vector-set representations were pioneered by 3DShape2VecSet and used by Michelangelo, CLAY, and Dora [11].Source: §7.1 · p. 18
• Shape generation progressed from VAE, GAN, normalizing-flow, and autoregressive models to diffusion and Transformer architectures; Hunyuan3D 2.0 uses a scalable flow-based Transformer.The cited early paradigms include [98] [76] [110], while diffusion models [16] enabled stable cross-category generation.Source: §7.1 · p. 18
• The release of Objaverse [20] and Objaverse-XL [19] helped enable scaling in 3D generation, and Hunyuan3D 2.0 releases its system to advance the open-source community.The paper links these open datasets to its high-fidelity, high-resolution assets and frames its release as a contribution to open-source 3D generation.Source: §7.1 · p. 18

7.2 Texture Map Synthesis

• Early text-to-texture models trained on category-specific data [10] [5] [23] [80] [26] [27] struggled to generalize; later diffusion-based methods often generate geometry-conditioned multi-view images.Score-distillation approaches [87] [51] [62] [70] can produce over-saturated colors and geometry misalignment.Source: §7.2 · pp. 18–19
• Training-free texture methods such as TEXTure [73] use view masks or synchronized denoising, but Stable Diffusion’s viewpoint bias can cause Janus problems and multi-view artifacts.Training multi-view diffusion models [61] [2] has shown stronger texture consistency than training-free approaches.Source: §7.2 · p. 19
• Image-guided texture methods often prioritize semantic rather than precise reference alignment, whereas Hunyuan3D-Paint injects image details to enrich well-aligned geometry.The paper notes that image guidance can provide more diverse information than text prompts and motivates faithfully following reference details.Source: §7.2 · p. 19
• Multi-view methods use 3D-aware attention masks to coordinate view latents, including Zero123++ [78], MVDiffusion [89], and MVAdapter [36]; Hunyuan3D-Paint adds multi-task attention for consistency and image alignment.The framework is trained on a large 3D rendering dataset and is described as producing consistent textures aligned to the reference image.Source: §7.2 · p. 19

• Hunyuan3D-ShapeVAE uses novel importance sampling to compress each 3D object into a few latent tokens while minimizing reconstruction losses.Source: §8 · p. 20
• Hunyuan3D-DiT generates visually appealing shapes that align precisely with input images.Source: §8 · p. 20
• Hunyuan3D-Paint creates textures for both generated meshes and user-crafted meshes.Source: §8 · p. 20
• Together, the shape and texture models produce high-resolution, high-fidelity textured 3D assets from a single image.The authors hope Hunyuan3D 2.0 will serve as a robust baseline for large-scale 3D foundation models in the open-source community and facilitate future research.Source: §8 · p. 20