3DTopia-XL: Scaling High-quality 3D Asset Generation via Primitive Diffusion

1. Introduction

• High-quality assets for films, gaming, and virtual reality require substantial manual labor and expertise, motivating automatic generation from visual or textual inputs.Source: §1 · pp. 1–2
• Existing methods trade off speed, geometry, diversity, and PBR capability: SDS methods require time-consuming optimization, reconstruction methods face representation limits, and few native generators produce high-quality PBR assets.The comparison covers SDS, sparse-view reconstruction, and native 3D generative models.Source: §1 · p. 2
• PrimX encodes shape, albedo, and material in a compact N × D tensor, using surface-anchored primitives with spatially varied SDF, RGB, and material payloads.The representation supports differentiable rendering and is reported to tensorize from a textured mesh ten times faster than a triplane under the same setting.Source: §1 · p. 2
• Primitive Patch Compression uses a 3D VAE to compress each primitive into latent tokens, while Latent Primitive Diffusion uses DiTs to model global correlations among those tokens.The framework trains at high resolution without super-resolution or post-hoc optimization-based mesh refinement.Source: §1 · p. 2
• PrimX assets are extracted by zero-level contouring the SDF and sampling texture and material in high-resolution UV space for GLB-ready meshes.The authors report that this yields high-quality extraction with considerably fewer vertices.Source: §1 · p. 2
• 3DTopia-XL is reported to achieve superior quality and support image-to-3D and text-to-3D generation, with experiments evaluating both tasks.The paper also reports qualitative and quantitative experiments and ablations targeting the efficiency-quality trade-off.Source: §1 · p. 2

Sparse-view Reconstruction Models

• Sparse-view reconstruction can be fast and generalizable, but triplane representation inefficiency limits quality and deterministic methods have low diversity.Multi-view diffusion inputs can improve visual quality, though their generated views may lack accurate 3D consistency.Source: §2 · pp. 2–3

Native 3D Diffusion Models

• Native 3D representations trade off memory and surface quality: voxels face memory demands, point clouds struggle with watertight solids, and implicit representations balance memory and quality.Mesh- and primitive-based methods are also described as struggling with generalization or high-quality assets.Source: §2 · p. 3
• Latent 3D diffusion can improve training effectiveness and generalization, but prior methods may produce low-resolution results or omit PBR materials.The paper proposes PrimX as a representation that can be computed from textured meshes and unpacked into high-resolution geometry with PBR materials.Source: §2 · p. 3

3.1. PrimX: An Efficient Representation for Shape, Texture, and Material

• PrimX encodes a textured mesh as a compact tensor of surface-anchored voxel primitives, each carrying position, scale, and spatial payloads for SDF, RGB, and material.The representation is designed to be parameter-efficient, tensorizable, and differentiably renderable.Source: §3.1.1 · p. 3, §3.1 · p. 3

3.1.1. Definition

• The primitive field combines voxel-grid interpolants with spatial weights, then renders RGB along camera rays using an SDF-derived opacity field.The paper also extracts meshes by zero-level contouring and samples albedo and material maps in high-resolution UV space.Source: §3.1.1 · pp. 3–4

3.1.2. Computing PrimX from Textured Mesh

• PrimX fitting initializes primitive positions, scales, and payloads from the mesh, then finetunes the representation against SDF, albedo, and material values.Finetuning uses separate 1,000-iteration stages for geometry and appearance losses.Source: §3.1.2 · pp. 4–5

3.2. Primitive Patch Compression

• Primitive Patch Compression uses a 3D VAE to compress each primitive’s voxel payload into latent tokens for generative modeling.Its local compression yields a lower-dimensional parameter space for diffusion.Source: §3.2 · p. 5

3.3. Latent Primitive Diffusion

• Latent Primitive Diffusion denoises sets of latent PrimX primitives, treating each primitive as a Transformer token to model inter-primitive correlations.The largest model is a 28-layer Transformer with cross-attention for conditioning and self-attention for primitive correlations.Source: §3.3 · pp. 5–6
• Under a 1.05M-parameter budget, PrimX has the best fitting quality, especially for geometry, and the fastest runtime among evaluated representations.The representation evaluation reports highest geometry and appearance fidelity alongside runtime efficiency.Source: Table 1 · p. 6, Figure 4 · p. 6

4.1. Representation Evaluation

• PrimX achieves the least approximation error among the evaluated representations and converges nearly 7 times faster than the second-best method.The evaluation fixes the parameter count across representations; qualitative comparisons report smooth geometry and fine-grained details for PrimX.Source: §4.1 · p. 6

4.2. Image-to-3D Generation

• 3DTopia-XL achieves the best visual and geometry quality among the compared image-to-3D methods, with spatially varied PBR assets producing vivid reflectance under harsh illumination.The comparison includes sparse-view reconstruction and image-conditioned diffusion methods; CraftsMan has comparable surface quality but generates shapes without textures or materials.Source: §4.2 · p. 7, Figure 5 · p. 7

4.3. Text-to-3D Generation

• 3DTopia-XL achieves better alignment between input text and generated-asset renderings than Shap-E and 3DTopia on unseen prompts.The evaluation uses CLIP Score; Table 4 reports the results for input prompts and front-view renderings.Source: §4.3 · pp. 7–8, Table 4 · p. 8

4.4. Further Analysis

• At a fixed parameter count, larger sets of compact primitives improve approximation, while longer sequences increase DiT computation and training difficulty.The final VAE setting, N = 2048 with compression rate f = 48, achieves the optimal reported reconstruction; the N = 256, f = 48 setting has lower reconstruction quality and diffusion convergence difficulty.Source: §4.4 · p. 8, Table 3 · p. 8
• The paper demonstrates 3D inpainting and interpolation between text-to-3D samples as applications of native 3D diffusion.Further analyses of model scaling, sampling diversity, initialization, and user study are deferred to the supplementary material.Source: §4.4 · p. 8, Figure 7 · p. 8

• PrimX encodes shape, albedo, and material in a compact N×D tensor for high-resolution geometry with PBR assets.The representation is described as parameter-efficient, tensorial, and renderable.Source: §5 · p. 8
• Latent Primitive Diffusion builds scalable 3D generation on top of PrimX.Source: §5 · p. 8
• Practical export techniques convert generated assets into PBR assets ready for graphics pipelines.Source: §5 · p. 8

A.1. Further discussions

• Section A.1 discusses PrimX's differences from existing 3D representations and its limitations.Source: A. Appendix · p. 12

A.2. Further experiments and evaluations

• Section A.2 covers additional dataset evaluations, a user study, scaling and diversity analyses, ablations, and qualitative results.Source: A. Appendix · p. 12

A.3. Implementation details

• Section A.3 documents implementation details, including datasets, model settings, and reversible PrimX-mesh conversion algorithms.The supplementary material also includes a demo video.Source: A. Appendix · p. 12

PrimX v.s. Implicit Vector Set

• Unlike implicit vector sets [74, 77], PrimX jointly encodes shape, texture, and material, avoiding a separate shape-and-texture generation stage [77].The paper notes that surface-aligned texture and material may be more expensive and difficult for dense whole-volume modeling.Source: §A.1.1 · p. 12

PrimX v.s. PrimDiffusion

• Unlike PrimDiffusion [8], PrimX needs no template mesh and simplifies primitive parameters to position, one scale, and voxelized payload.PrimDiffusion uses template-anchored primitives; PrimX models geometry as SDF and can learn from 3D and 2D data, while the compared work models opacity and learns from 2D images.Source: §A.1.1 · p. 12

PrimX v.s. M-SDF

• Unlike shape-only M-SDF [72], PrimX represents shape, texture, and material together and can be differentiably rendered into 2D images.Source: §A.1.1 · p. 12

PrimX v.s. 3DGS

• PrimX uses N = 2048 primitives, versus hundreds of thousands of Gaussians for high-quality 3DGS objects, reducing sequence length for DiT modeling [23] [45].The paper characterizes PrimX as an interpolation between point-based and voxel-based representations, grouping primitives into structured local voxels.Source: §A.1.1 · p. 12

A.1.2. Limitations and Future Work

• The authors say quality still has room for improvement despite training on a considerably large-scale dataset.PrimX's differentiable rendering may support learning from mixed 2D and 3D data, which the authors suggest could address the lack of high-quality 3D data.Source: §A.1.2 · p. 13

A.2.1. Quantitative Comparisons on Objaverse and GSO

• 3DTopia-XL achieves the best 3D geometric quality, while sparse-view reconstruction methods have slightly better visual quality but worse 3D metrics.The authors attribute the visual-quality difference to reconstruction methods having more input visual information, and note their cascaded pipelines can yield 3D distortions from multiview inconsistency.Source: §A.2.1 · p. 13
• 3DTopia-XL is the only evaluated method capable of producing PBR materials from images or text.The evaluations cover image-to-3D on Objaverse and GSO against sparse-view reconstruction and native 3D diffusion methods.Source: §A.2.1 · pp. 13, 16

A.2.2. Additional Comparisons

• Unique3D excels in texture quality but suffers geometry artifacts in novel views, which the authors associate with inconsistent 2D diffusion outputs.The authors suggest PrimX-based feedforward reconstruction could provide geometry initialization and a mesh constraint to shorten Unique3D's optimization time.Source: §A.2.2 · p. 16
• A casual phone capture demonstrates 3DTopia-XL's generalizability to real-world inputs.The example is shown in Fig. 10(b).Source: §A.2.2 · p. 16

A.2.3. User Study

• 3DTopia-XL has the highest user preference among the compared methods across four image-to-3D dimensions.The study used 48 paired samples presented to 27 volunteers; geometry quality and physically based materials are identified as keys to the best overall rendering preference.Source: §A.2.3 · p. 16, Figure 11 · p. 17

A.2.4. Scaling

• Scaling model depth and width improves both FID measures, and a longer sequence leads to better performance at a fixed 1.05M parameter budget.The measures are Latent-FID and Rendering-FID; Table 7 compares {N = 256, a = 16} with {N = 2048, a = 8}.Source: §A.2.4 · p. 16, Table 7 · p. 16, Figure 13 · p. 18

A.2.5. Sampling Diversity

• With the same input image, varying random seeds produces diverse geometry and spatially varied PBR materials.The paper also presents image- and text-conditioned generation results.Source: §A.2.5 · p. 16, Figure 14 · p. 19, §A.2.11 · p. 19

A.2.6. Generation of Inner Geometric Structures

• PrimX generates well-defined, diverse inner structures from images or text, with potential use in physical simulation and compositional object generation.Additional examples include thin structures.Source: §A.2.6 · p. 16, Figure 10 · p. 17, Figure 15 · p. 19

A.2.7. Ablation study on PrimX Initialization

• Uniform-plus-farthest initialization achieves quality comparable to Coverage while avoiding its unnecessary computation overhead.Direct farthest-point sampling is topology-sensitive and can undersample flattened surfaces, leaving gaps; Table 8 reports quantitative evaluations of the initialization strategies.Source: §A.2.7 · p. 16, Table 8 · p. 18

A.2.8. Ablation study on VAE Designs

• Patch-wise VAE compression outperforms PCA and DCT at higher compression, while PCA loses smooth geometry and DCT fails at stronger compression.DCT achieves reasonable quality at the lower compression rate of 3072 to 384.Source: §A.2.8 · p. 17, Figure 17 · p. 20

A.2.9. Ablation study on PrimX2Mesh Algorithm

• PrimX2Mesh's UV texturing, 1024 × 1024 UV space, and adjacency-based inpainting avoid the PBR, texture-noise, and zig-zag artifacts seen in the ablations.Vertex coloring and omitting materials fail to support PBR modeling; low-resolution UVs introduce texture and material noise, while omitting inpainting produces zig-zag artifacts.Source: §A.2.9 · p. 18, Figure 18 · p. 21

A.2.10. PrimX can Learn from Both 2D and 3D

• Differentiable rendering lets image-reconstruction gradients refine PrimX texture, supporting learning from 2D images as well as 3D data.The authors also identify potential use in sparse-view reconstruction models.Source: §A.2.10 · pp. 18–19, Figure 19 · p. 21