Step1X-3D: Towards High-Fidelity and Controllable Generation of Textured 3D Assets

• 3D generation remains less mature than other generative modalities, constrained by scarce high-quality data, complex geometry-texture assessment, and gaps in reproducibility and controllability.Large web-sourced collections can have heterogeneous quality, while some systems lack training code or conditional generation support.Source: §1 · p. 2
• Step1X-3D refines over 5M assets into 2M high-quality assets through texture filtering and watertight mesh conversion, with almost 800K public-data assets slated for release.The pipeline assesses texture resolution, normal-map accuracy, material transparency, and surface complexity.Source: §1 · pp. 2–3
• Step1X-3D separates geometry and texture synthesis to produce topologically sound meshes with geometrically aligned textures, supporting image and semantic inputs and planned release of models, training code, and LoRA adaptation modules.Source: §1 · pp. 2–3
• Its geometry stage encodes point clouds into latent vectors and uses a VAE-DiT diffusion model to generate TSDFs, with sharp-edge sampling and dual cross-attention to preserve detail.Adapting MMDiT to the 1D latent space enables transfer of 2D parameter-efficient methods such as LoRA to 3D synthesis.Source: §1 · p. 2
• The texture pipeline conditions multi-view image generation on input images and geometric maps, synchronizes texture-space latents, then uses back-projection and inpainting to complete UV maps.Trimesh post-processing and xAtlas UV parameterization precede synthesis; the stages address surface artifacts, cross-view coherence, and occlusion artifacts.Source: §1 · pp. 2–3
• Step1X-3D surpasses existing open-source 3D generation approaches in asset quality and achieves performance comparable to proprietary state-of-the-art solutions.Source: §1 · p. 3

2.1 Optimization-based 3D Generation

• Optimization-based methods align rendered 3D representations with prompt-derived 2D features, but their long optimization cycles, weak parameter generalization, and inferior quality hinder practical scalability.They can process arbitrary text prompts but have limited support for image-conditioned inputs.Source: §2.1 · p. 3

2.2 Feed-forward 3D Generation

• Feed-forward methods synthesize 3D assets by forward propagation, avoiding iterative optimization and achieving order-of-magnitude acceleration while improving fidelity and generalization.They divide into 2D-lifting-to-3D and 3D-native generation paradigms.Source: §2.2 · p. 3
• 2D-lifting methods transfer 2D priors through single-image reconstruction or multi-view diffusion followed by 3D reconstruction, but often neglect geometric fidelity and fine detail.Single-stage methods map image embeddings to 3D representations; two-stage methods synthesize multi-view images and reconstruct geometry, with methods targeting view consistency or reconstruction quality.Source: §2.2 · pp. 3–4
• 3D-native methods generate geometry directly and use it to guide texture generation, enabling more precise geometry than 2D-lifting approaches.Their progress has been driven by expanded datasets and algorithmic advances, including larger 3D latent representations, diffusion architectures, and sharp-edge sampling for geometric detail.Source: §2.2 · p. 4
• Texture methods use diffusion, UV refinement, or synchronization to create textures, while Step1X-3D adopts 3D diffusion for high-fidelity shapes and textures with controllable modules.Independent view generation can cause consistency problems; synchronization methods address this in latent space or image space, with MVPaint also using 3D point-cloud inpainting.Source: §2.2 · p. 5

3.1 Geometry Data Curation

• The curation pipeline filters low-quality assets, converts non-watertight meshes, and samples surface points, yielding roughly 2 million high-quality 3D assets.Sources include 320k valid Objaverse samples, 480k from Objaverse-XL, and selected data from ABO, 3D-FUTURE, and internal assets.Source: §3.1 · p. 5
• 20% higher watertight conversion success on Objaverse comes from combining winding-number classification with visibility tests to form the occupancy mask.The method uses a generalized winding-number threshold of 0.75 before Marching Cubes reconstruction.Source: §3.1 · p. 6, Figure 3 · p. 6

3.2 Step1X-3D Shape Generation

• The 1D Rectified Flow Transformer generates latent shape sets, conditioning on DINOv2 image tokens and complementary CLIP features for local and global visual information.The model adapts an MMDiT structure for 1D latent space and removes positional embeddings because the latent sets lack explicit spatial correspondence.Source: §3 · p. 5, §3.2 · pp. 7–8
• A transformer-based shape VAE encodes point clouds as latent sets and decodes TSDF geometry, using sharp-edge sampling and dual cross-attention to preserve detail.Latent queries are initialized from uniformly and saliently sampled points; hierarchical volume decoding accelerates inference.Source: §3.1 · p. 6, §3.2 · p. 7

3.3 More flexible control for 3D generation

• LoRA modules trained on geometric labels such as symmetry or detail level enable lightweight domain-specific shape control without fine-tuning the full network.The passage describes skeleton, bounding-box, caption, and IP-image conditions as planned future additions.Source: §3.3 · p. 8

3.4 Training Rectified Flow Model

• Training uses flow matching with increased sampling weight for intermediate timesteps and EMA decay 0.999, followed by two 100k-iteration phases that scale latent-set size from 512 to 2048.The first phase uses learning rate 1e-4 and batch size 1920 on 96 A800 GPUs; the second uses 5e-5 and batch size 960.Source: §3.4 · p. 9

4.1 Geometry Postprocess

• Mesh post-processing verifies watertightness, fills holes, subdivides and smooths faces, then uses xAtlas to generate UV coordinates.The trimesh toolkit supports the optimization pipeline, which aims to improve topological uniformity and minimize UV seam artifacts.Source: §4.1 · p. 9

4.2 Texture datatset preparation

• The texture-training set comprises 30K assets curated from a cleaned 320K Objaverse dataset, rendered in six views with albedo, normal, and position maps at 768 × 768.The six views are front, back, left, right, top, and bottom.Source: §4.2 · p. 9

4.3 Geometry-guided Multi-view Images Generation

• MV-Adapter generates 768×768 multi-view images, preserving SD-XL spatial attention while adding parallelized attention for generalization, view consistency, and conditional adherence.Memory-efficient epipolar attention supports high-resolution generation under training batch-size constraints.Source: §4.3 · p. 10
• Normal maps guide fine geometric detail, while world-coordinate 3D position maps maintain spatial correspondence between textures and mesh vertices across views.Image-based encoders feed both geometric representations into the generator through cross-attention.Source: §4.3 · p. 10
• Texture-space latent alignment synchronizes views within one diffusion backbone, reducing UV-space blur and seams without an auxiliary refinement pipeline.The method jointly optimizes in a unified latent space at 96×96 resolution, which the authors report provides sufficient texture representation capacity.Source: §4.3 · p. 10

4.4 Bake Texture

• Multi-view images are upsampled to 2048 × 2048, projected into UV texture space, then continuity-aware inpainting fills discontinuities and holes while preserving high-frequency detail.The workflow reuses texture baker tools from Hunyuan3D 2.0 [94].Source: §4.4 · p. 11

5.1 The Visual Quality Results of Step1X-3D Assets

• Across varied styles and object layouts, geometry matched input images and plausibly reconstructed occlusions, while textures preserved style and detail with consistent views and geometry alignment.The authors attribute geometry detail and generalization to their 3D diffusion model, VAE architecture, and large-scale high-quality training data. Texture view completion uses normal and position maps as guidance while preserving the original SD-XL parameters.Source: §5.1 · pp. 11–12, Figure 6 · p. 12, Figure 7 · p. 13

5.2 High Controllable 3D generation with LoRA Finetuning

• LoRA fine-tuning on approximately 30,000 Step1O-annotated models enabled symmetry/asymmetry and sharp/normal/smooth controls, which generated meshes followed consistently.The annotations described symmetry properties and geometric detail levels; results were shown using multiview normal maps.Source: Figure 8 · p. 14, Figure 9 · p. 14, §5.2 · p. 15

5.3 Comparison Results with SOTA Methods

• Step1X-3D achieved the highest CLIP-Score and multiple second-highest rankings on geometric-semantic matching metrics in comparisons with open-source and proprietary SOTA methods.The geometric metrics measure image-to-geometry alignment, while CLIP-Score measures texture semantic alignment. The quantitative evaluation is reported in Table 1.Source: Table 1 · p. 15, §5.3 · pp. 15–16
• In a 20-participant study of 110 images, Step1X-3D performed comparably to leading methods; pose-normalized visual comparisons found comparable or superior results.Participants scored geometric plausibility, input similarity, texture clarity, and texture-geometry alignment on a 5-point scale. Visual comparisons aligned models in Unreal Engine and composited multiple objects for direct comparison; all evaluated methods remained below the theoretical upper bound.Source: Figure 10 · p. 16, §5.3 · p. 16

• Step1X-3D introduces an open-source, high-fidelity framework that decouples geometry and texture synthesis.Source: §6 · p. 16
• Its 2M-asset data curation and hybrid VAE-DiT architecture achieve promising results while enabling 2D-to-3D control transfer.Source: §6 · p. 16
• The planned release includes models, training code, and training data excluding self-collected assets, to foster community progress toward production-ready 3D generation.Source: §6 · p. 16

7 Limitations

• The mesh-to-TSDF conversion currently uses grid resolution 2563, with higher resolution planned to improve geometric detail.The current texture component is limited to albedo generation; future work plans to add input-image relighting and physically based rendering (PBR) material textures.Source: §7 · p. 16