DirectUV: Image-Conditioned UV Texture Generation with Surface-Aware Positional Encoding

• Mesh texturing must balance realistic, detailed appearance with consistency across the surface, including freedom from seams, view-dependent artifacts, and misaligned details.Source: §1 · p. 1
• Projection-based methods can produce realistic local appearance but suffer from multi-view inconsistency and projection errors, while direct UV methods retain attention tied to UV positions rather than surface geometry.Source: §1 · p. 1
• DirectUV generates textures in latent UV space from a single image and coarse UV map, using both only as conditioning signals rather than intermediate textures to refine.Source: §1 · p. 2
• SAPE embeds per-token 3D surface coordinates in positional encodings, and Multi-Level SAPE assigns progressively finer positional granularities to different attention heads.Source: §1 · p. 2
• DirectUV produces sharper, more seam-consistent textures than projection-based and direct-generation baselines, with the largest gains in occluded and view-unseen regions.These regions are poorly covered by camera-based projection.Source: §1 · p. 2

2.1 Multi-View Projection-Based Texturing

• Multi-view pipelines synthesize and project views onto the mesh, but finite viewpoints leave occluded or grazing-angle regions uncovered and projection must reconcile mismatched signals.Source: §2.1 · pp. 2–3

2.2 Direct Texture Generation in UV Space

• Direct UV methods avoid projection artifacts, but often inject 3D structure through auxiliary mechanisms rather than making UV-space diffusion itself surface-aware.Source: §2.2 · p. 3

2.3 Positional Encoding in Diffusion Transformers

• View-space 3D-aware positional encoding still defers seam coherence to projection and baking, whereas DirectUV uses surface-aware attention during UV-texture denoising.Source: §2.3 · p. 3
• Figure 1 depicts the reference image and coarse UV map conditioning the model, with the decoded UV texture wrapped onto the mesh at inference.Source: Figure 1 · p. 4

3.1 Overview

• DirectUV uses a diffusion transformer in latent UV space, guided by an input image and coarse UV map, with SAPE injecting surface geometry into attention.Source: §3.1 · p. 4

3.2 Latent UV Diffusion Formulation

• The UV map is the sole generation target: a frozen Flux VAE encodes it to a spatially downsampled latent for denoising and decodes the result as the texture.The model uses a single diffusion process in this latent representation.Source: §3.2 · pp. 4–5

3.3 Conditioning in UV-Space Generation

• The reference image supplies appearance information and the coarse UV map supplies surface-aligned layout; both are conditioning signals, not intermediate textures to refine.Source: §3.3 · p. 5

3.4 Surface-Aware Positional Encoding (SAPE)

• SAPE replaces UV-grid positional codes with per-token coordinates from the CCM, so attention relates tokens by surface offsets rather than UV-plane distance.This lets surface-neighboring texels interact even when separated across UV seams.Source: §3.4 · pp. 5–6
• Multi-Level SAPE reads the CCM at progressively finer subdivisions of the same latent patch and assigns a single granularity to each attention head.Heads integrate the different granularities downstream without additional latent maps, parameters, or auxiliary geometry branches.Source: §3.4 · p. 6

3.5 Training and Inference

• Training uses rectified flow matching, while inference starts from Gaussian noise and applies Euler denoising steps before decoding the final UV texture.The conditioning bundle and CCM are reused across inference steps.Source: §3.5 · pp. 6–7

4.1 Dataset

• The training set retains approximately 100K textured meshes; each has a 1024 × 1024 UV mask, ground-truth texture, CCM, and four canonical rendered views.The front view serves as the reference image for DINOv2 and CLIP.Source: §4.1 · p. 7

4.2 Implementation Details

• The Flux-style model uses 24 attention heads and three Multi-Level SAPE subdivision levels, with image conditioning from DINOv2 and CLIP and a frozen FLUX.1-dev VAE.Training uses randomly selected 0, 1, 2, or 4 projected views to construct the coarse UV map.Source: §4.2 · p. 7

4.3 Evaluation

• Evaluation uses the disjoint GSO dataset as an out-of-distribution test set, with one front-facing reference image per object and full-directional-coverage render comparisons.PSNR and SSIM measure per-pixel fidelity; FID and KID measure distributional similarity.Source: §4.3 · p. 7

4.4 Comparison with State-of-the-Art Methods

• Qualitatively, DirectUV preserves finer details, produces coherent content in occluded regions, and handles multi-view inconsistencies better than the compared methods.Examples include the number block, well and trash-bin interiors, and the fan.Source: §4.4 · p. 8
• DirectUV achieves the best PSNR and SSIM, with competitive FID and KID and the best overall balance of reconstruction fidelity and perceptual quality.The comparison is on the GSO dataset.Source: Table 1 · p. 8, §4.4 · p. 8

4.5 Ablation Study

• Removing SAPE causes drops across metrics and seam mismatches because attention relates tokens by UV-plane proximity, not 3D surface proximity.Mesh-adjacent texels split across UV charts cannot interact directly in this variant.Source: §4.5 · p. 9
• Removing the multi-level design preserves seam coherence but blurs textures because heads cannot distinguish sub-token positions within a latent patch.Multi-level coordinates expose finer sub-patch granularity to different heads, restoring high-frequency detail.Source: §4.5 · p. 9

5 Conclusion

• DirectUV generates textures in latent UV space, using projected multi-view results only as conditioning signals.SAPE embeds surface geometry in positional encodings, and a multi-level design supports reasoning at multiple spatial scales.Source: §5 · p. 9
• DirectUV outperforms other baselines, especially in occluded and view-unseen regions.The conclusion identifies these as the areas of strongest improvement.Source: §5 · p. 9
• DirectUV remains reliant on UV parameterization quality, which the authors identify as a limitation for future work.Source: §5 · p. 9

A Coarse UV as Soft Conditioning

• The coarse UV map supplies surface-aligned layout cues but may contain inaccurate or misleading colors; generation can deviate from those values.The map guides rather than constrains the output.Source: A Coarse UV as Soft Conditioning · p. 12
• The keyboard’s white coarse-UV underside becomes a dark metallic casing in the final texture, consistent with the reference image.The example illustrates the model overriding inaccurate coarse-UV values.Source: Figure 1 · p. 12, A Coarse UV as Soft Conditioning · p. 12

B Robustness to Coarse UV Completeness

• Training samples 0, 1, 2, or 4 views for the coarse UV, and the zero-view case yields plausible textures from the reference image and geometry-conditioned denoising.The authors report this as evidence of robustness to incomplete coarse-UV inputs at inference.Source: B Robustness to Coarse UV Completeness · p. 12

C Application to Generated Meshes

• DirectUV textures meshes from image-to-3D generators using the same reference image that conditioned shape generation.The examples span varying generated-mesh topologies and UV layouts.Source: C Application to Generated Meshes · p. 12

D Failure Cases

• DirectUV's texture quality is sensitive to the underlying UV parameterization, particularly layouts fragmented into small slivers and corner patches.The per-token surface coordinate aggregates information over relatively large surface regions for each fragment.Source: D Failure Cases · pp. 12–13
• Figure 3 illustrates DirectUV texturing meshes generated from images as part of a complete image-to-textured-mesh pipeline.Each block pairs the input reference image with generated textures rendered from multiple viewpoints.Source: Figure 3 · p. 13
• Highly fragmented UV layouts produce coarse CCM signals, limiting multi-level subdivision's recovery of fine local detail and yielding blur or color leakage.Figure 4 shows UV maps with many small islands and slivers above the corresponding wrapped textures.Source: D Failure Cases · p. 13, Figure 4 · p. 14
• The reported failure modes arise from UV parameterization and mesh quality, not from SAPE's design; proposed future directions include UV-aware preprocessing or learned tokenization.Figure 2 varies coarse UV completeness across 0, 1, 2, and 4 projected views while holding the reference image and mesh fixed.Source: D Failure Cases · p. 13, Figure 2 · p. 13

E More Results

• Additional results cover diverse object categories and mesh complexities.Figure 5 presents these additional texture-generation examples.Source: E More Results · p. 14, Figure 5 · p. 15