PaperlayerView exact revision on arXiv

Source-linked AI summary

DiDE:Direct Injection with Color-Texture DEcoupling for 3D Stylization

Tao Wu, Alexandra Gomez-Villa, Senmao Li, Yaxing Wang, Joost van de Weijer, Kai Wang

arXiv:2610.02044v1cs.CV

TL;DR

Existing 3D stylization transfers color and texture together, despite creators' need to control them independently. DiDE partitions latent channels to inject separate color and texture references while preserving content geometry, and is reported to outperform 2D and 3D baselines across the benchmark's three evaluation dimensions.

  • Problem

    Existing 3D stylization enforces all-or-nothing transfer of reference color and texture, leaving fine-grained, decoupled 3D stylization open.

    Source: §1 · p. 2

  • Method

    DiDE processes content, texture, and color images in dedicated branches and partitions style-significant channels to compose texture and color signals independently while preserving content geometry.

    Source: §1 · pp. 2–3

  • Results

    DiDE consistently outperforms 2D and 3D stylization baselines in color fidelity, texture transfer, and content preservation on Disen3D-Bench.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Disen3D enables artists to selectively transfer color or texture while preserving content geometry.

    Source: §1 · p. 2

Abstract

from arXiv · show

Recent advances in rectified flow-based image-to-3D generative models have enabled high-fidelity 3D asset generation. Building on this, a growing line of work has exploited these strong 3D priors for training-free stylization, transferring visual attributes from a reference image onto a generated 3D asset. However, existing methods enforce an all-or-nothing paradigm: color and texture are transferred jointly, with no mechanism to control them independently -- a limitation we formalize as Disentangled 3D Stylization(Disen3D). To address this, we propose DiDE, the first training-free framework for Disen3D. Key to our approach is the observation that the structured latent space of image-to-3D models is overcomplete with respect to texture: texture information occupies only a small subset of the style-significant channels, leaving a free subspace available for independent color encoding. DiDE exploits this via a channel partition mechanism that processes a content image, a texture reference, and a color reference through dedicated branches and composes both style signals interference-free at every self-attention layer, preserving content geometry throughout. Experiments on Disen3D-Bench, our newly collected multi-reference benchmark, show that DiDE consistently outperforms 2D and 3D stylization baselines in color fidelity, texture transfer, and content preservation.

1 Introduction

Existing 3D stylization transfers color and texture jointly, while creators may want to borrow each from a different reference. DiDE formalizes independently controllable color and texture as Disen3D and uses channel partitioning to inject both while preserving content geometry.

1 Introduction

  • Because prior style embeddings from separate references cannot simply be composed in 3D latent space, the 2D disentangled-stylization approach does not directly solve this 3D problem.The authors identify SADis [32] as the only prior work addressing color-texture disentanglement, but it operates in 2D.Source: §1 · p. 2
  • DiDE processes a content image, texture reference, and color reference through dedicated branches, partitioning channels to compose both style signals without interference while preserving content geometry.Texture occupies only a subset of style-significant channels, leaving capacity for independent color encoding; the partition is applied at every self-attention layer of the texture generation stage.Source: §1 · pp. 2–3
  • Disen3D addresses the all-or-nothing transfer of color and texture by making the two attributes independently controllable.Creators may want to retain their own palette while borrowing texture, or combine a texture and color palette from different references.Source: §1 · p. 2
  • DiDE consistently outperforms 2D and 3D stylization baselines in color fidelity, texture transfer, and content preservation on Disen3D-Bench.The benchmark is a newly collected multi-reference dataset for disentangled 3D stylization.Source: §1 · p. 3

2 Related Work

Related work spans 3D generative models, controllable 3D generation, and image-based 3D stylization. The cited work also motivates exploiting TRELLIS channels and reconsidering conventional attention inputs for style transfer.

3D Generative Models

  • TRELLIS [49] uses a structured latent representation for multi-format 3D generation, and this work inherits its versatility for diverse 3D modalities.The related-work discussion contrasts native 3D generators with approaches that distill 2D priors into 3D reconstruction or optimization.Source: §2 · p. 3

Controllable 3D Generation

  • Controllable 3D generation includes shape-prior and reference-style approaches, while fixed-geometry style pipelines restrict control to texture and leave geometric style unchanged.This work uses TRELLIS latent features to support control over texture and geometric style.Source: §2 · p. 3

Image-based 3D Stylization

  • Image-based 3D stylization transfers patterns from a style reference onto target geometry, with work spanning explicit meshes, neural implicit fields, Gaussian splatting, and diffusion-based priors.The cited methods share an all-or-nothing style-transfer limitation that restricts practical utility.Source: §2 · pp. 3–4
  • Texture similarity rises sharply through 25% of retained channels and then plateaus, indicating that texture information is concentrated in a small channel subset.Visual inspection also shows texture reproduction beyond this threshold with negligible improvement from additional channels.Source: Figure 2 · p. 4
  • Using query, key, and value from the reference image improves style consistency compared with the conventional content-query and reference-key/value setup, which leads to content bias.Source: Figure 3 · p. 4

3 Methods

DiDE builds on TRELLIS to preserve content geometry while independently transferring texture and color, with cross-view consistency as a goal.

3.1 Preliminaries

  • TRELLIS first predicts sparse geometry and then refines voxel features for texture and material; DiDE leaves the first-stage structure prediction unchanged and transfers style in Stage II.The active voxel positions from Stage I and image embedding are supplied to Stage II.Source: §3.1 · p. 4

3.2 Overcompleteness of the TRELLIS Feature Space

  • Texture similarity is ≈0.6125 with only a small fraction of channels, rises to ≈0.625–0.63 above 25%, and stays within that range through 100%.PCA reconstruction analysis also supports low intrinsic dimensionality of texture and a free subspace for another style attribute.Source: §3.2 · p. 5

3 Methods

  • DiDE processes content, texture, and color inputs in dedicated branches, keeping the content branch unchanged to preserve geometry while extracting separate style embeddings.Its style branches use independent self-attention rather than content-derived queries, avoiding the structural leakage described for prior cross-attention methods.Source: §3.3 · p. 5, §3.4 · pp. 5–6

3.3 Color-Texture Attributes Embedding

  • Texture references are converted to grayscale, while color references use low-frequency image components or palette-based Voronoi images before DINOv2 encoding.The low-frequency filtering suppresses high-frequency texture details; palette images encode color distribution without texture structure.Source: §3.3 · p. 5

3.4 Disentangled 3D Style Transfer (Disen3D)

  • At every Stage II block, DiDE partitions high-variance texture and complementary color channels, fuses separate style attention outputs, and gates the fusion into the content branch.The gated update preserves the content branch’s spatial structure and diffusion conditioning.Source: §3.4 · pp. 5–6, Figure 4 · p. 6

4 Experiments

Experiments on Disen3D-Bench compare DiDE with adapted 2D and 3D baselines and assess its disentanglement, content preservation, and style fidelity. DiDE leads on several reported quantitative measures, while ablations examine preprocessing, model adaptation, channel selection, and branch weighting.

4.1 Experimental Setup

  • Disen3D-Bench combines 10 content, 12 texture, and 15 palette images into 1,800 assets, compared against adapted SADis, StyleSculptor, and MorphAny3D.Evaluation covers content preservation, color fidelity, and texture consistency; StyleSculptor and MorphAny3D were not designed for disentangled two-reference stylization, while SADis operates in 2D.Source: §4.1 · p. 7

4.2 Experimental Results

  • Qualitatively, DiDE aligns texture structure with the target color while keeping generated assets consistent with the content, without distortions, flips, or other visual artifacts.The comparison includes single-reference results in the first two rows and disentangled texture-and-color references in the remaining rows.Source: §4.2 · pp. 7–8, Figure 5 · p. 8
  • DiDE leads CLIP-I (0.75), Chamfer distance (15.38), Hausdorff distance (7.87), C-Hist (0.87), VLM color accuracy (70.94 vs. 40.05), SSIM (125.19), and VLM texture accuracy (30.72 vs. 23.22).Compared with StyleSculptor, DiDE reduces Chamfer and Hausdorff distances by approximately 14.1% and 6.1%, respectively; LPIPS scores are comparable across methods (0.65–0.69).Source: §4.2 · pp. 8–9, Table 1 · p. 8

4.3 Ablation Study

  • Removing grayscale conversion leaves residual color in the texture branch, while removing Voronoi tessellation causes generated colors to deviate from the reference palette.Both preprocessing components are ablated separately while the rest of the pipeline is held unchanged.Source: §4.3 · p. 9, Figure 6 · p. 9
  • DiDE maintains robust, consistent disentanglement when applied to TRELLIS.2, whose three-stage pipeline processes texture and color features separately.The authors report that this stage-wise separation makes adaptation straightforward.Source: §4.3 · p. 9, Figure 7 · p. 9
  • Using texture-branch or color-branch variance for channel selection yields consistent disentanglement, with only minor saturation differences.Texture-branch variance is the default because it produces marginally more faithful color rendition in practice.Source: §4.3 · p. 9, Figure 8 · p. 9
  • Sweeping λ from 0.5 to 2.0 shows that λ = 1.2 balances color transfer and texture quality, with the lowest C-Hist (0.89) and MS-SWD (19.25).The reported results average 600 randomly sampled cases; too little λ weakens color transfer, while too much begins to compromise texture quality.Source: §4.3 · p. 9, Table 2 · p. 9
  • DiDE receives the highest user preference rates for geometry preservation (51.7%), texture consistency (58.2%), and color consistency (72.4%).The authors describe the study as complementary perceptual evidence for the reported improvements.Source: §4.3 · p. 10, Table 3 · p. 10

5 Conclusion

The conclusion frames DiDE as a training-free approach to independently control color and texture in 3D stylization. Evaluations on Disen3D-Bench report improved disentangled control relative to baselines.

5 Conclusion

  • Existing 3D stylization methods lack independent control over distinct attributes such as color and texture, which the paper formalizes as Disen3D.Source: §5 · p. 10
  • DiDE processes color and texture references through dedicated branches and combines them by channel partition in self-attention layers while preserving content geometry.Source: §5 · p. 10
  • Qualitative and quantitative evaluations on Disen3D-Bench report that DiDE outperforms baselines in disentangled color and texture control.The authors hope the work will inspire further research on multi-attribute style disentanglement.Source: §5 · p. 10

A Statements

The statements describe DiDE's potential practical uses and misuse risks, alongside constraints on its validated models, geometry refinement, and supported style attributes. They also address responsible use and reproducibility.

Limitations

  • DiDE is validated only on TRELLIS and TRELLIS.2 rectified-flow models, and its geometry depends on the underlying model because DiDE has no dedicated geometric refinement module.The current design supports independent color and texture control, not disentanglement of attributes such as material, lighting, or geometric style.Source: A Statements · p. 15

Broader Impacts

  • The statements cite potential benefits for customizable 3D creation across several industries and risks of unauthorized copying, counterfeiting, or commercial exploitation.They advocate content authentication, usage norms, legal compliance, and responsible deployment.Source: A Statements · p. 15

Ethical Statement

  • The authors state that their models use publicly available datasets and encourage responsible use while noting privacy, impersonation, and synthetic-asset misuse risks.Source: A Statements · p. 15

Reproducibility Statement

  • The authors plan to release source code and training scripts after peer review, and report that experiments use public datasets with implementation and evaluation details in the paper and appendices.Source: A Statements · p. 15

B VLM-based Evaluation Protocol

The protocol uses InternVL2.5-8B for binary evaluation of style consistency and color fidelity. It compares generated images with texture and color references, tokenizing and concatenating image patches while preserving image boundaries.

B VLM-based Evaluation Protocol

C Hyperparameter visualization

The ablation finds λ=1.2 provides the best balance between texture preservation and color controllability. Smaller and larger settings respectively under-guide color or overemphasize it.

C Hyperparameter visualization

D Ablation on k

The channel ablation shows that recognizable texture survives when only a small subset of TRELLIS feature channels is retained, supporting the feature space's overcompleteness with respect to texture.

D Ablation on k

  • The ablation retains only the top-k channels and zeros the remainder, testing k values from 0 to 100 in steps of 25.Source: D Ablation on k · p. 17
  • At k=25, generated 3D assets already exhibit clearly recognizable texture patterns.Source: D Ablation on k · p. 17
  • This indicates that a relatively small channel subset preserves most texture information, further demonstrating TRELLIS feature-space overcompleteness.Source: D Ablation on k · p. 17

E Additional Visualization Results

Additional visualizations report effective color-texture disentanglement across diverse, challenging references. The generated assets retain texture-reference structure and geometry while matching the target color distribution and maintaining content consistency.

E Additional Visualization Results

  • Across diverse, challenging texture and color references, DiDE consistently achieves effective disentanglement between texture structure and color attributes.Source: E Additional Visualization Results · p. 17
  • Generated assets preserve structural patterns and geometric details from the texture reference while matching the color distribution of the color reference.Source: E Additional Visualization Results · p. 17
  • The visualizations report strong content consistency without noticeable distortions, structural artifacts, or undesired color leakage, supporting DiDE's generalization and stability.Source: E Additional Visualization Results · p. 17

F PCA Reconstruction Analysis

PCA analysis of TRELLIS style-significant channels finds that texture similarity stabilizes with as few as 10 principal components, supporting the claim that texture occupies a low-dimensional subspace.

F PCA Reconstruction Analysis

Related papers

Loading 2610.02044v1…