PaperlayerView exact revision on arXiv

Source-linked AI summary

BTC3D: Blended Tile Conditioning for Detail-Enhancing Image-to-3D Generation

Junyu Li, Qiuyu Chen, Pengcheng Wang, Shiqi Yang, Alexandra Gomez-Villa, Joost van de Weijer, Ruilin Li, Kai Wang

arXiv:2609.39709v1cs.CV

TL;DR

Image-to-3D pipelines can lose fine texture when global conditioning compresses spatial information. BTC3D adds patch-level conditioning and gradually increases its influence during later diffusion. The paper reports improved texture fidelity while maintaining global structure, without retraining.

  • Problem

    Global image conditioning compresses spatial detail, limiting fine-grained texture reproduction in image-to-3D generation.

    Source: §1 · p. 2

  • Method

    BTC3D combines foreground-weighted local patch embeddings with global conditioning and gradually increases tile guidance during later, low-noise diffusion stages.

    Source: §1 · p. 2, §5 · p. 9

  • Results

    BTC3D improves results across reported metrics over TRELLIS-family and Hunyuan3D-2.1 baselines, with larger gains on ULIP-2 and Uni3D native 3D metrics.

    Source: §4.2 · p. 7

  • Takeaways & Limitations

    The framework serves as a plug-and-play, training-free enhancement that improves appearance-detail preservation while maintaining coherent global structure.

    Source: §4.2 · p. 8, §5 · p. 9

Abstract

from arXiv · show

Recent diffusion-based pipelines have achieved promising progress in image-to-3D synthesis. However, generating high-fidelity details remains challenging, especially when the input image contains rich details. Existing approaches often rely on globally encoded conditioning features, which compress spatial information and limit the model to reproduce fine-grained details. This common design often leads to a phenomenon we term detail attenuation. Moreover, improving image-to-3D synthesis quality typically requires retraining or fine-tuning large diffusion models, which can be computationally expensive and impractical for complex 3D pipelines. In this work, we present Blended Tile Conditioning for image-to-3D generation (BTC3D), a training-free inference time framework that enhances fine-grained detail preservation in image-to-3D diffusion pipelines. To alleviate detail attenuation, we first examine the image feature additivity in image-to-3D models. Based on this property, we introduce a blended tile embedding that extracts local conditioning signals from split image regional patches, allowing the diffusion model to better preserve fine-grained visual details. To integrate the global and local conditioning guidance stably, we propose a dynamic conditioning schedule that gradually increases the influence of tile-level conditioning during later low-noise stages of diffusion. Our proposed method BTC3D operates entirely at inference time and can be seamlessly integrated into existing image-to-3D diffusion pipelines. Experimental results demonstrate that the proposed approach significantly improves texture quality and visual fidelity of the base model while maintaining global structural consistency in a training-free manner.

1 INTRODUCTION

Image-to-3D pipelines can attenuate fine texture because global conditioning compresses spatial detail and the same conditioning is used throughout diffusion. BTC3D combines local patch features with global guidance at inference time, reporting improved benchmark performance without retraining.

1 INTRODUCTION

  • Global image conditioning compresses spatial detail, often producing over-smoothed textures, while early and late diffusion stages have different structural and detail-generation roles.Standard pipelines use the same conditioning representation throughout sampling.Source: §1 · p. 2
  • Image feature additivity lets local patch representations produce semantically aligned 3D outputs that preserve details suppressed by global conditioning.The paper identifies this property as the underlying mechanism of its training-free framework.Source: §1 · p. 2
  • BTC3D aggregates patch-derived local features using foreground-based weighting, while DyCond increases their influence during later diffusion to preserve details and global structure.The schedule emphasizes tile conditioning during the stage when fine details form.Source: §1 · p. 2
  • The introduction reports quantitative and qualitative improvements on 3D-Arena and Toys4K when applying the training-free framework to existing image-to-3D models.The passage names both benchmarks but does not give numerical results.Source: §1 · p. 2
  • BTC3D achieves state-of-the-art image-to-3D performance across multiple benchmarks and integrates into existing diffusion pipelines without retraining.The introduction reports extensive experiments but provides no metric values here.Source: §1 · p. 2

2 RELATED WORKS

Related work distinguishes image-to-3D approaches that distill 2D priors from native 3D generative models. The paper builds on native 3D foundation-model priors while targeting example-driven refinement without training.

3D Generation via 2D priors

  • Distillation methods transfer 2D priors to 3D, but the cited discussion says they lack an explicit 3D latent space and struggle with multi-view consistency and precise texture-geometry alignment.The paper describes these limitations as rooted in the distillation pipeline.Source: §2 · p. 3

Native 3D Generation

  • Native 3D generators learn compact latent representations and offer strong geometric priors, but most are restricted to explicit or implicit representations.TRELLIS introduces SLAT for flexible multiformat generation, which BTC3D inherits to accommodate diverse 3D modalities.Source: §2 · p. 3
  • The related-work discussion frames BTC3D as training-free refinement that leverages native 3D foundation-model priors for user-specified, example-driven control.The passage contrasts this goal with existing models that mainly sample from learned data distributions.Source: §2 · p. 3

3 METHODOLOGY

BTC3D is a training-free, inference-time framework built on observed compositionality and compatibility of global and local image features. It combines foreground-weighted tile embeddings with a schedule that shifts tile conditioning later in generation.

3.1 EMPIRICAL OBSERVATIONS ON DINO-ENCODED FEATURES

  • Global and averaged-tile embeddings cluster, and tile embeddings can improve fine detail while sometimes disturbing global structure, motivating feature additivity as BTC3D's basis.The observations are presented for DINO-encoded features and are described as generalizable to other 3D generation models.Source: §3.1 · p. 4, Figure 2 · p. 4

3.2 BTC3D: BLENDED TILE CONDITIONING FOR IMAGE-TO-3D GENERATION

  • BTC3D combines a blended tile embedding with dynamic conditioning to integrate local and global features during texture generation.The framework operates at inference time and targets detail preservation in image-to-3D models.Source: §3 · p. 3, §3.2 · p. 4, Figure 3 · p. 5

3.2.1 BLENDED TILE CONDITIONING EMBEDDING (BTCemb)

  • BTCemb aggregates local features from overlapping image tiles using foreground coverage to emphasize informative tiles, while retaining individual tile features and their priority scores.Foreground-dominated tiles contribute more than background-dominated tiles, which may provide limited evidence about the target object.Source: §3.2 · p. 4, §3.2.1 · p. 5

3.2.2 DYNAMIC CONDITIONING (DyCond)

  • DyCond introduces tile cues later (higher-priority tiles first), keeping global conditioning dominant early; compared with static fusion, it further refines texture while preserving structure.Lower-weight tiles are gradually introduced as generation proceeds.Source: §3.2.2 · p. 6, Figure 4 · p. 6

4 EXPERIMENTS

BTC3D is evaluated on two benchmarks, across several image-to-3D backbones, and improves reported metrics and qualitative detail preservation. Ablations examine its components and conditioning settings.

4.1 EVALUATION SETUPS

  • Evaluations use 3D-Arena and Toys4K, compare six representative baselines, and assess alignment, perceptual similarity, pixel fidelity, local structure, and native 3D alignment.BTC3D is applied to TRELLIS, TRELLIS.2, and Hunyuan3D-2.1; comparisons render assets at 1024 × 1024 under a shared protocol.Source: §4.1 · pp. 6–7

4.2 EXPERIMENTAL RESULTS

  • BTC3D improves all reported metrics over TRELLIS-family and Hunyuan3D-2.1 baselines on 3D-Arena and Toys4K, with larger gains on ULIP-2 and Uni3D.The reported improvements also include detail-related metrics PSNR, SSIM, and LPIPS.Source: Table 1 · p. 7, §4.2 · p. 7
  • On the illustrated backbones, BTC3D better preserves appearance details, object identity, and input-specific cues while retaining coherent global structure.The comparison describes smoother textures and less distinct local patterns in some baseline outputs; TripoSG and Hi3DGen focus primarily on geometry.Source: §4.2 · pp. 7–8, Figure 5 · p. 8

4.3 ABLATION STUDIES

  • Adding BTCemb to TRELLIS.2 improves ULIP-2 and Uni3D, and adding DyCond yields further gains; together, the modules achieve the best overall results.The qualitative ablation compares the baseline, BTCemb, and BTCemb with DyCond; the text attributes local cues to foreground-based tile weighting and additional gains to progressive low-noise conditioning.Source: Figure 6 · p. 9, Table 2 · p. 9, §4.3 · p. 9
  • N = 3, the setting for dividing the input image into tiles, performs best on both ULIP-2 and Uni3D.Source: Table 3 · p. 9, §4.3 · p. 9
  • β = 2 gives the best results, while every tested dynamic-conditioning β configuration outperforms static conditioning.β controls the slope of the progressive tile-activation schedule.Source: Table 4 · p. 9, §4.3 · p. 9
  • α = 0.4 performs best on ULIP-2 and Uni3D; smaller values weaken tile conditioning, while larger values slightly harm structural consistency.α balances global coherence against detail enhancement.Source: Table 5 · p. 9, §4.3 · p. 9

5 CONCLUSION

BTC3D combines local tile embeddings with a dynamic conditioning schedule to refine image-to-3D results at inference time. The paper also states its validation scope, potential impacts, and reproducibility plans.

5 CONCLUSION

  • BTC3D uses image-feature additivity to combine patch-derived local conditioning with global guidance, increasing tile-level influence during later low-noise diffusion to enhance texture while preserving global structure.The framework operates at inference time and is intended to integrate into diverse image-to-3D diffusion pipelines.Source: §5 · p. 9

A STATEMENTS

  • Validation focuses mainly on flow-matching native 3D diffusion pipelines, and applicability to other paradigms remains unevaluated; the method targets texture and uses DINO-series encoders.Adapting to other visual encoders may require re-tuning.Source: A STATEMENTS · p. 14
  • The paper identifies lower production costs and broader asset-creation uses as potential benefits, while noting risks of unauthorized duplication and counterfeiting.It calls for responsible deployment, content authentication, and clear usage norms.Source: A STATEMENTS · p. 14
  • The authors describe privacy, impersonation, and misuse risks, state that their models use publicly available datasets under their policies, and plan to release implementation details and reproduction scripts.They also report using generative AI for literature searches and manuscript assistance, not for method development, experiments, or result interpretation.Source: AI USE STATEMENT · p. 10, A STATEMENTS · p. 14

B IMPLEMENTATION AND EVALUATION DETAILS

  • Evaluation spans TRELLIS, TRELLIS.2, and Hunyuan3D-2.1, covering different DINO encoders and 3D latent representations.The paper identifies TRELLIS and Hunyuan3D-2.1 as using DINOv2 and TRELLIS.2 as using DINOv3.Source: B IMPLEMENTATION AND EVALUATION DETAILS · p. 14

B.1 EVALUATION DATASETS

This appendix describes the evaluation sets and complementary metrics, then examines BTC3D’s high-frequency response, feature compatibility, and runtime overhead.

B.1 EVALUATION DATASETS

  • 3D-Arena is evaluated on its complete set, while Toys4K uses 210 objects sampled two per category from 105 categories.The Toys4K sample uses a fixed seed of 2026; quantitative comparisons use the common subset with results available for all methods.Source: §B.1 · pp. 14–15

B.2 EVALUATION METRICS

  • Evaluation combines rendered-view metrics for texture and appearance with 3D-aware image-to-asset alignment scores from ULIP-2 and Uni3D.Rendered views use 12 viewpoints; the 3D-aware metrics operate on colored point clouds sampled from generated meshes.Source: §B.2 · p. 15

C.1 HIGH-FREQUENCY PERCEPTION

  • BTCemb responds more strongly than Global toward 256–448 cycles/image, while Global responds more strongly at lower sampled frequencies.The comparison uses normalized angular distance between conditioning tensors for sinusoidal gratings and matched flat references; this is encoder-level evidence, not a direct measure of generated texture fidelity.Source: Figure 7 · p. 18, §C.1 · p. 18

C.2 IMAGE FEATURE ADDITIVITY AND SAME-CATEGORY COMPATIBILITY

  • BTCemb increases global-local feature cosine distance and same-category cosine margin over Average tiles, while blended embeddings combine color characteristics from two inputs.The positive same-category margins indicate compatibility beyond coarse category separation; the color-blending example uses a frozen TRELLIS.2 generator.Source: §C.2 · p. 19, Figure 8 · p. 19, Table 6 · p. 19

C.3 RUNTIME OVERHEAD

  • BTC3D’s reported overhead mainly comes from feature extraction and is described as negligible, especially for computationally intensive pipelines.Runtime comparisons use each pipeline’s default sampling schedule and report means and standard deviations over three runs.Source: §C.3 · pp. 19–20, Table 7 · p. 20

C.4 ADDITIONAL QUANTITATIVE RESULTS

Supplementary evaluations report benefits across backbones and texture-sensitive diagnostics, alongside cautions about interpreting no-reference detail metrics and a region-level evaluation protocol.

C.4 ADDITIONAL QUANTITATIVE RESULTS

  • On 3D-Arena, BTC3D improves CLIP-FID and DINO for TRELLIS, TRELLIS.2, and Hunyuan3D-2.1, with additional LPIPS (Best view) and Laplacian Variance gains for TRELLIS-family backbones.The passage presents these as supplementary diagnostics across different backbones.Source: §C.4 · p. 20
  • On Toys4K, BTC3D improves texture-sensitive diagnostics across backbones, including DINO, LPIPS (Best view), Edge Density, and Laplacian Variance.For Hunyuan3D-2.1, the reported improvements also include CLIP-FID; for TRELLIS and TRELLIS.2, they include multiple supplementary metrics.Source: §C.4 · p. 20
  • High Laplacian Variance can reflect sharp silhouettes, hard boundaries, or artifact-like edges rather than faithful texture recovery, so detail metrics are auxiliary diagnostics.The passage specifically notes this interpretation for 3DTopia-XL on Toys4K.Source: §C.4 · p. 20
  • The region-level Toys4K evaluation uses a fixed 4 × 4 grid from the ground-truth alpha mask and measures fidelity across 4,903 valid regions from 210 objects.It reports region PSNR and SSIM alongside paired win rates over 12 matched views per object.Source: §C.4 · p. 20

C.5 ADDITIONAL QUALITATIVE RESULTS

Additional visual comparisons show BTC3D’s texture and detail results across image-to-3D models; the user-study passages describe how TRELLIS.2 outputs were compared interactively.

C.5 ADDITIONAL QUALITATIVE RESULTS

  • Direct visual comparisons of TRELLIS.2 and methods integrated with BTC3D show marked enhancement in final-texture detail, preserving local condition-image features while mitigating multi-view inconsistency and over-smoothing artifacts in standard diffusion-based 3D generation.Source: §C.5 · p. 21
  • BTC3D examples include intricate treasure-chest carvings, metallic specularity, and subtle pig-surface variations, with normal maps and multi-view renderings accompanying the results.These visualizations are presented as evidence of complex material properties and geometrically consistent details.Source: §C.5 · pp. 21–22, Figure 10 · p. 23
  • Additional comparisons report that BTC3D consistently improves ULIP scores over baseline models and produces assets with accurate geometry and high-fidelity textures without additional training.Figure 9 shows results from integrating BTC3D into different baseline models.Source: §C.5 · pp. 20–21, Figure 9 · p. 22

C.6 USER STUDY

  • The user study compares paired TRELLIS.2 assets in interactive viewers, with 20 participants each evaluating 20 randomly selected pairs from 101 samples.Methods are hidden, display order is randomized, and participants judge which asset better matches the reference in shape, appearance, and visual details.Source: §C.6 · pp. 22–23

C.6.1 RESULTS

Participants preferred BTC3D-enhanced TRELLIS.2 outputs in the user study; the accompanying tiling ablation supports N = 3 as the default setting.

C.6.1 RESULTS

  • BTC3D wins on a majority of samples, with particularly strong preferences for texture-rich objects and approximately balanced preferences on a small number of texture-sparse samples.The reported pattern is consistent with benefits in detail-rich scenarios without quality degradation on texture-uniform inputs.Source: §C.6.1 · p. 23
  • 67.25% of 400 pairwise comparisons (269 votes) preferred BTC3D-enhanced outputs, versus 32.75% (131 votes) for TRELLIS.2.Figure 13 also reports the per-item preference distribution.Source: §C.6.1 · p. 23, Figure 11 · p. 24

D.1 ABLATION OF THE TILING PARAMETER N

  • Among N = 2–6, N = 3 has the highest mean PSNR and SSIM and lowest mean LPIPS; scores are similar for N = 2–4, while N = 5–6 perform worse.The ablation uses TRELLIS.2 on the full 101-image 3D-Arena dataset at resolution 1024.Source: §D.1 · p. 24

D.2 ABLATION OF THE TILE EXPANSION DIRECTION

The ablation compares tile-coverage schedules and finds the local-to-global schedule strongest on ULIP-2 and Uni3D. In this schedule, high-priority local tiles are emphasized first and tile coverage broadens as sampling progresses.

D.2 ABLATION OF THE TILE EXPANSION DIRECTION

  • The local-to-global schedule initially emphasizes high-priority tiles and gradually broadens effective tile coverage, unlike the global-to-local schedule.The global-to-local schedule starts with broad coverage and progressively concentrates on fewer local tiles.Source: §D.2 · pp. 24–25
  • The global condition remains active in both dynamic variants, so schedule direction refers to local tile coverage.Source: §D.2 · p. 25
  • The local-to-global schedule achieves the highest ULIP-2 and Uni3D scores among the evaluated settings.These results support progressively incorporating broader regional evidence after initially emphasizing high-priority local information.Source: §D.2 · p. 25

D.3 ABLATION OF THE DYNAMIC CONDITIONING PARAMETER β.

The ablation varies β in the dynamic conditioning schedule and finds β = 2 best among the evaluated settings. Dynamic settings outperform static conditioning on the illustrated example, with β = 2 scoring highest.

D.3 ABLATION OF THE DYNAMIC CONDITIONING PARAMETER β.

  • β controls the onset and slope of progressive activation and the sharpness of rank-based tile weighting.Source: §D.3 · p. 25
  • β = 2 achieves the best scores among the evaluated settings.The reported metrics are PSNR, ULIP-2, and Uni3D.Source: §D.3 · pp. 25–26
  • In this example, all three dynamic settings outperform static conditioning in both PSNR and ULIP-2, with β = 2 achieving the highest scores, supporting progressive tile activation over fixed tile conditioning.Figure 14 qualitatively compares the baseline, static conditioning, and different dynamic settings; highlighted engine and wing regions illustrate local detail preservation.Source: §D.3 · p. 26

D.4 ABLATION OF THE FUSION RATIO α.

The fusion-ratio ablation finds α = 0.4 best among the evaluated settings. The results emphasize balancing global guidance and local tile information to preserve details while maintaining semantic alignment.

D.4 ABLATION OF THE FUSION RATIO α.

  • α = 0.4 achieves the best results among evaluated settings on 3D-Arena.The reported metrics are PSNR, SSIM, LPIPS, ULIP-2, and Uni3D.Source: §D.4 · p. 26
  • The fusion ratio α controls the relative contributions of global guidance and local tile information, with β fixed at 2.Source: §D.4 · p. 26
  • An appropriate fusion ratio improves detail preservation while maintaining semantic alignment, whereas overly small or large values may reduce PSNR and sometimes ULIP-2.The qualitative comparisons are shown in Figure 15.Source: §D.4 · p. 26

E.1 DISCUSSION

The discussion describes complementary but incomplete ways to evaluate image-to-3D assets, and outlines future evaluation and representation directions. The algorithm section presents BTC3D as foreground-weighted tile embedding and timestep-dependent conditioning integrated into pretrained-generator sampling.

E.1 DISCUSSION

  • Image-to-3D assets can have multiple plausible geometries, scales, orientations, and unseen-region appearances, so reconstruction metrics require attention to alignment conditions.The same input may correspond to different plausible assets, unlike posed reconstruction with strict multi-view constraints.Source: §E.1 · p. 26
  • Rendered-view metrics depend on viewpoint, lighting, resolution, background, and single-view versus multi-view evaluation; global image-to-point-cloud similarity may miss fine texture fidelity and surface detail.The discussion combines rendered-view metrics with 3D-aware similarity measures and human preference evidence because these capture complementary aspects.Source: §E.1 · pp. 27–28

E.2 FUTURE WORK

  • Future work proposes exploring DINO features for local correspondence, semantic part awareness, texture placement, and geometry–texture alignment.The authors also suggest testing global and tiled features in texture generation, asset editing, image-conditioned refinement, and multi-view content generation.Source: §E.2 · pp. 28–29
  • Future evaluation work could develop detail-sensitive benchmarks, standardized rendering settings, and native 3D metrics that better reflect human visual perception.Source: §E.2 · p. 29

F ALGORITHM

  • BTCemb extracts global and local features, assigns tile priorities from foreground coverage, normalizes weights, and sorts tile features for dynamic conditioning.The algorithm overview describes this foreground-weighted embedding construction.Source: F ALGORITHM · p. 29
  • DyCond progressively changes relative tile emphasis and blends the dynamic tile embedding with the global feature using α; the features are reused through pretrained-generator sampling without changing model parameters.The global and tile features are computed once, while each sampling step receives a conditioning feature from DyCond.Source: F ALGORITHM · p. 29

Related papers

Loading 2609.39709v1…