PaperlayerView exact revision on arXiv

Source-linked AI summary

Interp3D: Correspondence-aware Interpolation for Generative Textured 3D Morphing

Xiaolu Liu, Yicong Li, Qiyuan He, Jiayin Zhu, Wei Ji, Angela Yao, Jianke Zhu

arXiv:2601.14103v1cs.CV

TL;DR

Textured 3D morphing requires smooth transitions that preserve both structure and fine-grained appearance, while prior geometry-based and 2D-derived methods leave correspondence and texture gaps. Interp3D is a training-free framework using generative priors and progressive semantic, structural, and texture alignment; on Interp3DData, it outperforms existing baselines, though 3D interpolation remains less explored and difficult for semantically distant assets.

  • Problem

    Existing textured 3D morphing approaches inadequately jointly preserve geometric consistency, semantic correspondence, and texture alignment across transitions.

  • Method

    Interp3D progressively aligns condition embeddings, SLAT-guided structure generation, and fine-grained texture features using a training-free generative-prior framework.

  • Results

    Interp3DData evaluations show that Interp3D outperforms existing baselines, producing more coherent transitions with structural fidelity and detail-preserving textures.

  • Takeaways & Limitations

    Correspondence-aware progressive alignment supports textured 3D morphing with geometric fidelity, coherent transitions, and texture detail preservation within the evaluated setting.

  • Takeaways & Limitations

    3D interpolation remains underexplored in structural and semantic alignment, and reliable correspondences remain challenging for source–target pairs with little semantic relevance.

Abstract

from arXiv · show

Textured 3D morphing seeks to generate smooth and plausible transitions between two 3D assets, preserving both structural coherence and fine-grained appearance. This ability is crucial not only for advancing 3D generation research but also for practical applications in animation, editing, and digital content creation. Existing approaches either operate directly on geometry, limiting them to shape-only morphing while neglecting textures, or extend 2D interpolation strategies into 3D, which often causes semantic ambiguity, structural misalignment, and texture blurring. These challenges underscore the necessity to jointly preserve geometric consistency, texture alignment, and robustness throughout the transition process. To address this, we propose Interp3D, a novel training-free framework for textured 3D morphing. It harnesses generative priors and adopts a progressive alignment principle to ensure both geometric fidelity and texture coherence. Starting from semantically aligned interpolation in condition space, Interp3D enforces structural consistency via SLAT (Structured Latent)-guided structure interpolation, and finally transfers appearance details through fine-grained texture fusion. For comprehensive evaluations, we construct a dedicated dataset, Interp3DData, with graded difficulty levels and assess generation results from fidelity, transition smoothness, and plausibility. Both quantitative metrics and human studies demonstrate the significant advantages of our proposed approach over previous methods. Source code is available at https://github.com/xiaolul2/Interp3D.

1 INTRODUCTION

Textured 3D morphing must jointly preserve structural coherence, appearance details, and smooth transitions, but existing geometry-based and 2D-extended approaches leave important correspondence and texture challenges. Interp3D addresses these issues through training-free, progressively aligned interpolation and evaluates it on a graded benchmark.

  • Motivation: Textured 3D morphing seeks smooth transitions between assets while preserving structural evolution and fine-grained appearance.The capability supports applications including animation, editing, motion tasks, effects visualization, and filmmaking.
  • Limitations of Existing Methods: Geometry-based methods establish explicit correspondences but rely on strict alignment and consistent topology, restricting them to shape-only interpolation.
  • Limitations of Existing Methods: Extending 2D interpolation strategies to 3D can introduce semantic ambiguity, structural misalignment, and texture-related artifacts.
  • Interp3D: Interp3D is a training-free framework that progressively aligns semantic conditions, generated structure, and texture features using generative priors.Its stages use semantic-aligned condition interpolation, SLAT-guided structure interpolation, and weighted fine-grained texture fusion.
  • Interp3D: The progressive strategy targets semantic ambiguity, geometric inconsistency, and texture blurring while producing structurally faithful and texturally coherent trajectories.
  • Evaluation: Interp3DData organizes textured 3D morphing into three difficulty levels and reports superior performance over prior baselines, especially on hard cases.The evaluation considers fidelity, transition smoothness, and plausibility.

2 RELATED WORK

Prior work studies 3D morphing through geometric deformation and develops generative interpolation mainly in image or latent spaces. Compared with 2D settings, 3D interpolation remains less explored in its structural and semantic alignment requirements.

  • 3D Morphing: Traditional 3D morphing methods interpolate shapes through manifold geodesics, explicit deformations, or deformation fields over 3D representations.
  • 3D Generative Models: 3D generative models have progressed from GANs to diffusion models operating on point clouds, meshes, radiance fields, and 3D Gaussians.
  • Interpolation in Generative Models: Generative interpolation has been explored for morphing and editing in images and video, including latent-space interpolation in GANs and VAEs.
  • Interp3D Pipeline: The pipeline overview organizes interpolation around semantic-aligned condition interpolation, SLAT-guided structure interpolation, and fine-grained texture fusion.
  • Research Gap: 3D interpolation remains underexplored, particularly regarding the structural and semantic alignments required beyond 2D morphing.

3 PRELIMINARIES

The preliminaries describe TRELLIS as the 3D generative prior and attention interpolation as a mechanism for blending source and target features during generation. Together, these components provide the representation and fusion basis for Interp3D.

  • TRELLIS Structure: TRELLIS encodes a unified Structured Latent representation and generates active voxel positions before recovering texture-aware SLAT features.Its two diffusion stages separately generate structure and construct SLAT features conditioned on embedded images.
  • TRELLIS Structure: The SLAT decoder maps texture-aware latent features to corresponding 3D Gaussians after the generative stages.Both stages iteratively denoise latent codes into structural and textural representations.
  • Attention Interpolation: Attention interpolation fuses source and target features within generative-transformer attention for a sample-specific interpolation ratio.
  • Attention Interpolation: The formulation linearly combines source and target keys and values while using the interpolated features inside the attention mechanism.

4 METHODOLOGY

Interp3D progressively aligns semantic conditions, 3D structure, and texture features to generate smooth, plausible transitions between textured 3D assets. Its pipeline combines condition interpolation, SLAT-guided structural correspondence, and fine-grained texture fusion.

  • 4 METHODOLOGY: Interp3D generates a sequence of textured 3D assets with smooth and plausible transitions from source and target image prompts.The source and target assets occupy the sequence endpoints, while intermediate assets are generated at interpolation ratios.
  • 4.1 SEMANTIC-ALIGNED CONDITION INTERPOLATION: Semantic-aligned condition interpolation matches source and target embeddings before token-wise interpolation, reducing category-level mismatches.DINOv2 embeddings are matched through patch correspondences, then interpolated as convex combinations of aligned token pairs.
  • 4.2 SLAT-GUIDED STRUCTURE INTERPOLATION: SLAT-guided structure interpolation extends semantic alignment into 3D by dynamically matching source and target patches during coarse-to-fine denoising.Patch granularity decreases as denoising proceeds, enabling coarse early alignment and finer later alignment.
  • 4.2 SLAT-GUIDED STRUCTURE INTERPOLATION: The aligned SLAT correspondences permute target geometric keys and values before fused attention, preserving source spatial cues while integrating target structure.The resulting fused attention combines aligned source and target structural information with interpolated queries.
  • 4.3 FINE-GRAINED TEXTURE FUSION: Fine-grained texture fusion uses bidirectional similarity-based aggregation to combine source, target, and intermediate features despite differing voxel counts.Intermediate tokens retain their evolving identity, while L2 normalization limits feature-magnitude drift and the same update applies to value tokens.
  • 4.3 FINE-GRAINED TEXTURE FUSION: The texture-fusion updates jointly preserve endpoint information and intermediate structural cues, supporting coherent texture alignment across varying voxel resolutions.The same fusion scheme is applied consistently to keys and values.

5 EXPERIMENTS

Experiments evaluate Interp3D on a graded benchmark using quantitative metrics, user preference, and visual analysis. The method shows stronger fidelity, smoothness, plausibility, and component-wise improvements than selected baselines.

  • Evaluation Setup: Interp3DData contains 57 source-target pairs across easy, medium, and hard difficulty levels, with 19 pairs per level.The dataset covers humans, objects, buildings, cartoon characters, and other categories, using seven-frame morphing sequences.
  • Evaluation Setup: The evaluation measures fidelity, transition smoothness, and plausibility through FID, PPL, LPIPS, and user studies.User studies collect ratings on fidelity, smoothness, plausibility, and overall quality.
  • Quantitative Evaluation: Interp3D achieves 78.97 FID and 0.086 LPIPS, while competing methods show poor volumetric quality, 2D-to-3D inconsistencies, or reduced texture fidelity.MorphFlow records 104.88 FID; DiffMorpher and FreeMorph record PPL values of 4.42 and 5.61, respectively.
  • User Study: Interp3D receives 54.12% overall preference from 30 volunteers evaluating 15 morphing cases.The study focuses on transition smoothness, structural consistency, and textural plausibility.
  • Qualitative Analysis: Visual comparisons show smooth, semantically coherent transitions with preserved geometry and fine-grained textures, unlike baselines with blurred textures, structural collapse, or inconsistent frames.The red rabbit-to-tiger analysis links semantic alignment to corrected mismatches and sharper structural boundaries, while SLAT-guided interpolation strengthens structural consistency.
  • Ablation Study: Semantic alignment, structure interpolation, and texture fusion produce consistent improvements across difficulty levels, with texture fusion strongest on hard cases.Semantic alignment reduces easy-case FID by 4.06, while the full design improves hard-case PPL by 0.59 and LPIPS by 0.024.

6 CONCLUSION

Interp3D progressively enforces correspondence during generative textured 3D morphing, combining semantic conditions, SLAT-guided structure interpolation, and texture fusion. Evaluations on Interp3DData show more coherent transitions than existing baselines, while very weak semantic overlap remains challenging.

  • Conclusion: Interp3D enforces alignment progressively through semantically aligned condition interpolation, SLAT-guided structure interpolation, and fine-grained texture fusion.The stages target semantic correspondence, geometric fidelity, and detail-preserving texture transitions.
  • Conclusion: Interp3DData evaluates consistency, transition smoothness, and plausibility across graded difficulty levels.The dataset supports assessment of textured 3D morphing under varying case difficulty.
  • Conclusion: Interp3D outperforms existing baselines and delivers more coherent 3D transitions.The conclusion characterizes the resulting transitions as more coherent and elegant.
  • Conclusion: Cases with little semantic relevance remain a significant challenge because reliable correspondences for plausible transitions are difficult to build.The paper identifies this as a promising direction for future work.

A APPENDIX

The appendix organizes supplementary analyses covering generalization, implementation, ablations, dataset analysis, qualitative results, applications, and failure cases.

  • Appendix Organization: The appendix includes a generalization ability analysis and implementation and baseline-method details.These topics are listed as Sections A.1 and A.2.
  • Appendix Organization: Additional sections cover detailed ablations, Interp3DData analysis, and more qualitative results.These topics are listed as Sections A.3 through A.5.
  • Appendix Organization: The appendix also includes application and future analysis together with failure case analysis.These topics are listed as Sections A.6 and A.7.

A.1 INTERP3D GENERALIZATION ABILITY ANALYSIS.

Interp3D is presented as model-agnostic: it uses each 3D generator’s native feature cues to establish progressive correspondence. Tests on LN3Diff and 3DTopia-XL produce smooth and plausible morphings.

  • Generalization Ability: Interp3D is designed as a model-agnostic concept applicable to different 3D generation models.Its progressive correspondence modeling is guided by native feature cues from source and target generation processes.
  • Generalization Ability: The framework is instantiated on LN3Diff and 3DTopia-XL using semantic condition interpolation and model-native geometric embeddings.Examples include LN3Diff’s 3D latent tokens and 3DTopia’s primitive-level descriptors.
  • Generalization Ability: Both additional baselines produce smooth and plausible morphings, supporting Interp3D as an architecture design rather than a model-specific enhancement.The comparison is visualized in Figure 6.

A.2.1 PSEUDO CODE

The pseudocode implements Interp3D as a progressive three-stage pipeline, from semantic correspondence and condition interpolation through structure interpolation, texture fusion, and 3D decoding.

  • A.2.1 PSEUDO CODE: Interp3D progressively aligns semantic conditions, structure, and texture during diffusion-based morphing.The framework is explicitly organized around semantic, structural, and textural alignment.
  • A.2.1 PSEUDO CODE: The pipeline extracts source and target DINOv2 embeddings, estimates semantic correspondences, and interpolates aligned embeddings for each ratio αi.These operations establish the semantic-aligned condition interpolation stage.
  • A.2.1 PSEUDO CODE: For each interpolation ratio, SLAT-guided structure interpolation projects features to KV resolution, partitions patches, estimates correspondences, permutes target maps, and fuses attention.This stage uses dynamic patch correspondences to guide structural transformation during diffusion.
  • A.2.1 PSEUDO CODE: Fine-grained texture fusion matches similar source and target tokens, then aggregates and normalizes them before decoding updated features into 3D Gaussians.The decoded Gaussians form the intermediate morphing states.

A.2.2 EVALUATION DETAILS

The evaluation combines perceptual, structural, and semantic metrics with baseline comparisons, ablations, dataset construction, qualitative results, and failure-case analysis. Interp3D shows stronger continuity, geometric stability, and visual coherence, but struggles with extreme source–target gaps and out-of-distribution inputs.

  • A.2.2 EVALUATION DETAILS: The evaluation uses FID, LPIPS, and PPL to assess fidelity, adjacent-frame smoothness, and perceptual trajectory consistency.FID compares generated and reference feature distributions, while LPIPS and PPL evaluate transitions across rendered frames.
  • A.2.2 EVALUATION DETAILS: P-KID measures geometric fidelity, while CLIP-Dis and CLIP-Sim measure semantic continuity using generated intermediate shapes and rendered frames.Lower P-KID and CLIP-Dis are better, whereas higher CLIP-Sim indicates smoother semantic evolution.
  • A.3.1 EVALUATION ON SEMANTIC AND STRUCTURE FIDELITY: Interp3D consistently achieves better semantic continuity and geometric stability than prior morphing approaches on the whole Interp3DData dataset.Table 4 evaluates these properties using the additional semantic and structure fidelity metrics.
  • A.2.2 EVALUATION DETAILS: The ablation evaluates initial interpolation, semantic-aligned interpolation, SLAT-guided structure interpolation, and fine-grained texture fusion.Semantic-aligned condition interpolation substantially reduces FID and improves intermediate-state consistency.
  • A.2.2 EVALUATION DETAILS: Beta(5, 5) sampling produces more smoothly transformed intermediate shapes than uniform sampling by concentrating interpolation ratios toward the midpoint.The comparison concerns the distribution of αi used for intermediate states.
  • A.2.2 EVALUATION DETAILS: Interp3DData combines filtered assets from three sources into meaningful nontrivial source–target pairs organized into three difficulty levels.The dataset draws from Objerverse-XL, TRELLIS repositories, and Sketchfab, with examples including characters, vehicles, and architecture.
  • A.2.2 EVALUATION DETAILS: Qualitative comparisons show smoother, more coherent transitions with better-preserved geometry and appearance details than previous approaches.Baselines may exhibit blurred textures, structural collapse, or inconsistent intermediate frames.
  • A.7 FAILURE CASES ANALYSIS: When source and target assets are extremely distant, correspondences become unreliable and morphs can collapse toward one endpoint with abrupt transitions.This failure occurs when semantic and structural gaps prevent reliable correspondence establishment.
Loading 2601.14103v1…