PaperlayerView exact revision on arXiv

Source-linked AI summary

MorphAny3D: Unleashing the Power of Structured Latent in 3D Morphing

Xiaokun Sun, Zeyu Cai, Hao Tang, Ying Tai, Jian Yang, Zhenyu Zhang

arXiv:2601.00204v3cs.CV

TL;DR

3D morphing remains challenging because generating smooth, semantically consistent deformations across categories is difficult. MorphAny3D uses training-free SLAT feature fusion in attention, with temporal-fusion and orientation correction strategies. The authors report state-of-the-art results for smooth, high-quality morphing across diverse categories.

  • Problem

    Generating smooth, semantically consistent 3D deformations remains challenging, particularly across object categories.

    Source: §1 · pp. 1–2

  • Method

    MorphAny3D performs training-free 3D morphing with SLAT, using cross-attention to fuse source and target features, self-attention to incorporate prior-frame features, and orientation correction.

    Source: §1 · p. 2, §5 · p. 8

  • Results

    The authors report state-of-the-art results for smooth, high-quality 3D morphing across diverse object categories.

    Source: §5 · p. 8

  • Takeaways & Limitations

    MorphAny3D supports decoupled morphing and 3D style transfer and generalizes to other SLAT-based models.

    Source: §5 · p. 8

    High runtime restricts application, and the method may produce artifacts on extremely fine structures.

    Source: §5 · p. 8

Abstract

from arXiv · show

3D morphing remains challenging due to the difficulty of generating semantically consistent and temporally smooth deformations, especially across categories. We present MorphAny3D, a training-free framework that leverages Structured Latent (SLAT) representations for high-quality 3D morphing. Our key insight is that intelligently blending source and target SLAT features within the attention mechanisms of 3D generators naturally produces plausible morphing sequences. To this end, we introduce Morphing Cross-Attention (MCA), which fuses source and target information for structural coherence, and Temporal-Fused Self-Attention (TFSA), which enhances temporal consistency by incorporating features from preceding frames. An orientation correction strategy further mitigates the pose ambiguity within the morphing steps. Extensive experiments show that our method generates state-of-the-art morphing sequences, even for challenging cross-category cases. MorphAny3D further supports advanced applications such as decoupled morphing and 3D style transfer, and can be generalized to other SLAT-based generative models. Project page: https://xiaokunsun.github.io/MorphAny3D.github.io/.

1. Introduction

3D morphing remains difficult because plausible, smooth deformations are hard to model, while existing approaches leave gaps in cross-category structure and temporal coherence. MorphAny3D addresses these challenges with training-free SLAT feature fusion in attention, alongside temporal and orientation strategies.

1. Introduction

  • Matching-based 3D morphing establishes correspondences before interpolation, but can overlook texture evolution and generalize poorly across categories.Inaccurate correspondences can produce structurally implausible results.Source: §1 · pp. 1–2
  • Lifting independently generated frames from a 2D morph can lose temporal consistency, while direct interpolation of Trellis noise and conditions lacks explicit structural and temporal constraints.Both strategies can produce suboptimal 3D morphing quality.Source: §1 · p. 2
  • MorphAny3D is a training-free SLAT-based framework designed for semantically coherent, smooth morphing across diverse 3D object categories.Its attention-level feature fusion underlies the approach.Source: §1 · p. 2
  • MCA fuses source and target features for structural coherence, while TFSA incorporates prior-frame features for temporal coherence and orientation correction mitigates abrupt pose shifts.The framework also supports decoupled morphing, 3D stylization, and transfer to other SLAT-based models.Source: §1 · p. 2

2. Related Work

  • Despite advances in diffusion-based 2D morphing, extending smooth, semantically coherent transformations to 3D remains an open challenge.Existing 3D correspondence methods also struggle to match objects reliably across diverse categories.Source: §2 · p. 3
  • Although SLAT supports several downstream tasks without retraining, its use for 3D morphing had not been explored; MorphAny3D investigates SLAT feature fusion in attention to address this gap.The paper builds on SLAT’s regular, explicit structure and transferability.Source: §2 · p. 3

3. Methodology

MorphAny3D uses Trellis’s SLAT representation for training-free 3D morphing, combining source–target attention, previous-frame features, and orientation correction. The method analyzes SLAT fusion patterns and introduces components intended to improve plausibility and temporal coherence.

3.1. Preliminaries

  • SLAT encodes local latent vectors anchored at active sparse voxels; Trellis predicts sparse structure, then local latents, and decodes the result into standard 3D representations.The two stages are the Sparse Structure (SS) stage and Structured Latent (SLAT) stage.Source: §3.1 · p. 3

3.2. Overview

  • MorphAny3D uses SLAT to morph source objects into targets, introducing attention-based fusion and orientation correction to improve plausibility and temporal coherence.For real assets, the method obtains initial noised latents and image conditions via 3D inversion; it uses cached features and conditions for Trellis-generated assets.Source: §3.2 · pp. 3–4, Figure 3 · p. 4

3.3. SLAT Fusion Patterns in Attention Mechanism

  • KV-Fused CA gives the lowest FID and KV-Fused SA the lowest PPL, but combining them compromises plausibility, motivating tailored attention modules.FID measures plausibility and PPL measures smoothness in the reported comparison.Source: §3.3 · p. 4, Figure 4 · p. 5

3.4. Morphing Cross-Attention (MCA)

  • MCA computes source and target attention outputs separately before weighting them, preserving attention to semantically consistent regions and avoiding KV-Fused CA’s artifacts.The reported feature trajectories for MCA are stable and smooth, while those for KV-Fused CA are erratic and interrupted.Source: §3.4 · p. 5, Figure 5 · p. 5, Figure 6 · p. 5

3.5. Temporal-Fused Self-Attention (TFSA)

  • TFSA fuses current-frame attention with attention from the previous frame, using β = 0.2 to balance their influence and enhance temporal smoothness.Unlike KV-Fused SA, TFSA fuses features from already plausible neighboring morphing frames.Source: §3.5 · pp. 5–6, Figure 7 · p. 6

3.6. Orientation Correction

  • Orientation jumps cluster near α ≈ 0.5 and favor 90°, 180°, and 270° yaw shifts, which also occur among non-canonical Trellis object poses.The correction tests four yaw-rotated sparse-structure candidates and selects the one with the lowest Chamfer Distance to the previous frame’s structure.Source: §3.6 · p. 6

4. Experiments

MorphAny3D is evaluated against matching-based, 2D-lifted, direct-interpolation, and modern 3D morphing baselines, with quantitative, qualitative, ablation, application, and generalization results. It achieves the best scores on four metrics and the second-best PPL, while supporting additional morphing applications and other SLAT-based models.

4.1. Experimental Settings

  • The evaluation uses 50 diverse source-target pairs, four metrics, and baselines spanning matching-based, 2D-lifted, direct-interpolation, and modern 3D morphing methods.RGB views are rendered from 3DGS and normal maps from mesh representations.Source: §4.1 · pp. 6–7, §4 · p. 6

4.2. Evaluation

  • MorphAny3D achieves the best FID, PDV, AS, and UP scores and the second-best PPL, which is just 0.06 above the lowest.Qualitative comparisons report smoother, more plausible sequences than the baselines.Source: Table 1 · p. 7, §4.2 · p. 7

4.3. Ablation Study

  • MCA lowers FID from 125.47 to 112.18; TFSA reaches PPL 2.87 and PDV 0.0007, while OC further lowers them to 2.47 and 0.0006.The ablations describe fewer local artifacts, improved temporal coherence, and mitigated orientation jumps.Source: §4.3 · pp. 7–8, Table 2 · p. 7, Figure 9 · p. 8

4.4. Applications

  • Selective use of Trellis’s SS and SLAT stages enables structure-detail disentanglement and dual-target morphing, while a style-image SLAT target transfers style and preserves source structure and aesthetics.Source: §4.4 · p. 8

4.5. Generalization Ability

  • Applied to Hi3DGen and Text-to-3D Trellis, MorphAny3D consistently produces smooth, high-quality morphs.Source: §4.5 · p. 8

5. Conclusion

MorphAny3D is a training-free framework using Trellis’s SLAT representation, with attention modules for structural and temporal coherence and orientation correction to reduce abrupt shifts. The authors report state-of-the-art morphing results, while noting limits involving fine structures, yaw-symmetric rotations, and runtime.

5. Conclusion

  • MorphAny3D combines MCA for structural and semantic fidelity, TFSA for temporal coherence, and orientation correction guided by pose patterns in Trellis outputs.Source: §5 · p. 8
  • The method supports decoupled morphing and 3D style transfer and generalizes to other SLAT-based models.Source: §5 · p. 8, Figure 10 · p. 8, Figure 11 · p. 8
  • Artifacts may occur on extremely fine structures, orientation correction may miss yaw-symmetric rotations, and high runtime restricts application.The authors suggest stronger 3D backbones, finer-detail SLAT representations, and acceleration such as KV cache as future directions.Source: §5 · p. 8, Figure 12 · p. 8

6. Supplemental Material Overview

The supplemental material lists six groups of additional content covering initialization, experiments, results, applications, and generalization.

6. Supplemental Material Overview

  • The supplement includes initialization details and additional experimental details.Source: §6 · p. 13
  • It provides extended 3D morphing results and additional qualitative experiments.Source: §6 · p. 13
  • It also includes additional application details and generalization details.Source: §6 · p. 13

7. Initialization Details

For real-world assets, initialization begins with 3D inversion to obtain initial noisy latents and image conditions. Frame features are then interpolated between source and target initial features, with sparse-voxel matching required for the SLAT stage.

7. Initialization Details

  • 3D inversion obtains initial noisy latents and image conditions through Sparse Structure and Structured Latent stages.Source: §7 · p. 13
  • SS frame features use spherical interpolation between source and target initial features, with deformation weight α_n spanning endpoints that correspond to source and target.Source: §7 · p. 13
  • SLAT-stage interpolation first finds corresponding sparse voxels using Euclidean distance.Source: §7 · p. 13

8. Additional Experimental Details

The appendix specifies Trellis-based implementation details, evaluation prompts, and metrics for comparing morphed assets.

8. Additional Experimental Details

  • The method replaces Trellis cross-attention with MCA and self-attention with TFSA, then applies orientation correction after obtaining each frame’s structure.The modules require no training; both stages use 25 sampling steps and CFG scale 5.0.Source: §8 · p. 13
  • Aesthetics Score records how often Gemini-2.5 and ChatGPT-5 select each method’s morphed asset as most visually appealing.The evaluation prompt asks models to judge harmony, surface detail, style consistency, and visual appeal.Source: §8 · p. 13, Figure 13 · p. 13
  • FID uses 12 yaw-sampled renders, while PPL and PDV use six morphing videos; user preference reflects selections by 49 experienced participants across 10 pairs.Participants chose based on quality, smoothness, and realism.Source: §8 · p. 13

9. Extended 3D Morphing Results

Extended results show smooth morphs across varied object categories and illustrate disentangled morphing and visual comparisons.

9. Extended 3D Morphing Results

  • Additional examples show high-quality, smooth morphs across animals, vehicles, buildings, and humanoids.RGB renders and mesh normal maps visualize appearance and geometric fidelity.Source: §9 · pp. 13–14
  • Figure 14 compares MorphAny3D qualitatively with 3DMorpher using results reproduced from the published paper.The figure caption identifies the comparison but does not state an outcome.Source: Figure 14 · p. 14
  • Figure 15 illustrates independent control of structural and detail deformation by applying MCA and TFSA exclusively in either the SS or SLAT stage.The caption presents this as disentangled 3D morphing.Source: Figure 15 · p. 14

10. Additional Qualitative Experiments

Additional comparisons report stronger morphing quality and temporal smoothness than baselines, alongside a qualitative comparison with 3DMorpher.

10. Additional Qualitative Experiments

  • MorphAny3D consistently outperforms baselines in morphing quality and temporal smoothness.For 3DMorpher, results are reproduced from its paper because full code is unavailable; the comparison uses the same source and target assets, lifted to 3D with Trellis.Source: §10 · p. 14
  • Compared with 3DMorpher, MorphAny3D’s results exhibit superior detail and coherence, while 3DMorpher struggles with complex structures.The passage also notes that 3DMorpher relies solely on 3DGS, which lacks high-fidelity normals and standard 3D-software compatibility.Source: §10 · p. 14

11. Additional Application Details

The appendix describes stage-specific morphing controls and applications that combine structural and detail information from different sources.

11. Additional Application Details

  • Applying MCA and TFSA only in the SS stage changes global shape while preserving local details; applying them only in SLAT transfers texture and fine geometry without changing overall structure.These settings provide structure-only and detail-only morphing, respectively.Source: §11 · p. 14
  • Dual-target morphing assigns one target’s global shape to SS and another target’s local details to SLAT, enabling asset compositions from distinct sources.Source: §11 · p. 14
  • For 3D style transfer, the SLAT stage uses a style reference while SS remains fixed to the source structure, preserving shape while adopting the reference’s geometric and textural style.Source: §11 · p. 14

12. Generalization Details

MorphAny3D extends to other SLAT-based generators and supports text-prompt adaptation, dual-target morphing, and 3D style transfer. Text conditions are more abstract than visual ones and reduce deformation quality and temporal coherence.

12. Generalization Details

  • Hi3DGen requires no retraining because it shares Trellis and image conditioning, but its geometry-only focus limits reported results to normal maps.Source: §12 · p. 14
  • Dual-target morphing assigns distinct assets to the SS and SLAT stages to combine global shape and local detail from different sources.Source: Figure 16 · p. 15
  • In 3D style transfer, a reference guides the SLAT stage while the SS stage preserves source structure, transferring texture and geometric style without changing shape.Source: Figure 17 · p. 15
  • Text prompts replace Trellis image conditions for zero-shot adaptation, but their abstraction reduces deformation quality and temporal coherence.The authors recommend image-guided SLAT models when possible.Source: §12 · pp. 14–15
  • The user-preference evaluation asks participants to choose among methods using rendered images and judge quality, smoothness, and realism.Source: Figure 18 · p. 16
  • Extended examples cover animals, vehicles, and buildings, with captions reporting high-fidelity, temporally smooth transitions and consistent quality across challenging categories.Source: Figure 19 · p. 17, Figure 20 · p. 18
  • Qualitative comparisons report superior morphing quality and smoother transitions, with further examples highlighting geometry preservation and temporal coherence.Source: Figure 21 · p. 19, Figure 22 · p. 19

Related papers

Loading 2601.00204v3…