PaperlayerView exact revision on arXiv

Source-linked AI summary

LatentReRig: An SDF-Based VAE with Dual Decoders for Latent-Space Deformation Conditioning

Daniele Dolci, Fabrizio Poggioni, Carlo Melchiorri

arXiv:2609.31720v1cs.CVcs.GR

TL;DR

Deformation transfer across characters is difficult when their geometry and topology differ. LatentReRig learns SDF-based geometric codes and uses relative latent changes to condition explicit vertex prediction. Several poses show coherent cross-identity directions, while accurate mesh prediction is less reliable for localized and corrective changes.

  • Problem

    Direct deformation transfer across differing character identities is challenging, and conventional approaches often rely on correspondences or other explicit transfer structures.

    Source: §1 · p. 1

  • Method

    LatentReRig couples an SDF-based variational representation with implicit reconstruction and an explicit vertex decoder conditioned on latent deformation information.

    Source: §1 · p. 1

  • Results

    The experiments support reusable deformation structure, but latent organization and successful geometric prediction remain distinct problems.

    Source: §6 · p. 12

  • Takeaways & Limitations

    Several poses produce coherent latent directions across identities that can guide held-out characters toward intended deformations, though explicit predictions are less accurate for localized and corrective changes.

    Source: §7 · p. 12

    The explicit decoder still predicts vertices on the source topology, and evaluation is limited to controlled humanoid bipeds rather than arbitrary character classes or unconstrained production data.

    Source: §6 · p. 12

Abstract

from arXiv · show

Transferring deformation between characters with different geometry and topology is challenging because conventional rigs encode behaviour through character-specific structures and correspondences. We present LatentReRig, an experimental framework that investigates whether pose-associated changes can instead be represented as reusable directions in a learned geometric latent space. An SDF-based variational autoencoder is coupled with two decoders: one reconstructs the implicit field, while the other predicts target vertex positions from source geometry and latent deformation conditioning. The source geometry may be neutral or already deformed. Experiments on a controlled humanoid dataset show that several poses induce coherent latent directions across identities, particularly for broad articulated motions. These signals can guide deformation of unseen characters, but explicit predictions remain less accurate for localized changes and corrective contributions. Diagnostic comparisons with repeated SDF sampling show that inter-identity distances exceed same-geometry resampling variability on average, while pose signals exhibit different margins above this baseline. The results support the presence of reusable pose-related structure and identify stable local conditioning and accurate mesh decoding as complementary requirements for improving transfer.

1 Introduction

LatentReRig investigates whether pose-related latent changes can condition deformation prediction across character identities, where direct vertex displacements depend on geometry, topology, and correspondence. Its scope is pose-associated geometry transfer rather than complete rig transfer.

1.1 Research Problem

  • Pose-associated deformation transfer asks whether information from a semantic pose on source characters can condition prediction of that pose on an unseen identity.Direct vertex displacements depend on the source identity's geometry, topology, vertex correspondence, and proportions, so reuse on another character is generally not well defined.Source: §1.1 · pp. 1–2

1.2 Central Hypothesis and Scope

  • LatentReRig tests whether learned pose displacements can be sufficiently identity-independent for reuse, without requiring complete disentanglement of pose and identity.The hypothesis assumes a shared semantic pose vocabulary and a neutral reference for each identity.Source: §1.2 · p. 2
  • The study is limited to pose-associated geometric transfer, not production-rig reconstruction, skeletal hierarchy inference, animation-control recovery, or general-purpose rig generation.The proposed target is prediction of deformed geometry on a different character.Source: §1.2 · p. 2

1.3 Contributions

  • The contributions include an SDF-based variational representation, dual implicit and explicit decoders, latent-difference analysis, and evaluation of transfer to unseen identities.The evaluation also examines identity separation, pose consistency, sensitivity to SDF resampling, failure modes, and pose–identity entanglement.Source: §1.3 · p. 2

2 Related Work

Related work spans 3D geometry representations, latent shape spaces, deformation transfer, and neural rigging. LatentReRig measures deformation as a difference between learned geometric codes and uses it to condition an explicit mesh predictor.

2.1 3D Geometric Representations and Implicit Fields

  • Implicit fields represent geometry continuously, while LatentReRig uses sampled SDFs for representation learning and retains polygon vertices for explicit deformation prediction.Unlike polygon meshes, SDF input is independent of vertex indexing and connectivity, though the final predictor still uses vertices.Source: §2.1 · p. 3

2.2 Autoencoders and Latent Shape Spaces

  • VAEs offer probabilistic latent shape representations but can under-use dimensions or suppress weak signals; LatentReRig tests whether poses induce repeatable latent displacements across identities.The vector-direction analogy motivates the hypothesis, but the method does not assume geometric deformation follows exact vector arithmetic.Source: §2.2 · p. 3

2.3 Deformation Transfer and Learned Deformation

  • Classical and learned deformation-transfer methods address cross-character deformation, while LatentReRig instead conditions an explicit mesh predictor with differences between learned geometric codes.Classical approaches may depend on correspondences, registration, cages, or other transfer structures.Source: §2.3 · p. 3

2.4 Neural Rigging and Related Systems

  • Neural rigging predicts animation structures such as joints, hierarchy, skinning weights, or controls; LatentReRig learns from rig-generated geometry to predict deformed meshes.Its focus is whether latent differences can condition deformation prediction on previously unseen identities.Source: §2.4 · p. 4

3 LatentReRig Method

LatentReRig learns a geometric latent representation from SDF samples and uses two decoders for implicit reconstruction and explicit mesh deformation. Relative latent changes condition vertex predictions using source geometry.

3.1 Method Overview

  • The method encodes SDF samples into a variational representation for implicit reconstruction and semantic regularization, then uses relative latent variation to condition explicit vertex prediction.The two decoders respectively assess geometry preservation and translate latent variations into mesh-space deformation.Source: Figure 1 · p. 4, §3.1 · p. 4

3.2 Dataset and Geometric Representation

  • A controlled corpus of rig-generated humanoid states spans identities, semantic poses, and deformation regimes, with SDF samples and explicit vertex data representing the same states.The explicit data provide source–target supervision, while SDFs encode geometry without directly depending on polygon connectivity.Source: §3.2 · pp. 4–5

3.3 SDF-Based Variational Representation

  • The variational SDF branch encodes character states and reconstructs signed distances, while identity and pose classifiers provide semantic pressure without establishing disentanglement.Latent analysis uses the posterior mean rather than a stochastic sample so deformation differences need not depend on resampling.Source: §3.3 · p. 5

3.4 Latent Deformation Conditioning

  • Deformation conditioning represents a transition as the difference between deterministic latent means, separating its direction from its magnitude.The latent difference conditions the vertex decoder; it is not directly decoded as a mesh translation.Source: §3.4 · p. 5

3.5 Explicit Vertex Decoder

  • The vertex decoder combines source geometry and latent conditioning in a coarse-to-fine prediction pipeline with virtual influences, per-vertex masking, local refinement, and pair-kind conditioning.A residual detail branch adds capacity for small localized components, including corrective-shape prediction.Source: §3.5 · pp. 5–6

3.6 Learning Objectives

  • The learning objectives pair SDF reconstruction and semantic supervision with explicit geometric losses, including a specialized corrective term for localized changes.The corrective weighting addresses the risk that affected vertices are numerically outweighed by the many expected to remain static; latent regularizers encourage direction consistency and pose separation but do not guarantee them.Source: §3.6 · p. 6

3.7 Staged Training Strategy

  • Training first learns the latent representation, then freezes the encoder while training the vertex decoder on explicit source–target mesh pairs.This staged procedure separates tests of semantic representation learning from tests of whether that representation supports explicit deformation prediction.Source: §3.7 · pp. 6–7

4 Experimental Evaluation

The evaluation tests whether pose-related latent structure is consistent across identities and can condition explicit predictions for held-out characters. Broad motions show more coherent latent directions and recognizable predicted actions, while localized changes and surface reconstruction remain less reliable.

4.1 Evaluation Setup

  • Evaluation uses an identity-level split that excludes test characters from training and considers latent consistency alongside explicit mesh quality.The criteria include intended-region movement, static-region preservation, surface coherence, normal consistency, and SDF reconstruction.Source: §4.1 · p. 7, §4 · p. 7

4.2 Latent Representation Evaluation

  • Latent representation quality is assessed through relative pose-code changes because accurate SDF reconstruction alone does not establish semantically meaningful latent differences.The evaluation asks whether the same semantic pose produces similar latent transitions across identities, rather than identical absolute codes.Source: §4.2 · pp. 7–8

4.3 Cross-Identity Pose Consistency

  • ElbowBentLf reaches a mean cosine similarity of approximately 0.958 ± 0.030, while localized poses show weaker cross-identity consistency.Broad or structurally constrained motions tend to produce more coherent directions than small facial, finger, and corrective-style signals.Source: §4.3 · p. 8

4.4 Explicit Mesh Prediction on Unseen Identities

  • Broad limb and clavicle motions often reproduce the requested semantic direction on unseen identities, but predictions remain less accurate than artist-authored targets.Small finger, facial, and corrective-style changes can be weak or incomplete; neither mean-pose nor nearest-identity conditioning dominates across all poses.Source: §4.4 · p. 8
  • For cornerUpLf, nearest-identity conditioning gives a more localized prediction than the global mean, which produces unintended mouth opening.The comparison is qualitative; no ground-truth mesh is displayed for measuring error.Source: Figure 2 · p. 8, §4.4 · p. 8

4.5 Application-Level Case Study

  • Held-out elbow predictions show recognizable bending but retain distortion and pinching around the joint.The neutral source and ground-truth targets are not displayed in the figure.Source: §4.5 · p. 8, Figure 3 · p. 9
  • The combined brow-lowering prediction responds on both sides but reconstructs them asymmetrically.The image does not isolate whether the discrepancy originates in the latent condition or the decoder.Source: §4.5 · p. 8, Figure 4 · p. 9
  • For elbowBentLf, the predicted corrective contribution partially restores elbow volume but remains weaker than the target and leaves artifacts.The source already contains the broad skeletal motion; the comparison shows skinning-only input, prediction, and the skinning-plus-blendshape target.Source: §4.5 · pp. 8–9, Figure 5 · p. 9

5 Identity, Pose, and the Latent Noise Floor

The diagnostics find that neutral identity codes remain separated beyond same-geometry SDF-resampling variability, while pose-conditioning signals exceed that baseline by varying margins. Pose-direction consistency also varies by pose: broad motions are more consistent across identities than some localized conditions, with residual identity–pose entanglement and static leakage in predictions.

5.1 Identity Separation

  • The mean inter-identity distance is approximately 27.01, and neutral identity codes show non-zero off-diagonal separation in the standardized latent representation.The comparison uses 20 neutral identities; Table 1 reports descriptive statistics.Source: Figure 6 · p. 10, §5.1 · p. 10

5.2 Encoder Noise Floor

  • Across single-pose experiments, the mean pose-signal-to-resampling-noise ratio is approximately 2.03 and the median is 2.06, with margins varying substantially across poses.Strong knee, elbow, and clavicle conditions approach or exceed 3, while several other conditions fall near 1.2–1.5; these are diagnostic summaries, not an absolute representability threshold.Source: Figure 7 · p. 10, §5.2 · p. 11
  • A set of 190 same-geometry resampling distances has mean 8.30, compared with the mean inter-identity distance of 27.01.Different SDF samplings of the same geometry can produce different deterministic posterior means; repeated-encoding statistics are diagnostic only.Source: Table 1 · p. 10, §5.2 · p. 10, §5.1 · p. 10

5.4 Identity–Pose Entanglement

  • ElbowBentLf reaches approximately 0.952 pose-delta cosine similarity while identities remain separated, whereas weaker finger and brow conditions fall to approximately 0.79–0.80.Reduced pose consistency does not correspond to identity collapse; it indicates that the pose direction depends more on character-specific geometry.Source: Figure 8 · p. 11, §5.4 · pp. 11–12

5.3 Kolmogorov–Smirnov Analysis

  • The observed distance distributions are strongly separated, but the nominal KS p-value is not calibrated for inference because the 190 inter-identity distances reuse 20 latent codes.The comparison measures separation from SDF-resampling variability, not semantic organization or disentanglement by itself.Source: §5.3 · p. 11

5.4 Identity–Pose Entanglement

  • Predictions can deform regions intended to remain static, including the opposite leg during a left-knee bend and the opposite side of the face during a left puff action.The figure labels this motion outside the intended region as static leakage.Source: Figure 9 · p. 12

6 Discussion and Limitations

The experiments distinguish weak latent conditioning from inaccurate geometric decoding: reusable deformation structure can be present even when mesh predictions remain inadequate. The discussion also identifies residual identity dependence and limits in topology and evaluation scope.

6 Discussion and Limitations

  • Weak conditioning affects small facial, finger, and corrective changes near the resampling baseline, so improving the explicit decoder alone is unlikely to resolve them.The authors characterize this as local or regime-specific under-representation rather than complete posterior collapse.Source: §6 · p. 12
  • For broad poses, coherent latent conditions can still yield inaccurate geometry, and semantic plausibility does not imply production-level surface accuracy.Outputs may under-express motion, deform static regions, contain surface noise, oversmooth, or distribute deformation incorrectly.Source: §6 · p. 12
  • Static leakage can accompany recognizable motion: kneeBentLf affects the opposite leg, while puffLf deforms the opposite side of the face.These examples illustrate inadequate spatial isolation of predicted motion.Source: §6 · p. 12
  • Occasional gains from nearest_identity_delta over the global mean indicate that semantic pose directions are not fully identity-invariant.The stated objective is a shared semantic direction with identity-aware adaptation, not complete factorization.Source: §6 · p. 12
  • The explicit decoder predicts vertices on source topology, and evaluation is limited to controlled humanoid bipeds rather than arbitrary character classes or unconstrained production data.Thus, the framework is not fully topology-independent, and broader generalization is not established.Source: §6 · p. 12

7 Conclusion and Future Work

LatentReRig combines SDF-based variational representation learning with explicit vertex prediction to investigate deformation transfer across identities. Several poses yield coherent cross-identity latent directions that can guide held-out characters, while accurate reconstruction is less reliable for localized changes and correctives.

7 Conclusion and Future Work

  • LatentReRig couples SDF-based variational representation learning with explicit vertex prediction to investigate deformation transfer across character identities.Source: §7 · p. 12
  • Several semantic poses produce coherent cross-identity latent directions that guide held-out characters toward the intended deformation, but reconstruction is less reliable for localized changes and corrective contributions.Source: §7 · p. 12
  • Further development should stabilize conditioning for weak deformations and improve geometric interpretation, combining global SDF representations with local features and broader evaluation.The proposed evaluation extensions include more diverse identities, subtle correctives, and combined poses.Source: §7 · pp. 12–13

Related papers

Loading 2609.31720v1…