PaperlayerView exact revision on arXiv

Source-linked AI summary

PhysSkin++: Real-Time and Generalizable Physics-Based Animation via Self-Supervised Neural Skinning

Yuanhang Lei, Baofei Liu, Xingxuan Li, Tao Cheng, Zixuan Wang, Boming Zhao, Yujun Shen, Siyuan Huang, Ruizhen Hu, Peter Yichen Chen, Hujun Bao, Zhaopeng Cui

arXiv:2610.26067v1cs.CVcs.GR

TL;DR

Learning real-time physics-based animation across diverse shapes and spatial discretizations from static geometry remains challenging. PhysSkin++ predicts continuous neural skinning fields and learns them through physics-informed self-supervision, then uses reduced dynamics for animation. Experiments demonstrate cross-shape generalization and real-time simulation, with support for control, contact, and rig-complementary responses.

  • Problem

    Learning physics-consistent deformation subspaces that generalize across shapes and discretizations from static geometries, without supervised training data, remains an open challenge.

    Source: §I · p. 1

  • Method

    A neural skinning fields autoencoder predicts continuous deformation subspaces from static geometry, trained with physics-informed self-supervision and used in reduced dynamics.

    Source: §I · p. 1, §V · p. 14

  • Results

    Experiments demonstrate cross-shape generalization of learned skinning fields and real-time physics-based animation across controls, interactions, and spatial discretizations.

    Source: B. Evaluation of Learned Skinning Fields · p. 12, §V · p. 15

  • Takeaways & Limitations

    The continuous deformation map supports animation binding to meshes, point clouds, and 3D Gaussians, while rig-space complementarity supports elastic responses without runtime complementarity constraints.

    Source: §V · p. 15

Abstract

from arXiv · show

Generalizing real-time physics-based animation across diverse 3D shapes and spatial discretizations remains challenging. In this paper, we introduce PhysSkin++, a physics-informed framework that learns continuous neural skinning fields from static geometries to construct compact deformation subspaces. Specifically, we propose a Neural Skinning Fields Autoencoder that predicts shape-dependent deformation subspaces without per-object network training. To learn these subspaces without simulation trajectories, we develop a Physics-Informed Self-Supervised Learning strategy that combines skinning-field normalization with conflict-aware gradient correction, stabilizing joint optimization under physical and orthogonality constraints. The learned representation supports efficient reduced dynamics with user control and contact interactions, together with direct animation binding to meshes, point clouds, and 3D Gaussians. For prescribed rig animation, we further introduce a self-supervised complementarity objective to learn complementary elastic responses without enforcing complementarity constraints at runtime. Experiments demonstrate cross-shape generalization of the learned skinning fields, real-time reduced simulation, and applications to controllable, collision-aware, and rig-complementary animation.

I. INTRODUCTION

PhysSkin++ addresses the challenge of learning physics-consistent deformation subspaces that generalize across 3D shapes and discretizations from static geometry. It combines a neural skinning autoencoder and self-supervised optimization with reduced dynamics for interactive and rig-complementary animation.

I. INTRODUCTION

  • Classical subspace mappings are tied to a mesh topology and resolution, while neural mappings still require per-object training and trajectory-based approaches require extensive supervision.These limitations motivate learning a physics-consistent deformation mapping across shapes and discretizations directly from static geometries.Source: §I · p. 1
  • PhysSkin++ learns continuous skinning fields from static geometry, avoiding per-object network training, simulation trajectories, skinning annotations, and category-specific priors.The representation generalizes across object categories, topologies, and resolutions.Source: §I · p. 1
  • The Neural Skinning Fields Autoencoder uses a transformer-based shape encoder and cross-attention decoder to predict continuous skinning fields.The fields define a deformation subspace mapping from static geometries.Source: §I · p. 2
  • PISSL combines on-the-fly skinning-field normalization with conflict-aware gradient correction to stabilize joint optimization of self-supervised loss terms.Normalization regulates weight scale and prevents numerical drift; gradient correction resolves destructive interference among loss-term gradients.Source: §I · p. 2
  • Reduced dynamics supports Dirichlet boundary conditions, interactive control, environmental interaction, and multi-object contact, with binding to meshes, point clouds, and 3D Gaussians.The formulation supports rigid–deformable and deformable–deformable interactions.Source: §I · p. 2
  • For prescribed rig animation, a rig-Jacobian complementarity term learns complementary elastic responses without enforcing the complementarity constraint at runtime.The work also evaluates interactive rig control and user-authored animation sequences.Source: §I · pp. 2–3

II. RELATED WORK

Prior neural subspace methods either rely on simulation sequences or train per object, while data-free physics-informed methods can struggle to produce high-quality representations. PhysSkin++ predicts continuous skinning fields from static geometry and advances reduced coordinates for efficient animation.

II. RELATED WORK

  • Data-driven subspace methods fit neural mappings to simulation sequences, while data-free methods optimize low-energy mappings; Simplicits stabilizes training with orthogonality constraints but uses a per-object network.PhysSkin++ instead predicts skinning weights across shapes while retaining explicit reduced-order dynamics.Source: §II · p. 3
  • Supervised rigging methods depend on expert annotations, and category-specific priors can limit their applicability across object types.PhysSkin++ trains neural skinning fields from static 3D geometries without annotated data.Source: §II · p. 3

A. Framework Overview and Problem Formulation

  • PhysSkin++ uses continuous neural skinning fields as a subspace mapping and advances only reduced coordinates through implicit time integration to reduce computational cost.The mapping represents full-space deformation using handle transformations as reduced coordinates.Source: A. Framework Overview and Problem… · pp. 3–4
  • The framework samples surface points for shape encoding and volumetric cubature points for animation, then decodes shape-conditioned features into neural skinning fields.Self-supervised learning uses physical and orthogonality constraints.Source: Figure 2 · p. 4

B. Neural Skinning Fields Autoencoder

The Neural Skinning Fields Autoencoder predicts continuous, shape-dependent skinning weights from geometry, producing deformation subspaces without per-object network training. Physics-informed self-supervision organizes these fields using physical, smoothness, and orthogonality objectives, with normalization and gradient correction to stabilize joint optimization.

B. Neural Skinning Fields Autoencoder

  • A single feed-forward autoencoder predicts continuous skinning weights across shapes and discretizations, avoiding the separate network per object used by prior neural subspace methods,.It encodes a sampled point cloud with pretrained Michelangelo shape latents, then decodes those latents into handle and point-wise skinning features.Source: B. Neural Skinning Fields Autoencoder · p. 4
  • Interior and surface cubature points provide spatial queries for potential-energy training, capturing volumetric deformation behavior that surface-only sampling cannot.Interior points are sampled from a voxelized watertight mesh; Sharp Edge Sampling captures surface details.Source: B. Neural Skinning Fields Autoencoder · p. 5
  • The autoencoder’s continuous fields are queried at spatial points and orthogonalized with an ONI module, while ELU permits flexible skinning modes without non-negativity constraints.Cubature points also specify locations for physics-informed training and numerical integration.Source: B. Neural Skinning Fields Autoencoder · p. 5

C. Physics-Informed Self-Supervised Learning

  • PISSL learns subspaces from static geometry without annotated skinning data or precomputed trajectories by combining potential-energy, smoothness, and orthogonality objectives.The potential-energy expectation is evaluated over sampled subspace coordinates and cubature points; training interpolates from linear elasticity to Neo-Hookean energy for stability.Source: C. Physics-Informed Self-Supervised… · p. 5
  • On-the-fly ℓ2 normalization regulates skinning-mode scale, while ConFIG corrects conflicting gradients to stabilize joint optimization and yield low-energy, smooth, approximately orthogonal fields.The optimization visualization shows initially disordered fields becoming physically consistent and spatially smooth.Source: C. Physics-Informed Self-Supervised… · pp. 5–6, Figure 3 · p. 5

D. Skinning-Based Real-Time Animation

The learned skinning subspace supports implicit reduced dynamics, with control and contact represented through added energy terms. Boundary-compatible deformation maps, force projection, and point-based collision energies support prescribed motion, user interaction, and multi-object contact.

D. Skinning-Based Real-Time Animation

  • Implicit reduced dynamics updates the low-dimensional coordinates, with the lower subspace dimensionality enabling efficient optimization and real-time physics-based animation.The potential energy supports general hyperelastic material models and the solve can use standard Newton-based methods.Source: D. Skinning-Based Real-Time Animation · p. 6
  • Boundary-compatible deformation maps enforce prescribed Dirichlet displacements while making reduced-coordinate displacement vanish on the constrained region.Selecting different boundary subsets produces distinct boundary-conditioned deformation behavior under the same formulation.Source: D. Skinning-Based Real-Time Animation · p. 6
  • External forces are mapped into reduced coordinates, and Gaussian-kernel-weighted local forces support interactive dragging around a selected point.Fig. 4 also shows prescribed-region constraints and planar-environment contact across diverse shapes.Source: D. Skinning-Based Real-Time Animation · p. 6, Figure 4 · p. 7
  • Boundary-penalty energies model environmental contact, while point-based IPC-style barriers support rigid–deformable and deformable–deformable interactions.For planar contact, the same boundary-penalty formulation supports diverse geometries and boundary orientations.Source: D. Skinning-Based Real-Time Animation · pp. 6–7
  • For deformable–deformable contact, per-object reduced states are coupled through contact barriers, whose derivatives assemble into a global gradient and Hessian that jointly resolve contact.The multi-object formulation sums each object’s inertial, elastic, and control energies.Source: D. Skinning-Based Real-Time Animation · pp. 7–8, Figure 5 · p. 8

E. Rig-Complementary Animation

Rig-complementary animation combines a prescribed LBS rig displacement with a learned elastic displacement parameterized by reduced coordinates. A self-supervised rig-space complementarity objective trains the elastic component to avoid duplicating rig motion, so runtime dynamics need not enforce complementarity constraints.

E. Rig-Complementary Animation

  • The rig displacement is computed from hierarchical bone transformations and LBS weights, while the learned skinning term supplies complementary elastic displacement.The rig is directly controllable, but its kinematic deformation alone cannot produce physical elastic responses.Source: E. Rig-Complementary Animation · p. 8
  • The rig Jacobian defines the rig-motion subspace, and the complementarity condition makes learned elastic displacement orthogonal to that subspace.The condition is formulated over batched cubature-point displacements.Source: E. Rig-Complementary Animation · p. 9
  • The rig-space complementarity loss augments the self-supervised subspace objective and is optimized with the same normalization and conflict-aware gradient correction.The objective uses the prescribed rig and its Jacobian evaluated on the same batch points as the complementary skinning representation.Source: E. Rig-Complementary Animation · p. 9
  • At runtime, prescribed rig poses drive implicit reduced dynamics for the complementary coordinates, without additional complementarity constraints in the solve.The rig trajectory is supplied as an external kinematic input.Source: E. Rig-Complementary Animation · p. 9
  • The formulation supports interactive rig control and artist-authored pose sequences enriched with physics-based complementary elastic response.The representation combines prescribed rig displacement with elastic displacement parameterized by reduced coordinates.Source: E. Rig-Complementary Animation · pp. 8–9

F. Discretization-Agnostic Animation Binding

Continuous neural skinning fields let PhysSkin++ bind one reduced physical state to different spatial discretizations, including 3D Gaussian primitives. For Gaussians, the method updates centers and covariances from local affine maps driven by the reduced coordinates.

F. Discretization-Agnostic Animation Binding

  • The continuous fields can be evaluated at mesh vertices, sampled points, and neural scene primitives, so one reduced physical state deforms different spatial discretizations.The paper demonstrates direct binding to anisotropic 3D Gaussian primitives.Source: F. Discretization-Agnostic Animation… · p. 9
  • For each Gaussian, PhysSkin++ evaluates skinning weights at its rest center and keeps those weights fixed throughout the kernel.The resulting local affine map is updated from the current reduced coordinates.Source: F. Discretization-Agnostic Animation… · pp. 9–10
  • At runtime, the reduced coordinates update Gaussian centers and covariances through the local affine maps, while opacity and spherical-harmonic appearance coefficients retain their rest values.The center follows skinned point motion; the covariance captures induced rotation and anisotropic deformation.Source: F. Discretization-Agnostic Animation… · p. 10

A. Implementation Details

The 3D shape encoder uses the pretrained Michelangelo model, and the framework samples surface points and cubature points for training. The evaluation includes a quantitative comparison on RigNet.

A. Implementation Details

  • For the 3D shape encoder, the method adopts the pre-trained Michelangelo model, and the passage specifies that this model consists of one cross-attention block.Source: A. Implementation Details · p. 10
  • Table I presents a quantitative comparison on the RigNet dataset.The supplied table passage gives the dataset scope but no result values.Source: Table I · p. 11
  • For each training batch, the method samples 1000 cubature points and 1024 subspace coordinate vectors; it uses 4096 surface points as encoder input.The cubature candidates are formed from surface samples and interior points in a 128^3 voxel grid.Source: A. Implementation Details · p. 11

B. Evaluation of Learned Skinning Fields

The evaluation compares neural skinning fields using structural metrics and qualitative visualizations across RigNet and ShapeNet. PhysSkin++ outperforms the compared baselines on all three metrics and produces smooth, orthogonal, physically consistent fields that generalize to unseen shapes.

B. Evaluation of Learned Skinning Fields

C. Ablation Studies

Ablations examine components of the self-supervised learning strategy and rig-space complementarity. The reported results connect these components to field stability, orthogonality, physical interpretability, and spatial organization.

C. Ablation Studies

  • Ablations remove or modify smoothness and orthogonality losses, skinning normalization, the ONI layer, and conflict-aware gradient correction.The studies use the RigNet dataset and include the rig-space complementarity term.Source: C. Ablation Studies · p. 12
  • The tested components maintain numerical stability and orthogonality, which the paper identifies as important for well-conditioned, physically consistent deformation.Table III reports the ablation studies on RigNet.Source: C. Ablation Studies · p. 13, Table III · p. 13
  • Without the potential-energy loss, the learned fields are not physically interpretable; adding the complementarity loss changes field distributions across character regions.Reported differences include ears, wings, and limbs across multiple characters.Source: C. Ablation Studies · p. 13

D. Diverse Real-Time Physics-Based Animation

PhysSkin++ combines efficient reduced simulation with controlled deformation, contact handling, rig-complementary motion, and animation of 3D Gaussian models.

Time Efficiency

Controllable and Collision-Aware Animation

  • Dirichlet constraints, localized dragging, and boundary penalties support controlled deformation and contact with planar environments across diverse shapes.Rigid–deformable and deformable–deformable interactions use point-pair IPC-style barriers, with floor contact handled by boundary penalties.Source: D. Diverse Real-Time Physics-Based… · p. 13

Rig-Complementary Animation

  • PhysSkin++ adds physics-based complementary deformation to interactive and user-authored rig animations.For user-authored sequences, Mixamo animations are retargeted to target character rigs using Auto-Rig Pro.Source: D. Diverse Real-Time Physics-Based… · p. 13

Discretization-Agnostic 3D Gaussian Animation

  • On ShapeSplat 3D Gaussian models, skinning fields queried at Gaussian rest centers update primitive centers and anisotropic covariances for coherent animation.The representation is evaluated directly on Gaussian primitives without an underlying mesh discretization.Source: D. Diverse Real-Time Physics-Based… · p. 13

Rig-Complementary Animation

  • With and without the rig-space complementarity term, learned skinning fields show different spatial distributions across characters and rig structures.The figure compares prescribed rig structures, LBS weights, and learned fields for both settings.Source: Figure 13 · p. 14

V. CONCLUSION

PhysSkin++ learns continuous skinning-based deformation subspaces from static geometries using self-supervised physical objectives. The framework supports reduced physics-based animation across shapes and discretizations, with user interactions, rig-complementary responses, and direct binding to several representations.

  • PhysSkin++ uses a neural skinning fields autoencoder to learn continuous skinning fields from static geometries and define a subspace mapping.Source: §V · p. 14
  • PISSL promotes physically plausible deformation and stabilizes optimization through on-the-fly field normalization and conflict-aware gradient correction.Source: §V · p. 14
  • Reduced dynamics supports Dirichlet conditions, interactive control, environmental interaction, and rigid–deformable and deformable–deformable contact.Source: §V · p. 15
  • A rig-space complementarity term supports elastic responses to prescribed rig animations without enforcing complementarity constraints at runtime.Source: §V · p. 15
  • The continuous deformation map binds to meshes, point clouds, and 3D Gaussian representations; the current formulation lacks semantic priors for complex geometries.Source: §V · p. 15

Related papers

Loading 2610.26067v1…