Source-linked AI summary
Neural Cages for Detail-Preserving 3D Deformations
Wang Yifan, Noam Aigerman, Vladimir G. Kim, Siddhartha Chaudhuri, Olga Sorkine-Hornung
TL;DR
The paper addresses the conflict between target alignment and preservation of source details in 3D deformation. It predicts a source cage and its deformation with a differentiable neural architecture trained end-to-end, then demonstrates feature-preserving shape variation synthesis and deformation transfer. The method is not tied to a single source shape or mesh topology, although some structures and deformation types remain imperfectly handled.
Problem
3D deformation must simultaneously match a target while minimizing distortion and preserving local geometric features, but these objectives can conflict and existing methods rely heavily on correspondences and parameter tuning.
Method
A neural architecture predicts a coarse source cage and its deformation, while a differentiable cage-based layer interpolates cage motion to deform the enclosed source shape.
Results
The method succeeds in generating feature-preserving deformations for shape variation synthesis and deformation transfer, better preserving salient geometric features than competing methods.
Takeaways & Limitations
Cage-based deformation provides a low-dimensional, detail-preserving deformation space usable with shapes that are not co-registered and do not share a single mesh topology.
Takeaways & Limitations
The losses do not always perfectly preserve rectilinear, planar, or parallel structures, and other parameterizations may be more natural for some deformations such as articulations.
Abstract
from arXiv · showhide
We propose a novel learnable representation for detail-preserving shape deformation. The goal of our method is to warp a source shape to match the general structure of a target shape, while preserving the surface details of the source. Our method extends a traditional cage-based deformation technique, where the source shape is enclosed by a coarse control mesh termed \emph{cage}, and translations prescribed on the cage vertices are interpolated to any point on the source mesh via special weight functions. The use of this sparse cage scaffolding enables preserving surface details regardless of the shape's intricacy and topology. Our key contribution is a novel neural network architecture for predicting deformations by controlling the cage. We incorporate a differentiable cage-based deformation module in our architecture, and train our network end-to-end. Our method can be trained with common collections of 3D models in an unsupervised fashion, without any cage-specific annotations. We demonstrate the utility of our method for synthesizing shape variations and deformation transfer.
1. Introduction
The paper addresses the conflict between matching a target shape and preserving source details by predicting deformations in a coarse cage space. Its differentiable neural cage architecture supports detail-preserving deformation across shapes and topologies without requiring dense correspondences in general.
- Motivation: 3D deformation must balance target alignment against distortion minimization and preservation of local geometric features.These objectives can conflict because perfect alignment may preclude preserving source details.
- Motivation: Existing optimization methods require parameter tuning and rely heavily on inferred or manually supplied source-target correspondences.The relevant parameters vary with shape category, representation, and source-target dissimilarity.
- Limitations of prior learning methods: Neural deformation methods can corrupt features and produce distortion, especially around thin structures, fine details, and large source-target discrepancies.The paper attributes these artifacts to neural networks’ limitations in capturing, preserving, and generating high-frequency content.
- Approach: The proposed architecture predicts a source cage and its deformation, then differentiably interpolates cage-vertex positions to deform the enclosed source surface toward the target.The pipeline is trained end-to-end on randomly chosen shape pairs.
- Approach: Cages provide a low-dimensional, highly regular deformation space that is feature-preserving by construction and whose degrees of freedom depend on coarse-cage resolution.This representation lets the network predict deformations without directly modeling every surface detail.
- Capabilities: The method is not tied to one source shape or mesh topology and can use crude or noisy targets without dense correspondences in general.The authors demonstrate shape variation synthesis and deformation transfer, including cases with sparse landmarks or image targets.
2. Related work
Prior work learns deformation through template vertices or implicit point mappings, but struggles with intricate details and correspondence requirements. This paper instead parameterizes deformation with cage-based coordinates and differentiable cage layers.
- Learning 3D deformations: Learning-based 3D geometry methods often deform generic or category-specific templates to produce intricate shapes.Earlier approaches represent deformation as template vertex positions or learned mappings conditioned on shape descriptors.
- Learning 3D deformations: Point-mapping architectures support novel sources by conditioning deformation fields on source and target features, but tend to blur high-frequency details.The cited limitation is associated with network capacity limits.
- Traditional mesh deformation: Cage-based deformation encloses the source in a coarse mesh and represents surface points as linear combinations of cage vertices using generalized barycentric coordinates.Prior coordinate designs aim to preserve shape structure and details during interpolation.
- Applications and correspondence: Shape synthesis and deformation transfer commonly require source-target correspondence, either explicitly through matched points or implicitly through target-conditioned deformation fields.The paper positions cage parameterization as an alternative to these correspondence strategies.
- This work: The proposed representation uses differentiable cage weights and deformation to combine learning-based source-target conditioning with classical detail-preserving interpolation.The method’s MVC layer computes mean value coordinates and applies them to the deformed cage.
3. Method
The method learns cages and their deformations with a differentiable cage-based deformation layer, transferring coarse cage motion to detailed source surfaces. Its losses balance target alignment with geometric and feature preservation.
- Cage-based deformation: Cage-based deformation warps ambient space through a coarse enclosing mesh whose vertex translations are interpolated to the source shape.Mean value coordinates are chosen for feature preservation, interpolation, simplicity, and differentiability.
- Learned cage control: The network predicts an initial source cage and a deformed cage from source and target shapes, enabling end-to-end learning of the deformation.The two differentiable branches predict the cage and its offset jointly.
- Network design: The streamlined branches predict cages rather than detailed geometric features, using a template cage that defaults to a 42-vertex sphere.This design avoids requiring high-resolution inputs or elaborate architecture tuning.
- Loss terms: The basic loss combines cage validity, target alignment, and shape preservation, with alignment measured by Chamfer distance without correspondences or L2 distance with correspondences.Negative mean-value coordinates are penalized when the cage is concave, self-overlapping, or fails to enclose source points.
- Applications: Figure 3 depicts brown source shapes deformed toward green target shapes to synthesize shape variations.The comparison illustrates the intended source-to-target deformation operation.
- Loss terms: Shape preservation penalizes changes in local point-to-surface distances and, for man-made shapes, adds PCA-normal consistency and symmetry losses.The normal penalty supports preservation of planar elements such as tabletops, while the final shape loss differs between man-made shapes and characters.
4. Applications
The method supports shape variation synthesis and deformation transfer while preserving source details and aligning outputs to targets. Its applications include noisy or image-derived targets and novel humanoid characters with sparse correspondences.
- Stock amplification via deformation: The method synthesizes plausible shape variations by deforming randomly paired source and target shapes within categories such as chairs, cars, and tables.It is trained on ShapeNet categories and uses other shapes as targets.
- Stock amplification via deformation: Compared with optimization- and learning-based alternatives, the method better preserves source details while aligning deformed shapes to targets.The comparison includes non-rigid ICP, cycle-consistency, feature-preserving, and voxel-grid methods.
- Stock amplification via deformation: Anisotropic scaling preserves local structure but cannot model different proportion changes across regions, whereas the proposed deformation captures this regional variation.The method is compared against scaling the source bounding box to the target bounding box.
- Stock amplification via deformation: A coarse AtlasNet reconstruction from a real 2D image can serve as a target while the output preserves the source shape’s fine details.The proxy target may contain poor-quality geometry and artifacts.
- Stock amplification via deformation: Quantitative evaluation uses Chamfer distance for target alignment and cotangent-Laplacian difference for feature distortion; the method preserves shape more effectively while capturing target structure.The metrics are computed from dense sets of 5000 uniformly sampled points and neither metric is optimized by all methods.
- Deformation transfer: For deformation transfer, the network predicts cage deformations for novel humanoids, and sparse landmark correspondences align a novel source cage before applying the learned deformation.The transferred output preserves details such as faces and fingers, including for test poses not observed during training.
- Deformation transfer: The transfer pipeline optimizes a novel cage using mean value coordinate consistency and cotangent-Laplacian regularization, then applies the learned cage offset to produce the final deformed shape.The optimized cage is initialized from the static source cage and the final shape is deformed through cage-based interpolation.
- Deformation transfer: The approach produces more plausible inter-shape correspondences and deformations than per-pair optimization.This comparison is reported for deformation transfer.
5. Evaluation
The evaluation studies loss terms and components using alignment and feature-distortion metrics. It examines regularization effects and the contribution of shape-preservation losses.
- Evaluation setup: The ablation measures matching error with Chamfer distance over 5000 uniformly resampled points and feature distortion with cotangent-Laplacian distance.All models are normalized to a unit bounding box.
- Ablations: Higher LMVC regularization produces more conservative deformations.The effect is illustrated in Figure 12.
- Ablations: Table 1 evaluates the effects of LMVC, Lshape, and the number of components using scaled Chamfer distance and cotangent-Laplacian metrics.Chamfer distance is scaled by 10^2 and cotangent Laplacian by 10^3.
- Ablations: The full shape-preservation loss combines point-to-feature, normal, and symmetry terms, while the normal term prevents bending of rigid shapes.The study compares the full loss with versions using only individual terms.
6. Conclusion
The paper shows that classical cage-based deformation can serve as a low-dimensional, detail-preserving space for deep learning. Its differentiable cage layers support shape variation synthesis and deformation transfer, while several limitations remain in cage geometry, structural preservation, and deformation parameterization.
- Cage-based deformation provides a low-dimensional, detail-preserving deformation space directly usable in deep-learning architectures.
- Differentiable cage-weight computation and deformation layers enable integration of the representation into other neural architectures.
- The method generates feature-preserving deformations for shape variation synthesis and deformation transfer, preserving salient geometric features better than competing methods.
- The predicted cage geometry is not designed to match professionally created cages for 3D artists.
- The losses do not always perfectly preserve rectilinear, planar, or parallel structures in man-made shapes.
- For some deformations, alternative parameterizations such as skeleton-based deformation may be more natural, while Green Coordinates could reduce affine distortion for large articulations.
7. Supplemental
The supplemental material provides additional stock-amplification and deformation-transfer results, including category-specific tests, humanoid pose experiments, and transfer to novel characters. It also compares rest-pose and T-pose training variations and notes affine-invariance-related distortions.
- Stock amplification via deformation: Additional stock-amplification results cover chair, table, and car categories using 100 unseen source-target pairs sampled from ShapeNet.
- Deformation transfer: A manually created template cage is deformed to match novel target shapes, and the cage deformation is transferred to new characters with similar poses.
- Deformation transfer: Additional deformation-transfer results evaluate 100 unseen humanoid target poses and transfer deformations to a woman, skeleton, and robot.
- Training variations: Rest-pose training produces less distortion but underperforms on large arm articulations, whereas T-pose training matches articulation slightly better but distorts the arms more.
- Limitations: The supplemental material attributes observed distortions to MVC affine-invariance and suggests Green Coordinates as a possible improvement.