Deep Implicit Templates for 3D Shape Representation

1. Introduction

• DIFs represent complex shapes and fine detail, but their implicit form provides no information about relationships between shapes, limiting dense correspondence for downstream shape understanding and editing.Mesh templates allow vertex-to-vertex correspondences, whereas establishing them with DIFs is difficult.Source: §1 · p. 1
• Deep Implicit Templates decompose a conditional DIF into a category-level template implicit function and a conditional spatial warping function that deforms it into specific instances.The representation keeps both components implicit and uses the template as an intermediate shape for dense correspondences.Source: §1 · p. 1
• The method distinguishes itself from element-based representations with a global template, inspectable deformations, and direct dense correspondences across 3D space.The template can be replaced with a custom design without losing representation power.Source: §1 · p. 1
• The framework also supports adding semantic constraints to spatial deformation for tasks such as dynamic human modeling.The introduction presents this as a specific application of the deformation framework.Source: §1 · p. 2
• Spatial Warping LSTM decomposes conditional warping into multi-step point-wise transformations, while progressive reconstruction loss and two-level regularization target accurate reconstructions and plausible templates.The stated training goal is to learn accurate dense correspondences without supervision.Source: §1 · p. 2

Related Work

• Local implicit-function mixtures target complex scenes and geometric detail, while dense correspondence across shapes remains difficult.Other cited extensions address shape manipulation or temporally continuous 4D sequences [39].Source: §2 · p. 2
• Elementary representations can be compact, but consistent fitting across shapes is challenging, and local elements lack the global knowledge needed for tasks such as shape completion.The discussion covers learned and manually fitted primitives, parts, charts, and other shape elements.Source: §2 · pp. 2–3
• Mesh-based templates efficiently represent similar shapes, but struggle with topological changes, surface detail, and deformation artifacts for shapes far from the template.The paper contrasts these constraints with the greater representation capacity of an implicit template.Source: §2 · p. 3

3. Overview

• The Deep Implicit Template representation builds on DeepSDF, which uses a conditional signed-distance function to represent multiple object instances.In DeepSDF, each instance has a unique latent code that is optimized alongside network training.Source: §3 · p. 3, §3.1 · p. 3

3.1. Review of DeepSDF

• DeepSDF represents a surface as a signed-distance field's level set, with a condition variable allowing one neural network to represent multiple instances.The condition variable encodes a specific object's shape.Source: §3.1 · p. 3

3.2. Deep Implicit Templates

• Deep Implicit Templates model shape variation relative to a shared template SDF, with W warping input coordinates and T returning signed-distance values at the mapped coordinates.The warping maps points to canonical template positions before the template is queried.Source: §3.2 · p. 3, Figure 2 · p. 4
• The decomposition naturally induces correspondences between the template and instances, and across instances, but can overfit to a complicated transformer and oversimplified template without suitable design and regularization.The target is a common-structure template and accurate dense correspondences while preserving DeepSDF's representation power and generalization capacity.Source: §3.2 · p. 3
• The paper loosely treats a warped SDF as another SDF under truncated fields that vary only near the surface and meshes normalized to the same scale.The authors note that a warped SDF is not generally an SDF in the strict sense.Source: §3.2 · p. 3

4.1. Network Architecture

• Spatial Warping LSTM decomposes each point’s spatial transformation into recurrent steps, with S=8 steps in the experiments.Unlike independent iterative estimates, its LSTM aggregates information from multiple previous steps.Source: §4.1 · p. 4

4.2. Network Training

• The network’s training loss combines reconstruction and regularization losses.Table 1 reports progressive-loss parameter settings for every other warping step.Source: §4.2 · p. 4, Table 1 · p. 4

4.2.1 Progressive Reconstruction Loss

• Progressive reconstruction supervises intermediate warping outputs to move from smooth approximations toward local detail; Table 1 gives settings only for every other step.The progressive loss sums terms for outputs at different warping-step counts.Source: §4.2.1 · p. 4, Table 1 · p. 4

4.2.2 Regularization Loss

• Point-wise regularization constrains point displacement after warping, assuming meshes are normalized to a unit sphere and aligned in a canonical pose.The loss uses a Huber kernel.Source: §4.2.2 · p. 5
• Point-pair regularization discourages extreme spatial distortion while allowing relaxed smoothness, using distortion tolerance ε=0.5 in the experiments.The authors report that this setting helps prevent shape structures from collapsing.Source: §4.2.2 · p. 5

5.1. Experimental Setup

• Experiments use ShapeNet with preprocessing following DeepSDF and evaluate reconstruction, correspondence, interpolation, and technical contributions.Baselines include DeepSDF, SIF, AtlasNet, PointFlow, and DualSDF.Source: §5.1 · p. 5

5.2. Results

• Learned templates capture shared structure and plausible dense correspondences, including across large deformations and objects with different structures.For highly varied categories such as chairs and tables, individual templates can be trained for different subclasses.Source: §5.2 · pp. 5–6, Figure 4 · p. 6

5.3. Comparison

• Reconstruction is comparable to state-of-the-art methods, with slightly better results for cars, airplanes, and sofas but degradation for structurally varied chairs.Table 2 compares reconstruction accuracy on known and unknown shapes using mean and median Chamfer distance.Source: §5.3 · p. 6, Table 2 · p. 6
• Linear latent-code interpolation preserves DeepSDF’s generative capability while interpolating spatial warping fields rather than signed distance fields.Figure 5 shows interpolated meshes between endpoint shapes.Source: §5.3 · p. 6, Figure 5 · p. 6
• The method outperforms other representations on keypoint-detection PCK despite using no keypoint or correspondence annotations during training.Keypoint detection serves as a surrogate for correspondence accuracy; Table 3 reports PCK scores at thresholds 0.01 and 0.02.Source: §5.3 · p. 7, Table 3 · p. 7

5.4. Ablation Study

• The Spatial Warping LSTM generalizes better for latent-space interpolation than the tested MLP, which is prone to overfitting.The authors attribute this to large non-uniform deformations being easier to learn gradually than in a single step.Source: §5.4 · p. 7, Figure 7 · p. 7
• Without point-pair regularization, templates become oversimplified and correspondences become inaccurate, even when reconstruction accuracy is unaffected.Details such as plane wings and sofa backrests can collapse into tiny template regions.Source: §5.4 · p. 7, Figure 8 · p. 7

6.1. Extensions

• A manually specified SMPL template can model clothed humans, and its correspondences allow skinning-weight transfer for animation.The SMPL template’s skinning weights can be transferred to clothed human models.Source: §6.1 · p. 8, Figure 9 · p. 8, Figure 10 · p. 8
• When sparse or dense correspondence annotations are available, an additional loss can enforce known correspondences during training.Figure 11 shows results with this correspondence loss on D-FAUST.Source: §6.1 · p. 8, Figure 11 · p. 8

6.2. Applications

• Dense correspondences support transferring mesh attributes such as texture and mesh stretching across multiple objects, and the representation can also perform shape completion.Figure 12 presents application results.Source: §6.2 · p. 8, Figure 12 · p. 8

• The representation factors deep implicit functions into an implicit template and its conditional deformations, giving the latent space a meaningful interpretation.Source: §7 · p. 8
• Its network architectures and training losses recover accurate dense correspondences while learning a plausible template that captures common geometry structures.Source: §7 · p. 8
• Experiments demonstrate promising applications, and the authors conclude that semantic interpretation makes deep implicit functions a more powerful representation.Source: §7 · p. 8