PaperlayerView exact revision on arXiv

Source-linked AI summary

STAR: Sparse Trained Articulated Human Body Regressor

Ahmed A. A. Osman, Timo Bolkart, Michael J. Black

arXiv:2008.08535v1cs.CV

TL;DR

SMPL’s dense pose correctives, shape-independent deformation model, and limited training diversity constrain compactness, realism, and body-shape coverage. STAR replaces these choices with sparse local correctives, shape-conditioned deformation, and broader training data, yielding a smaller model that generalizes better while remaining compatible with SMPL.

  • Problem

    SMPL’s dense blend shapes create spurious long-range correlations, while separating pose from shape and relying on limited body-shape data restricts realism and population coverage.

  • Method

    STAR learns sparse spatially local correctives, uses quaternion features, conditions pose corrections on pose and shape, and trains on complementary CAESAR and SizeUSA scans.

  • Results

    STAR has 20% of SMPL’s parameters, is 80% smaller, and generalizes better or performs more accurately than SMPL when trained on the same data.

  • Takeaways & Limitations

    STAR provides a compact, more local and shape-aware body model that serves as a drop-in replacement for SMPL and generalizes better to unseen bodies.

Abstract

from arXiv · show

The SMPL body model is widely used for the estimation, synthesis, and analysis of 3D human pose and shape. While popular, we show that SMPL has several limitations and introduce STAR, which is quantitatively and qualitatively superior to SMPL. First, SMPL has a huge number of parameters resulting from its use of global blend shapes. These dense pose-corrective offsets relate every vertex on the mesh to all the joints in the kinematic tree, capturing spurious long-range correlations. To address this, we define per-joint pose correctives and learn the subset of mesh vertices that are influenced by each joint movement. This sparse formulation results in more realistic deformations and significantly reduces the number of model parameters to 20% of SMPL. When trained on the same data as SMPL, STAR generalizes better despite having many fewer parameters. Second, SMPL factors pose-dependent deformations from body shape while, in reality, people with different shapes deform differently. Consequently, we learn shape-dependent pose-corrective blend shapes that depend on both body pose and BMI. Third, we show that the shape space of SMPL is not rich enough to capture the variation in the human population. We address this by training STAR with an additional 10,000 scans of male and female subjects, and show that this results in better model generalization. STAR is compact, generalizes better to new bodies and is a drop-in replacement for SMPL. STAR is publicly available for research purposes at http://star.is.tue.mpg.de.

1 Introduction

STAR addresses SMPL’s dense, non-local pose corrections, shape-independent deformations, and limited shape diversity with a compact model that retains SMPL’s parametrization while improving generalization and realism.

  • SMPL limitations: SMPL’s global pose-corrective blend shapes exceed 4.2 million parameters and learn spurious long-range correlations between joints and mesh vertices.These correlations can make moving one elbow produce a bulge in the other elbow.
  • Sparse local pose correctives: STAR learns sparse, spatially local joint-to-vertex influences, using differentiable ReLU thresholding to mask irrelevant vertices.The learned activation regions restrict each joint’s corrective offsets to nearby mesh vertices.
  • Compact representation: Quaternion features reduce pose inputs from nine matrix elements per joint to four, and combined with sparsity give STAR 20% of SMPL’s parameters.Users can continue supplying SMPL pose parameters because the quaternion change is internal.
  • Shape-dependent deformation: STAR conditions pose-corrective blend shapes on both pose and body shape, using a BMI-correlated shape component to model different deformations across body shapes.This addresses SMPL’s separation of identity shape from pose-dependent deformation.
  • Shape diversity: Training STAR on complementary CAESAR and SizeUSA scans improves generalization to unseen body shapes beyond either dataset alone.The SizeUSA dataset contributes 10,000 more diverse subjects than the older CAESAR population.
  • Compatibility and release: STAR is a drop-in replacement for SMPL with the same pose and shape parametrization and a publicly available 300-component shape space.The model is released for research use.

2 Related Work

Prior body models established data-driven pose and shape representations, but STAR builds on their limitations by combining SMPL-compatible modeling with localized, sparse, and shape-aware pose deformations.

  • Linear Blend Skinning: LBS provides a simple foundation by transforming each mesh vertex using weighted transformations from associated kinematic-tree joints.
  • Learned Models: SCAPE learns factored body shape and pose deformations but lacks explicit joints, may not preserve limb lengths, and is incompatible with graphics pipelines.
  • Learned Models: SMPL improves on SCAPE through vertex-based corrective offsets, LBS compatibility, greater accuracy on the same data, and training on the full CAESAR dataset.
  • STAR Compared with SMPL: SMPL uses 207 globally effective pose blend shapes, whereas STAR reduces this to 23 through quaternions, grouped-joint regressors, and spatial sparsity.
  • Sparse Pose Corrective Blend Shapes: STAR infers localized joint support from posed scan data rather than requiring manually posed joints, while retaining the benefits of a simple SMPL-like model.

3 Model

STAR combines vertex-based LBS with shape and pose corrective functions that are conditioned on body shape and restricted to learned, spatially local joint support regions. Its training jointly learns corrective regressors, activation masks, and skinning weights while regularizing for accuracy and sparsity.

  • Model: STAR extends vertex-based LBS with learned shape and pose corrective functions, including a separate pose corrective function for each non-root joint.
  • Shape Blend Shapes: Shape blend shapes map identity coefficients β to vertex offsets through principal components, producing a shaped template that represents subject attributes and identity.
  • Pose and Shape Corrective Blend Shapes: STAR conditions pose correctives on pose and the second shape-space principal component, which is highly correlated with BMI, to model shape-dependent deformation.
  • Activation Function: ReLU-thresholded activation weights define sparse support regions, with vertices having non-positive activation receiving no corrective offset from that joint.
  • Pose Corrective Regressor: Each joint’s corrective regressor uses quaternion values from the joint and direct kinematic neighbors, then masks offsets using learned vertex activations.
  • Blend Skinning: STAR adds the corrected mesh to standard LBS, regresses shape-dependent joint locations from the shaped vertices, and exposes 72 axis-angle pose parameters with up to 300 shape parameters.
  • Model Training: Training minimizes vertex-to-vertex error while regularizing corrective weights, activation masks, and skinning weights, with parameters optimized by stochastic gradient descent.
  • Model Training: Figure 3 illustrates learned activation sparsity and compares SMPL’s global elbow-induced offsets with STAR’s spatially local corrective offsets.

4 Experiments

Experiments show that STAR learns sparse, local deformations, reduces model size, and generalizes better with expanded training data and shape-dependent pose correctives.

  • 4.1 Activation: 3.37×10^5 and 3.94×10^5 non-zero pose parameters in STAR’s male and female models, versus 4.28 × 10^6 in SMPL.At test time, only non-zero parameters need to be stored.
  • 4.1 Activation: STAR’s sparse correctives avoid SMPL’s elbow-to-elbow bulge by restricting deformations to spatially local regions.The qualitative comparison attributes the difference to STAR’s learned local sparse pose-corrective formulation.
  • 4.3 Extended Training Data: A shape space trained on one dataset cannot sufficiently explain variance in the other, motivating joint training on CAESAR and SizeUSA.The explained-variance experiment compares subjects from each dataset under shape spaces trained on the other.
  • 4.3 Extended Training Data: Training on CAESAR and SizeUSA improves generalization for both models, while STAR remains uniformly better than SMPL.The comparison is performed on the held-out test set as a function of the number of shape coefficients.
  • 4.3 Extended Training Data: Conditioning STAR’s pose correctives on both pose and body shape yields a modest but consistent further improvement in held-out generalization accuracy.The shape-dependent variant is evaluated after retraining on the combined datasets.

5 Discussion

STAR’s discussion emphasizes that learned spatial locality provides compactness, regularization, and more physically localized gradient propagation than SMPL’s dense formulation.

  • 5 Discussion: 93 pose corrective blend shapes make STAR 80% smaller than SMPL, while it uniformly performs better when trained on the same data.The discussion reports 93 pose corrective blend shapes for STAR versus 207 for SMPL.
  • 5 Discussion: Learning sparse support regions from data avoids the worse accuracy reported for SMPL’s hand-enforced sparse formulation.STAR learns which vertices are relevant to each joint rather than restricting them using fixed skinning weights.
  • 5 Discussion: Geodesic-distance initialization, local neighboring-joint pose inputs, and an L1 penalty bias activation weights toward sparse, local supports.These implementation choices jointly encourage each joint to influence nearby vertices.
  • 5 Discussion: Sparse correctives regularize STAR against spurious long-range correlations and make derivatives with respect to geodesically distant joints equal to 0.Each vertex is influenced by only a limited number of joints in the kinematic tree.

6 Conclusion

STAR combines sparse local pose correctives, expanded shape training, and shape-dependent deformation modeling to improve compactness, realism, and generalization relative to SMPL.

  • 6 Conclusion: STAR uses learned nonlinear joint activations to estimate sparse mesh support regions alongside linear corrective pose blend shapes.The model learns both corrective pose blend shapes and the activation regions influenced by each joint.
  • 6 Conclusion: STAR has only 20% of SMPL’s pose corrective parameters while addressing spurious long-range correlations and non-local deformations.The conclusion links compactness and localization to the model’s learned sparse formulation.
  • 6 Conclusion: Additional training scans improve model coverage because human bodies are more varied than the CAESAR dataset.The conclusion states that more training scans result in a better model.
  • 6 Conclusion: Making pose-corrective blend shapes depend on body shape produces more realistic deformations.This extends pose correction beyond pose-only conditioning.
Loading 2008.08535v1…