Source-linked AI summary
Building Statistical Shape Spaces for 3D Human Modeling
Leonid Pishchulin, Stefanie Wuhrer, Thomas Helten, Christian Theobalt, Bernt Schiele
TL;DR
Existing public statistical human-body models are limited by small training datasets and the effort required to process and align large scan collections. The paper rebuilds an efficient S-SCAPE representation from a large commercial scan database, develops preprocessing best practices, and releases models, scans, and code. The resulting spaces show better generalization and advantages over a state-of-the-art shape space for reconstructing 3D bodies from partial depth data.
Problem
Public statistical human-body models often rely on small datasets with limited shape variation, while processing and aligning large scan databases requires substantial effort.
Method
The paper constructs efficient S-SCAPE shape spaces from a large 3D body-scan database and evaluates preprocessing steps for producing high-quality models.
Results
The learned spaces achieve significantly better generalization and show advantages over a state-of-the-art shape space for partial-depth 3D body reconstruction.
Takeaways & Limitations
The released shape spaces, registered scans, raw scans, and preprocessing resources provide publicly available components for future human-body modeling.
Takeaways & Limitations
The reported preprocessing findings are best practices and specific solutions rather than entirely new methods.
Abstract
from arXiv · showhide
Statistical models of 3D human shape and pose learned from scan databases have developed into valuable tools to solve a variety of vision and graphics problems. Unfortunately, most publicly available models are of limited expressiveness as they were learned on very small databases that hardly reflect the true variety in human body shapes. In this paper, we contribute by rebuilding a widely used statistical body representation from the largest commercially available scan database, and making the resulting model available to the community (visit http://humanshape.mpi-inf.mpg.de). As preprocessing several thousand scans for learning the model is a challenge in itself, we contribute by developing robust best practice solutions for scan alignment that quantitatively lead to the best learned models. We make implementations of these preprocessing steps also publicly available. We extensively evaluate the improved accuracy and generality of our new model, and show its improved performance for human body reconstruction from sparse input data.
1. Introduction
The paper builds a publicly available, efficient S-SCAPE model from the largest commercially available 3D scan dataset and develops preprocessing practices for learning it. Evaluation reports better model quality, generalization, and reconstruction from partial depth data.
- Motivation: Publicly available human body shape models were often learned from small datasets with limited shape variation, partly because large-scale scan processing and alignment are difficult.The paper frames representative data and preprocessing effort as central barriers to expressive statistical shape spaces.
- Representation: S-SCAPE models identity variation with PCA and pose variation with low-dimensional skeleton-based surface skinning, making the representation versatile and computationally efficient.The representation operates directly on vertex coordinates rather than using the more computationally demanding SCAPE formulation.
- Preprocessing: Fitting an initial shape and posture model before non-rigid deformation improves registration, while multiple dataset passes improve initialization and overall fitting accuracy.These preprocessing choices are presented as best practices for automatically processing large scan databases.
- Evaluation: Posture normalization before shape-space learning leads to much better generalization, and the resulting models outperform a state-of-the-art shape space for reconstruction from partial depth observations.The evaluation compares both preprocessing methods and the learned shape spaces.
- Contributions: The authors construct S-SCAPE spaces from the largest commercially available 3D scan database and release the resulting models and preprocessing code.The model is based on a simplified SCAPE variant and is made available for research use.
- Applications: The approach targets applications including body-shape prediction, image and video modification, people-detection training data, and clothing simulation.The paper positions statistical human shape spaces as useful across computer vision, graphics, and ergonomic design.
2. Statistical modeling with SCAPE
SCAPE models human shape and posture by deforming an aligned template mesh, separating shape variation from posture transformations. The simplified S-SCAPE space trades posture-specific detail for faster reconstruction and efficient sampling.
- Original SCAPE model: SCAPE learns deformations by fitting a template mesh with N vertices to rigidly aligned scans in a database.Each fitted scan becomes an instance mesh Mi for model learning.
- Original SCAPE model: SCAPE represents posture through transformations Rm,i and Qm,i, with neighboring triangle deformations smoothed and transformation dimensionality reduced using a kinematic chain.Qm,i captures shape and non-rigid posture-dependent surface effects such as muscle bulging.
- Original SCAPE model: Shape deformations Cm,i are represented in a low-dimensional PCA space learned from training meshes captured in a standard posture.The PCA model uses parameter vector ϕ ∈ R^D, matrix C ∈ R^(4N×D), and mean body shape M.
- Original SCAPE model: Because SCAPE does not explicitly encode vertex positions, reconstructing the mesh surface requires solving a computationally expensive optimization problem.This computational overhead can be prohibitive where speed matters more than overall reconstruction quality.
- Simplified SCAPE (S-SCAPE) space: S-SCAPE sacrifices posture-specific details such as muscle bulging but provides much faster reconstruction, supporting applications that prioritize speed and repeated sampling.The authors argue that speed can be more important than overall reconstruction quality in many applications.
3. Data processing
The paper develops preprocessing procedures for aligning human scans, combining non-rigid template fitting, initialization, bootstrapping, and posture normalization to build improved S-SCAPE spaces.
- 3. Data processing: The preprocessing pipeline establishes scan correspondences through non-rigid template fitting, initialization strategies, human-in-the-loop bootstrapping, and posture normalization.Implementations of these preprocessing steps are made publicly available.
- 3.1. Non-rigid template fitting: Non-rigid fitting minimizes combined data, smoothness, and landmark errors by optimizing per-vertex affine transformations that align a template with a scan.Data terms use compatible nearest points, smoothness terms regularize neighboring transformations, and landmarks help avoid local minima.
- 3.1. Non-rigid template fitting: Only 64 landmarks can sufficiently bring the deformed template close to the scan and help avoid local convergence.The landmarks correspond to known anthropometric markers placed on CAESAR subjects before scanning.
- 3.2. Initialization: The two-step initialization first fits posture using landmarks, then jointly optimizes shape and posture parameters while recomputing nearest correspondences.Good initialization is required to find a useful local minimum.
- 3.3. Bootstrapping: Human-in-the-loop bootstrapping repeatedly inspects registered scans, discards poor fits, learns a new S-SCAPE space, and uses it to initialize the next pass.The process repeats until nearly all registered scans pass visual inspection because low average error may hide inaccurate localized regions.
- 3.4. Posture normalization: Posture normalization is applied because slight posture differences can cause PCA to learn posture-related variation as global shape variation.The paper examines two posture-normalization approaches based on localized Laplacian coordinates and skeleton-based deformation.
4. Evaluation of template fitting
The template-fitting evaluation uses 4,308 CAESAR scans and measures correspondence quality through thresholded vertex accuracy and mean per-vertex fitting error. Shape-space initialization, relaxed weights, and bootstrapping improve fitting, especially in difficult or high-variability regions.
- Dataset and setup: 4,308 CAESAR scans with 64 usable landmarks are evaluated after excluding scans without landmarks or with corruption.The template has 6,449 vertices, and each scan is subsampled to 19,347 vertices before fitting.
- 4.2. Quality measure: Fitting accuracy is measured by nearest-neighbor correspondence, counting vertices below an error threshold and reporting average vertex-to-vertex distance.A correspondence is accepted when distance is under 50 mm and normal deviation is under 60°.
- 4.3. Initialization: NRD performs well in the 0–10 mm range, while S-SCAPE + NRD improves fitting in the 10–20 mm high-error range.S-SCAPE + NRD also produces smoother deformations and lower arm errors than NRD alone.
- 4.3. Initialization: S-SCAPE alone has substantially higher fitting error, particularly around the arms, belly, and chest.NRD alone fits most vertices well but struggles with arms because template and scan postures differ.
- 4.4. NRD parameters: Relaxing smoothness and landmark weights improves fitting over constant weights, with β := 0.25β and γ := 0.25γ giving slightly better results and faster convergence.Constant weights enforce higher localized rigidity, whereas relaxation lets the data term fit the scan more accurately.
- 4.5. Bootstrapping: Bootstrapping enables learning from an increasing number of scans by using each learned S-SCAPE space to initialize the next fitting pass.The authors use the results after five passes to learn the subsequent S-SCAPE space.
5. Evaluation of statistical shape space
The statistical shape-space evaluation examines generalization and specificity across bootstrapping rounds, training-set sizes, and posture-normalization choices. Larger and more representative training sets improve quality, while posture normalization substantially improves both measures.
- Quality measures: Generalization uses leave-one-out reconstruction and mean vertex-to-vertex error, while specificity measures nearest-neighbor distance from random generated shapes to training samples.The study generates 10,000 random samples for specificity evaluation.
- Evaluation protocol: Comparisons across shape spaces use training samples after four bootstrapping iterations as ground truth, although changing geometry across rounds limits direct comparability.The authors explicitly note that geometry changes during bootstrapping make results across spaces incomparable without this reference.
- Bootstrapping: Generalization error is already low after one bootstrapping iteration, while later rounds have little influence on generalization or specificity.Four iterations yield slightly lower specificity error for a small number of shape parameters.
- Number of training samples: Increasing training samples consistently improves both measures, with the largest error reduction occurring from 50 to 100 samples.Increasing from 1,000 to 4,307 samples only slightly reduces both errors, showing that high-quality spaces can be learned from 1,000 samples.
- Number of training samples: A space trained on 100 samples generalizes well but has poor generative quality, whereas larger sets improve specificity more strongly.The evaluation compares subsets of 50, 100, 1,000, and 4,307 training samples.
- Posture normalization: Posture normalization significantly improves generalization and specificity, with WSX achieving the best result.The improvement is strongest for generalization with few shape parameters because normalized spaces are more compact.
- PCA components: The unnormalized space retains posture-related deformations in its principal modes, whereas WSX and NH largely factor posture variation out.The normalized spaces show shape changes more directly in their leading PCA components.
6. Human body reconstruction
The paper evaluates its shape spaces for reconstruction from paired front and back depth scans, using laser scans as ground truth. Models trained on CAESAR data outperform the prior space and remain accurate with few shape parameters, while posture normalization helps arm fitting.
- Reconstruction method: The reconstruction method fits shape and posture independently to two sequential depth images, then jointly optimizes shape and posture because the subject is identical but poses may differ.The evaluation uses the protocol of Helten et al. on paired front and back depth scans.
- Evaluation data: The evaluation uses Kinect scans from three males and three females, with high-resolution laser scans providing ground-truth body shape.Fourteen landmarks are manually placed on each depth and laser scan because the dataset lacks the required landmarks.
- Quantitative evaluation: CAESAR-trained S-SCAPE spaces outperform Jain et al.'s space, which was learned from the less representative MPI Human Shape dataset.The comparison varies both the number of shape parameters and the number of training samples.
- Quantitative evaluation: Good fitting accuracy is achieved with as few as 20 shape parameters and remains stable when increasing the number to 50.In contrast, Jain et al.'s space degrades as the number of parameters increases, possibly because noisy depth data induces unrealistic deformations.
- Quantitative evaluation: The CAESAR-trained models perform better even when all models use the same number of training samples, indicating greater shape variability in CAESAR data.The comparison covers models trained with and without posture normalization.
- Quantitative evaluation: NH posture normalization improves fitting to noisy depth data, while the unnormalized model outperforms WSX in most cases.Overall results support large representative training sets combined with posture-processing strategies.
- Qualitative evaluation: Per-vertex visualizations show better fitting around the belly and chest, while posture-normalized models better fit the arms.The reported regional differences are attributed to the larger CAESAR dataset and posture normalization.
7. Conclusion
The paper builds an expressive S-SCAPE body shape space from the largest commercially available 3D scan dataset and identifies preprocessing choices that improve fitting and model quality. The resulting spaces generalize better, outperform a state-of-the-art space for noisy depth reconstruction, and are released with supporting data and code.
- The authors build an expressive 3D body shape space from the largest commercially available scan dataset.They use the CAESAR database to learn the S-SCAPE spaces.
- Fitting an initial shape and posture space before non-rigid deformation considerably improves fitting results.This preprocessing order is identified as a key best practice for scan registration.
- Multiple passes over the dataset improve initialization, increasing overall fitting accuracy and statistical shape-space quality.
- Posture normalization before shape-space learning significantly improves S-SCAPE generalization and specificity.
- The learned spaces outperform the state-of-the-art space in human-body reconstruction from noisy depth data.The comparison is made against the shape space of Jain et al., learned from the largest publicly available dataset.
- The authors release S-SCAPE spaces, registered scans, raw-scan preprocessing, fitting, and evaluation code for public use.They present this release as supporting future development in human body modeling.