Source-linked AI summary
SMPLicit: Topology-aware Generative Model for Clothed People
Enric Corona, Albert Pumarola, Guillem Alenyà, Gerard Pons-Moll, Francesc Moreno-Noguer
TL;DR
Existing clothing models struggle to represent varied garment topologies while controlling clothing and body properties. SMPLicit uses a differentiable implicit model conditioned on SMPL pose and shape with interpretable clothing latents, and applies it to scan fitting and image reconstruction. The authors report complex garment recovery, multilayer handling, and outfit editing across these tasks.
Problem
Existing approaches require garment-specific models or template displacements and lack a unified representation for clothing with varied topologies, styles, cuts, and deformations.
Method
SMPLicit is a fully differentiable implicit generative model conditioned on body pose and shape, with low-dimensional latent representations controlling garment style and cut.
Results
SMPLicit fits 3D scans and reconstructs clothed people from images, retrieving complex geometries, handling multiple clothing layers, and supporting garment editing and body re-posing.
Takeaways & Limitations
A unified, controllable clothing representation can support scan fitting, image reconstruction, garment editing, and body re-posing across varied garments.
Takeaways & Limitations
Perfect disentanglement between garment cut and style is not possible, because learned representations can associate attributes such as looseness with clothing categories.
Abstract
from arXiv · showhide
In this paper we introduce SMPLicit, a novel generative model to jointly represent body pose, shape and clothing geometry. In contrast to existing learning-based approaches that require training specific models for each type of garment, SMPLicit can represent in a unified manner different garment topologies (e.g. from sleeveless tops to hoodies and to open jackets), while controlling other properties like the garment size or tightness/looseness. We show our model to be applicable to a large variety of garments including T-shirts, hoodies, jackets, shorts, pants, skirts, shoes and even hair. The representation flexibility of SMPLicit builds upon an implicit model conditioned with the SMPL human body parameters and a learnable latent space which is semantically interpretable and aligned with the clothing attributes. The proposed model is fully differentiable, allowing for its use into larger end-to-end trainable systems. In the experimental section, we demonstrate SMPLicit can be readily used for fitting 3D scans and for 3D reconstruction in images of dressed people. In both cases we are able to go beyond state of the art, by retrieving complex garment geometries, handling situations with multiple clothing layers and providing a tool for easy outfit editing. To stimulate further research in this direction, we will make our code and model publicly available at http://www.iri.upc.edu/people/ecorona/smplicit/.
1. Introduction
SMPLicit addresses the difficulty of modeling clothing with varied garment topologies, styles, cuts, and deformations. It introduces a differentiable, interpretable generative model applicable to garment modeling, scan fitting, and image-based reconstruction.
- Motivation: Existing clothing representations commonly model garments as displacements over SMPL bodies, requiring separate models for substantially different garment types.This limits practical use when upper-body clothing ranges from sleeveless tops to hoodies and open jackets.
- Method: SMPLicit represents clothing under different topologies using a low-dimensional, semantically interpretable control vector.The model controls garment style and cut while conditioning on body pose and shape.
- Method: Within upper- and lower-body garment clusters, one model represents geometrically and topologically different clothes and smoothly interpolates between them.Shoes and hair are modeled as independent categories.
- Method: SMPLicit is fully differentiable and can be integrated into larger end-to-end deep learning systems.Its conditioning variables include human pose, shape, and garment style or cut.
- Applications: The model is applied to fitting 3D scans and reconstructing clothed humans from images, including complex geometries and multiple clothing layers.The reconstruction system also supports garment editing and body re-posing.
2. Related work
Related work spans generative clothing models and image-based reconstruction, but existing approaches often remain constrained by training data, template displacements, garment-specific training, or limited controllability. SMPLicit combines implicit geometry with pose and clothing control to address these constraints.
- Generative cloth models: Clothing models learned from real data may generalize poorly beyond training deformation patterns, while SMPL-based displacement models cannot depart far from the naked-body shape.Capturing enough real data to represent clothing complexity remains an open challenge.
- Generative cloth models: Physics-simulation-based clothing models are constrained by simulation quality and generally rely on template displacement maps that cannot produce different topologies.
- Generative cloth models: Garment models based on alternative topologies, including sewing patterns, typically require training a specific model for each garment type.This requirement penalizes practical use across varied clothing categories.
- Reconstructing clothed humans: Image-based clothed-human reconstruction methods using SMPL offsets are prone to failure for loose garments with large displacements from the body.
- SMPLicit: SMPLicit’s architecture predicts unsigned distance to a cloth iso-surface, then uses dense sampling and Marching Cubes to generate a garment mesh.The network is conditioned on body shape and learned clothing representations.
- Reconstructing clothed humans: Implicit-function approaches retrieve rich geometric detail, but prior resulting surfaces lack simultaneous control over pose and clothing.SMPLicit uses implicit functions while producing multiple body and garment layers with pose and clothing control.
3. SMPLicit
SMPLicit is a differentiable implicit garment model that conditions clothing geometry on body shape and pose while using interpretable latent spaces for garment cut and style. Unlike fixed-vertex approaches, one model represents varied garment topologies and generates meshes through distance-field extraction.
- SMPLicit formulation: SMPLicit-core predicts unsigned distance to a clothing surface at queried points, then reconstructs garments by thresholding the field and applying Marching Cubes.The model operates on T-pose garments and uses a positive iso-surface threshold to extract smooth meshes.
- Input representation: The model conditions garment geometry on body shape through positional encoding and controls cut and style with separate latent representations.Cut represents the body area occluded by clothing, while style captures geometry such as tightness, wrinkles, and collar details.
- Unified garment modeling: A jointly trained model represents multiple garment categories within upper- or lower-body clusters, while separate models are retained for categories whose interpolation is not meaningful.At inference, the cut encoder is discarded and SMPLicit is controlled directly with zcut.
- Latent-space training: Latent regularization and Gaussian noise encourage a normally distributed, smoothly interpolable clothing representation, with noise especially helpful for garment types having little data.Training minimizes the distance-field loss together with latent regularization.
- Pose-dependent deformation: Pose-dependent deformation is learned in a second model that displaces T-pose garment vertices and then applies SMPL skinning to pose the deformed mesh.The deformation model uses body pose and a learnable pose-dependent latent variable.
4. Applications of SMPLicit
SMPLicit supports latent-space garment editing, fitting to 3D scans, and reconstruction from images. Its applications exploit a unified representation of varied clothing topologies and differentiable distance-field optimization.
- Generative properties: PCA reveals interpretable latent directions: the first cut component controls sleeve length, while the second changes garment length or the waist boundary.The authors note that cut and style are not perfectly disentangled because some garment categories correlate with fit.
- Generative properties: SMPLicit smoothly interpolates from T-shirts to prototypes such as open jackets, hoodies, and tops, and retargets garments to different body shapes.These operations demonstrate control over garment style, cut, and body fit.
- 3D scan fitting: Approximately 1 mm of fitting error is achieved by all three models on Sizer scans, while SMPLicit uses one model across varying clothing topologies.The compared cases include hoodies, tank tops, and long or short pants.
- Image reconstruction: For image reconstruction, SMPLicit combines estimated pose, shape, and cloth segmentation to fit layered 3D clothing, using off-surface and on-surface distance constraints.The minimum-distance selection for points projecting to one pixel helps prevent clothing thickening when reconstructing multiple layers.
5. Implementation details
The implementation uses publicly available code and models, with specified latent dimensions, deformation parameters, positional encoding, and an implicit-network architecture.
- The implementation details and supplementary information are provided alongside publicly available code.
- The cloth latent space uses |z|=18 for upper-body garments, pants, skirts, and hair, and |z|=4 for shoes.
- Pose-dependent deformation uses |zθ|=128, with K=500 positional encoding clusters, td=0.1 mm, and dmax=10 mm.
- The implicit network uses three 2-layer MLPs for zcut, zstyle, and Pβ, followed by a 5-layer MLP predicting the target unsigned distance field.
- The image encoder f is a refined pre-trained ResNet-18 using weight normalization instead of batch normalization.
6. Experiments
Experiments evaluate SMPLicit on scan fitting and image-based reconstruction, including varied garments, arbitrary poses, multiple layers, and multi-person occlusions.
- The experiments cover fitting 3D scans and reconstructing dressed people from images.
- Training combines BCNet garments, manually augmented sleeve variants, additional garment models, and USC-HairSalon hair point clouds.The reported augmented totals are 800k T-shirts, 973k pants, and 933k skirts.
- Pose-dependent deformation training uses TailorNet simulations for 200 shirt and pants instances, while other garments use manually generated Blender warps.Shoes are not further deformed.
- 6.2. Fitting SMPLicit to scans of dressed people: For scan fitting, SMPLicit uses one model for short- and long-sleeved garments, achieves results comparable to TailorNet, and performs significantly better than CAPE.The evaluation uses the Sizer dataset and compares against TailorNet and CAPE.
- 6.3. 3D reconstruction of clothed humans: Image reconstruction consistently improves over the compared approaches, especially for poses departing from upright, while recovering complex garments and multiple clothing layers.The comparison is against PIFuHD and Tex2Shape on in-the-wild images.
- 6.3. 3D reconstruction of clothed humans: After reconstruction, SMPLicit supports virtual try-on by changing garment style and reposing the person.
- 6.3. 3D reconstruction of clothed humans: In multi-person MPII images, reconstruction remains feasible under extreme occlusions, but segmentation errors can propagate into incorrect garment predictions.A reported failure incorrectly predicts two intersecting upper-body garments.
7. Conclusion
SMPLicit represents varied garment topologies while controlling style and cut with interpretable parameters, and its differentiable formulation supports reconstruction and geometric editing applications.
- SMPLicit represents different garment topologies while controlling style and cut with a few interpretable parameters.
- Its fully differentiable formulation supports fitting 3D scans and reconstructing clothed humans under multilayered garments and multi-person occlusions.
- The generative model supports geometric content editing, including dynamically changing garment attributes, and opens the door to virtual try-on systems.