Source-linked AI summary
Learning a Shared Shape Space for Multimodal Garment Design
Tuanfeng Y. Wang, Duygu Ceylan, Jovan Popovic, Niloy J. Mitra
TL;DR
Garment design is difficult because it spans sketches, sewing patterns, materials, body shapes, and draped 3D forms, traditionally requiring iterative physical or simulated adjustment. The paper learns a shared latent space linking these modalities so sketches can drive parameter inference, interactive editing, and fold-preserving retargeting without expensive simulation at design time. Evaluations report qualitative and quantitative support for interactive multimodal design across garment and body-shape variations.
Problem
Garment design lacks a unified workflow across sketches, parameterized sewing patterns, body shapes, and 3D draped garments, while conventional design repeatedly relies on expensive simulation or physical draping.
Method
The method jointly trains encoder-decoder networks on synthetic garment simulations to learn a shared latent space linking sketches, garment and body parameters, materials, and draped 3D shapes.
Results
Qualitative and quantitative evaluations against ground-truth test data show favorable performance and interactive garment sketching across usage scenarios.
Takeaways & Limitations
The shared latent space enables multimodal editing, simulation-free draped-shape prediction during design, direct texturing, garment variation, and fold-preserving retargeting to new body shapes.
Takeaways & Limitations
The method does not represent garment shapes outside its trained garment set and currently assumes garments are designed for a fixed target body pose.
Abstract
from arXiv · showhide
Designing real and virtual garments is becoming extremely demanding with rapidly changing fashion trends and increasing need for synthesizing realistic dressed digital humans for various applications. This necessitates creating simple and effective workflows to facilitate authoring sewing patterns customized to garment and target body shapes to achieve desired looks. Traditional workflow involves a trial-and-error procedure wherein a mannequin is draped to judge the resultant folds and the sewing pattern iteratively adjusted until the desired look is achieved. This requires time and experience. Instead, we present a data-driven approach wherein the user directly indicates desired fold patterns simply by sketching while our system estimates corresponding garment and body shape parameters at interactive rates. The recovered parameters can then be further edited and the updated draped garment previewed. Technically, we achieve this via a novel shared shape space that allows the user to seamlessly specify desired characteristics across multimodal input {\em without} requiring to run garment simulation at design time. We evaluate our approach qualitatively via a user study and quantitatively against test datasets, and demonstrate how our system can generate a rich quality of on-body garments targeted for a range of body shapes while achieving desired fold characteristics.
1 INTRODUCTION
Garment design spans sketches, sewing-pattern parameters, materials, body shapes, and 3D draped forms, making conventional workflows complex and iterative. The paper learns a shared latent space connecting these modalities for interactive inference, editing, and retargeting.
- Traditional design repeatedly adjusts sewing-pattern parameters after physical draping or virtual simulation until the desired look is achieved.
- Garment design operates across 2D sketches, sewing patterns and materials, and 3D garment shapes interacting with subject bodies.
- Free-flowing garments are especially challenging because folds depend on hidden fabric interactions and boundary conditions imposed by the underlying body shape.
- The method learns a shared latent space unifying sketches, garment and body parameters, materials, and final draped 3D garments.
- Jointly trained encoder-decoder networks link pairs of representations while operating at a common embedding that can be queried through different modalities.
- The learned space supports sketch-based parameter inference, simulation-free draped-shape prediction, latent garment variation, and fold-preserving retargeting across body shapes.
2 RELATED WORK
Prior garment systems bridge selected design spaces, often requiring existing draped garments, specialized sketch input, or computational capture setups. This work positions multimodal design and fold-aware retargeting as a broader alternative spanning sketches, parameters, and draped shapes.
- Traditional garment modeling converts sketches into sewing patterns, then uses physical draping or physics-based simulation to obtain final garment shapes.
- Earlier methods commonly bridge two design spaces, such as parsing 2D patterns into 3D or generating 3D garments from sketches.
- Li et al.'s system lets users sketch fold types on an existing draped garment and optimizes sewing patterns to produce those folds.
- Image- and RGB-D-based methods infer garments from silhouettes, landmark points, or depth data captured from garments on mannequins or bodies.
- Garment editing research includes bidirectional mappings between 2D patterns and 3D draped forms, as well as methods for mapping 3D edits to plausible patterns.
- Retargeting methods transfer garment styles across differently proportioned bodies, using constrained optimization or data-driven approaches to avoid expensive simulation.
- The proposed retargeting operates in a joint embedding and transfers garments across 100s of shape variations while preserving desired fold characteristics.
3 APPROACH
The approach links sketches, garment and body parameters, and draped garment shapes through a shared latent space, enabling multimodal editing and retargeting without repeatedly relying on direct cross-domain mappings.
- Overview: The system represents sketches, garment and body parameters, and draped garment meshes as complementary design domains for multimodal garment creation.Sketches express visual fold characteristics, parameters specify garment edits and body shapes, and draped meshes provide previews.
- Overview: Directly mapping sketches to draped garments can overfit and does not expose editable garment or body parameters.The shared-space formulation addresses both generalization and parameter access concerns.
- Joint latent space: A shared latent space jointly links sketches, sewing-pattern and material parameters, body shapes, and final draped garments through multiple encoder-decoder networks.The networks specialize in pairwise mappings while operating at a common embedding.
- Parameter space: The dataset models shirts, skirts, and kimonos using garment dimensions, material properties, and SMPL body-shape parameters.The combined garment and body parameter space is P = (G, B).
- Data generation: The training data combines sampled garment and body parameters, simulated draped meshes, PCA garment representations, and rendered sketch descriptors.For each garment type, 8000 sampled parameter combinations are simulated and represented using 200 PCA basis vectors.
- Joint latent space: The joint loss penalizes errors in sketch-to-parameter prediction, draped-shape prediction, and parameter reconstruction, with consistency terms improving test prediction quality.The reported weighting relation is ω1 = ω2 = 40ω3 = 40ω4.
- Garment retargeting: Retargeting learns a garment embedding whose distances resemble HOG-sketch distances, then optimizes garment parameters for a new body shape.This preserves the intended draped style while mapping to valid sewing-pattern parameters.
4 IMPLEMENTATION DETAILS
The implementation generates simulated garment data, converts draped meshes into consistent representations and sketches, and trains neural networks for shared-space prediction and retargeting.
- Data generation: Synthetic data generation samples garment parameters, triangulates sewing-pattern silhouettes, and simulates each garment on varying body shapes.The process covers shirts, skirts, and kimonos and uses particle-based FleX cloth simulation.
- Data generation: The mannequin-based simulation shrinks the rest-pose body to its skeleton, drapes the sewing-pattern mesh, stitches and reinflates the body, then animates it into the target pose.The converged drape is transformed into the final simulation result.
- Data generation: The pipeline remeshes simulated garments to a consistent topology before applying PCA to represent draped shapes.Consistent topology is required for PCA across samples with varying mesh topology and face count.
- Sketch processing: Rendered sketches are augmented by line removal, curve smoothing, and Gaussian blur, then cropped to 224 × 224 patches and encoded as 2208-dimensional DenseNet features.Suggestive Contours produce the sketch renderings from simulated garments.
- Network training: The shared-space networks are trained for 20000 epochs using stochastic gradient descent with learning rate 0.1 and batch size 64.These settings apply to the encoder-decoder training described in the implementation.
- Retargeting: The retargeting Siamese network maps 200-dimensional PCA vectors to a 100-dimensional embedding using six linear blocks.It is trained for 20000 epochs with learning rate 0.03 and batch size 64.
5 EVALUATION
The evaluation tests multimodal garment prediction, generalization, latent-space behavior, retargeting, interactive editing, and perceptual agreement with sketches. Results show plausible folds, improved test-time generalization from joint training, interactive-rate generation, and successful retargeting across body shapes.
- Real-image evaluation: The method generates draped garments that closely follow six real reference images and the users’ corresponding input sketches.These examples test the generality of the approach on real images.
- Latent-space evaluation: Jointly training additional mappings regularizes the shared latent space and improves test-time generalization over a direct single encoder-decoder mapping.The direct mapping overfits the training data and fails to generalize, whereas the jointly trained model performs better during testing.
- Comparison and retargeting: The predicted draped garment has similar quality to a computation-heavy comparison method while being generated at interactive rates.The comparison uses an example from Yang et al. (2016).
- Comparison and retargeting: Retargeting optimization identifies garment parameters that preserve the original garment style on a different body shape.Simply draping the same garment on different body shapes does not preserve its style.
- Interactive design: The interface supports interactive editing of sketches, garment and body parameters, and UV coordinates while visualizing the corresponding draped garment.The editing workflow spans the paper’s linked design modalities.
- User study: Users are accurate at pairing input sketches with predicted garments, with slightly higher accuracy when simulation results replace network-predicted shapes.Accuracy decreases when the input sketches are very similar, while the simulation-based result validates perceptual capture of fold characteristics.
6 CONCLUSION
The paper presents a joint latent space for sketches, garment and body specifications, and draped garment shapes, enabling multimodal design and body-shape retargeting. Its scope is limited by training coverage, fixed-pose assumptions, and garment-type-specific latent spaces.
- Conclusion: The framework links 2D sketches, sewing patterns, material properties, body specifications, and 3D draped garment shapes in a joint latent space.The paper evaluates the framework quantitatively and qualitatively across different usage scenarios.
- Conclusion: The latent space enables iterative garment creation without expensive physical simulations at design time and supports retargeting while preserving design intent.Retargeting is formulated as an optimization over different human body shapes.
- Limitations: The method cannot represent garment shapes absent from the training set and currently assumes garments are designed for a fixed target body pose.The authors identify pose variation as future work and note the challenge of selecting key poses.
- Limitations: The latent spaces are specialized to shirt, skirt, and kimono garment types rather than unified across garment categories.Unifying these spaces is difficult because transitions involve complex discrete and continuous changes.