Source-linked AI summary
GarmentWeaver: Schema-Aware Structured Synthesis for Multimodal Sewing Patterns
Yinwen Lu, Weihao Luo, Yueqi Zhong
TL;DR
Multimodal sewing-pattern generation seeks executable patterns from sketches and text, but flat sequence representations entangle structure with parameters and can impair local accuracy and simulation compatibility. GarmentWeaver constructs compact schema-aware targets, predicts structure before parameters with a pretrained vision-language model, and applies feasibility-aware regularization. It produces more accurate and executable patterns than strong baselines, with improved simulation results, while remaining limited by the garment components supported by GarmentCode-style representations.
Problem
Existing multimodal sewing-pattern methods often use flat long sequences that entangle garment structure with detailed parameters, causing redundant components, inaccurate local details, and poor simulation compatibility.
Method
GarmentWeaver activates garment-relevant schema branches, predicts a structure template before compatible parameters with a pretrained vision-language model, and applies feasibility-aware regularization.
Results
GarmentWeaver achieves a 98.3% simulation success rate, surpassing AIpparel (73.9%), Design2Garment (87.1%), SewingLDM (61.4%), and ChatGarment (97.9%).
Takeaways & Limitations
Schema-aware structured generation yields more accurate, coherent, structurally valid, and simulation-compatible sewing patterns than strong baselines.
Takeaways & Limitations
GarmentWeaver is restricted to garment components and parametric structures supported by GarmentCode-style representations, limiting finer details such as pockets and belt loops.
Abstract
from arXiv · showhide
Multimodal Sewing pattern generation aims to infer executable sewing patterns from design cues such as sketches and textual descriptions. As an interpretable and simulation-compatible representation, sewing patterns are particularly valuable for digital garment creation. However, existing methods often model garment specifications as flat long sequences, which entangles garment structure with detailed parameters and leads to redundant components, inaccurate local details, and poor simulation compatibility. In this paper, we present GarmentWeaver, a schema-aware framework for multimodal Sewing pattern generation. GarmentWeaver constructs compact hierarchical targets by activating garment-relevant structural branches and predicts executable Sewing patterns in a structured manner. Specifically, we introduce a schema-aware target construction strategy, build the generator on top of a pretrained vision-language model for multimodal garment understanding, and impose feasibility-aware regularization to encourage structurally valid and simulation-compatible outputs. Extensive experiments show that GarmentWeaver produces more accurate and more executable sewing patterns than strong baselines, while also yielding better simulation results. These findings demonstrate the effectiveness of schema-aware structured generation for reliable multimodal Sewing pattern prediction.
1 Introduction
GarmentWeaver addresses the difficulty of generating structurally valid, detailed, and executable sewing patterns from multimodal inputs. It replaces entangled flat-sequence prediction with schema-aware structured generation and feasibility constraints.
- Multimodal sewing-pattern generation must combine sketch cues about silhouette and contour with textual garment semantics and design intent.
- Direct full-sequence generation couples structural decisions with parameter prediction, producing redundant branches, mismatched parts, and unstable details that can degrade simulation.
- GarmentWeaver first predicts a garment structure template and then fills its corresponding design parameters using compact hierarchical supervision.
- Schema-aware target construction activates garment-relevant branches, while feasibility-aware regularization suppresses invalid branches and out-of-range parameters.
- Experiments report improved garment-category accuracy, finer structural details, panel reconstruction accuracy, structural correctness, and simulation success over representative baselines.
2 Related Work
Prior work spans garment geometry reconstruction, sewing-pattern estimation, executable garment representations, and multimodal generation. GarmentWeaver differs by predicting garment structure first and generating parameters only within valid schema branches.
- Garment geometry methods primarily reconstruct garment shape or draping from visual data and body-aware priors rather than executable sewing patterns.
- Sewing-pattern estimation methods connect garment surfaces or 3D geometries to underlying pattern structures.
- Executable representations such as GarmentCode provide programmatic parametric sewing-pattern forms, while GarmentCodeData supplies paired garment and pattern data.
- GarmentWeaver predicts garment structure before detailed parameters, addressing the limitation that generic structured sequences do not explicitly constrain parameters to structure-valid branches.
- Recent multimodal methods use text, sketches, vision-language models, tokenization, language-friendly garment code, or latent diffusion for structured garment generation.
- Its pipeline constructs compact hierarchical targets, performs two-stage VLM prediction, converts outputs into GarmentCode, and simulates the resulting sewing patterns.
3 Method
GarmentWeaver generates sewing patterns by separating garment structure from detailed parameters and activating only structure-valid schema branches. Its two-stage generation, pretrained multimodal backbone, and feasibility-aware regularization target compact, executable outputs suitable for simulation.
- Problem formulation: GarmentWeaver decomposes multimodal sewing-pattern generation into structure prediction followed by parameter prediction, reducing ambiguity from flat long-sequence modeling.The factorization separates structural component C from parameter component P and prevents probability mass on structurally invalid token combinations.
- Schema-aware dynamic encoding: Schema-aware encoding masks inactive branches, preserving only structure-valid parameter fields in the target representation.The activation mask is applied element-wise to the full parameter vector, removing branches that are invalid under the inferred structure.
- Schema-aware dynamic encoding: The method constructs a coarse-to-fine target pair: a structure template with <FILL> placeholders followed by a filled representation with compatible parameter values.This construction reduces output redundancy and aligns supervision with the hierarchical nature of garment specifications.
- Two-stage structured generation: A pretrained LLaVA-style vision-language model aligns sketch, text, and body cues, while autoregressive generation predicts the template first and parameters second.Stage II conditions on both the multimodal input and the generated structure template.
- Feasibility-aware regularization: Feasibility-aware regularization constrains completed representations to valid structural space by penalizing inactive-branch predictions and out-of-range active-branch values.The final loss combines template and filling objectives with invalid-branch and range penalties; λ_inv = 0.1 and λ_range = 0.05.
- Results: Qualitative and quantitative experiments report more accurate categories, finer structural details, structurally valid patterns, and improved simulation outcomes than representative baselines.The reported comparisons cover panel reconstruction accuracy, structural correctness, and simulation success rate.
4 Experiment
GarmentWeaver is evaluated against representative baselines across multimodal and text-only generation, compactness, ablations, and simulation. It achieves stronger structural and geometric accuracy, reduces target redundancy, and produces more simulation-compatible patterns.
- Qualitative comparison: GarmentWeaver better preserves garment categories, fine-grained structural details, and simulation stability than representative baselines in sketch-conditioned generation.It preserves sleeve shape, bodice structure, and skirt proportion while supporting stable downstream simulation.
- Qualitative comparison: In text-only generation, GarmentWeaver more accurately recovers distinctive components such as V-necks, square necklines, hoods, and collars than SewingLDM.Its pattern decompositions better match described garment structures and produce more complete part layouts.
- Quantitative comparison: 0.869 Panel IoU, 1.130 Trans L2, and 89.52% #Edge accuracy are reported for GarmentWeaver, which leads the compared methods on these metrics.These results correspond to panel reconstruction, spatial alignment, and fine-grained structural detail recovery.
- Compactness analysis: 49.53% reduction in target size is achieved by pruning inactive branches, reducing average target length from 118.24 to 59.75.The reduction is structure-dependent, with bottom-only garments showing the largest compression ratio.
- Ablation study: Ablations show that sketch guidance improves shape fidelity, feasibility-aware regularization suppresses redundant structures, and two-stage generation supports regular, simulation-compatible patterns.Direct full-sequence prediction produces distorted patterns, with some results failing simulation; increasing the backbone from LLaVA-7B to LLaVA-13B gives no clear qualitative improvement.
- Simulation evaluation: 98.3% simulation success rate is achieved by GarmentWeaver, compared with 73.9% for AIpparel, 87.1% for Design2Garment, 61.4% for SewingLDM, and 97.9% for ChatGarment.The evaluation randomly samples 1,000 results and uses the GarmentCode simulator.
5 Application
GarmentWeaver supports instruction-based editing of sketch-generated sewing patterns. It localizes updates to instruction-relevant components while preserving unrelated structures, enabling sequential garment refinement.
- Instruction-based editing: Schema-aware structure localizes edits so affected components or parameters change while unrelated garment structures remain unchanged.The resulting process supports controllable and localized refinement for interactive garment design.
- Instruction-based editing: Natural-language instructions can add a collar, change long sleeves to short sleeves, and shorten skirt length in generated sewing patterns.Editing starts from an initial pattern generated from the design input.
6 Limitations and Future Work
GarmentWeaver is limited by the garment representation underlying its schema, especially for localized accessory-like details. It can recover major garment silhouettes and main structural parts, but may fail to reconstruct components such as pockets and belt loops.
- Localized accessory-like details such as pockets may be omitted while the overall pants, skirt, or other garment shape is recovered.Figure 7 specifically shows failures to reproduce pocket structures despite their presence in reference inputs.
- GarmentWeaver relies on GarmentCode-style specifications and is restricted to the components and parametric structures supported by that representation.
- Future work will enrich GarmentCode's bottom-level structure with richer definitions for pockets and other localized attachments.
7 Conclusion
GarmentWeaver models multimodal sewing-pattern prediction structurally rather than as a flat sequence. Its structured targets, generation process, and feasibility-aware regularization improve output accuracy, coherence, structural validity, and simulation compatibility over strong baselines.
- GarmentWeaver is a schema-aware framework for multimodal sewing pattern generation.
- The method reduces ambiguity in flat-sequence generation by modeling garment prediction in a more structured way.
- Schema-aware target construction, structured generation, and feasibility-aware regularization together produce more accurate, coherent, and simulation-compatible sewing patterns.