Source-linked AI summary
IMAGDressing-v1: Customizable Virtual Dressing
Fei Shen, Xin Jiang, Xin He, Hu Ye, Cong Wang, Xiaoyu Du, Zechao Li, Jinhui Tang
TL;DR
Existing virtual try-on systems do not give merchants comprehensive, flexible control over garment displays, faces, poses, and scenes. The paper defines virtual dressing and introduces IMAGDressing-v1 with garment-specific feature extraction and hybrid attention, supported by plugins and the IGPair dataset. Extensive experiments report state-of-the-art human image synthesis under varied controlled conditions.
Problem
Existing virtual try-on technologies focus on fixed human conditions and lack flexible control for merchants to showcase garments comprehensively.
Method
The paper defines virtual dressing and develops IMAGDressing-v1 with a garment UNet, hybrid attention, plugin support, and the IGPair dataset.
Results
Extensive experiments report state-of-the-art human image synthesis performance under various controlled conditions.
Takeaways & Limitations
Virtual dressing enables editable garment-centered human images with text-controlled scenes and additional diversity and controllability from community plugins.
Abstract
from arXiv · showhide
Latest advances have achieved realistic virtual try-on (VTON) through localized garment inpainting using latent diffusion models, significantly enhancing consumers' online shopping experience. However, existing VTON technologies neglect the need for merchants to showcase garments comprehensively, including flexible control over garments, optional faces, poses, and scenes. To address this issue, we define a virtual dressing (VD) task focused on generating freely editable human images with fixed garments and optional conditions. Meanwhile, we design a comprehensive affinity metric index (CAMI) to evaluate the consistency between generated images and reference garments. Then, we propose IMAGDressing-v1, which incorporates a garment UNet that captures semantic features from CLIP and texture features from VAE. We present a hybrid attention module, including a frozen self-attention and a trainable cross-attention, to integrate garment features from the garment UNet into a frozen denoising UNet, ensuring users can control different scenes through text. IMAGDressing-v1 can be combined with other extension plugins, such as ControlNet and IP-Adapter, to enhance the diversity and controllability of generated images. Furthermore, to address the lack of data, we release the interactive garment pairing (IGPair) dataset, containing over 300,000 pairs of clothing and dressed images, and establish a standard pipeline for data assembly. Extensive experiments demonstrate that our IMAGDressing-v1 achieves state-of-the-art human image synthesis performance under various controlled conditions. The code and model will be available at https://github.com/muzishen/IMAGDressing.
Introduction
The paper reframes garment presentation as virtual dressing, enabling editable human images with fixed clothing and optional conditions beyond consumer-focused virtual try-on. It introduces CAMI, IMAGDressing-v1, extensible controls, and the IGPair dataset to support this task.
- Introduction: Virtual dressing generates freely editable human images with fixed garments and optional faces, poses, and descriptive texts for comprehensive merchant displays.Unlike virtual try-on, it emphasizes flexible clothing presentation for e-commerce and entertainment.
- Introduction: IMAGDressing-v1 uses a garment UNet for fine-grained clothing features and hybrid attention to balance garment conditioning with text-prompt scene control.The garment UNet captures semantic and texture features, while hybrid attention integrates them into the denoising UNet.
- Introduction: CAMI evaluates consistency between generated images and reference garments in the virtual dressing task.
- Introduction: IMAGDressing-v1 supports ControlNet and IP-Adapter extensions to enhance generated-image diversity and controllability.
- Introduction: The IGPair dataset provides more than 300,000 clothing-and-dressed-image pairs to address data scarcity for virtual dressing research.
Related Work
Prior work developed GAN- and diffusion-based virtual try-on systems, while conditional-control modules extend text-to-image diffusion with structured inputs. Dataset comparisons contextualize the data resources used for these systems.
- Related Work: Early virtual try-on methods warped clothing and used GAN generators to merge it onto human models, but GAN training could be unstable.
- Related Work: Recent virtual try-on methods use pretrained diffusion models, including explicit clothing warping, parallel UNets, or diffusion-based garment integration.
- Related Work: Table 1 compares IGPair with widely used datasets, situating dataset resources within virtual dressing research.
- Related Work: ControlNet and T2I Adapter add conditional encoders for inputs such as edges, depth, and human poses, addressing limits of natural-language control in text-to-image diffusion.
IGPair Dataset
IGPair is a large, publicly available garment-pairing dataset designed for virtual dressing, combining clothing images with diverse dressed-model views and rich annotations.
- Dataset scope: IGPair includes high-resolution garment and model images spanning diverse scenes and styles, with textual descriptions for virtual dressing.The dataset is described as the only publicly available resource combining textual descriptions, diverse scenes, varied styles, and resolution above 2K × 2K.
- Dataset examples: Figure 2 illustrates paired garments and dressed people alongside pose keypoints, dense poses, and human body segmentation masks.These examples show the annotation types available for dataset samples.
- Annotations: The dataset enriches each model image with pose keypoints, dense poses, body-part and clothing segmentation masks, and generated captions.OpenPose, DensePose, SCHP, and multiple vision-language models provide these annotations.
Methodology
IMAGDressing-v1 combines a garment UNet with a frozen denoising UNet to preserve garment semantics and textures while enabling text-controlled scene generation. Hybrid attention injects garment features, and external adapters extend control over faces and poses.
- IMAGDressing-v1: The model combines a trainable garment UNet with a frozen denoising UNet to integrate garment features while retaining text-to-image scene generation.The garment UNet extracts semantic and texture features, while the denoising UNet performs latent denoising under text and garment conditions.
- Garment Feature Extraction: The garment UNet extracts semantic information from CLIP and texture features from VAE representations of the input garment.A frozen VAE encoder produces garment latents, while a frozen CLIP image encoder and trainable projection layer provide token embeddings.
- Hybrid Attention: Hybrid attention combines frozen self-attention with trainable garment-conditioned cross-attention to preserve the denoising model’s original capabilities.The shared query uses denoising features, while garment keys and values are projected through trainable layers; λ regulates garment-condition strength in [0, 1.5].
- Training and Inference: The denoising objective trains only the remaining modules while conditioning noise prediction on text and garment features.Inference additionally applies classifier-free guidance using conditional and unconditional noise predictions.
- Extension Plugins: IMAGDressing-v1 can combine with ControlNet and IP-Adapter to support customized face, pose, and other controlled-generation scenarios.Freezing the denoising UNet’s basic modules makes the garment UNet compatible with community adapters.
Implementation Details
The model is trained by fine-tuning a garment UNet on paired IGPair images with fixed optimization and sampling settings.
- Implementation Details: Training uses paired IGPair images at 512 × 640 resolution for 200,000 steps on 10 NVIDIA RTX3090 GPUs with batch size 5.The AdamW optimizer uses a fixed learning rate of 5e-5; inference uses UniPC for 50 sampling steps with guidance scale 7.0.
Main Comparisons
IMAGDressing-v1 is compared with four state-of-the-art methods using quantitative and qualitative evaluations, including controlled conditions and plugin-based applications. The reported results show stronger garment-detail preservation, text alignment, and controllability than competing methods.
- Qualitative Results: The evaluation includes qualitative comparisons across unspecific and specific conditions against BLIP-Diffusion, Versatile Diffusion, IP-Adapter, and MagicClothing.Figure 4 organizes the comparison around multiple controlled-generation settings.
- Quantitative Results: IMAGDressing-v1 outperforms other SOTA methods across all evaluation metrics in the quantitative comparison.The comparison includes BLIP-Diffusion, Versatile Diffusion, IP-Adapter, and MagicClothing.
- Plugin Applications: IMAGDressing-v1 is demonstrated with ControlNet-Inpaint as a plugin-based virtual try-on application.The application masks the garment area to obtain high-fidelity virtual try-on results.
- Qualitative Results: IMAGDressing-v1 preserves fine-grained garment details while adhering to text prompts under unspecific conditions.Competing methods either lose garment textures, garment details, or prompt fidelity.
- Qualitative Results: Under specific face and pose conditions, IMAGDressing-v1 produces more detailed and realistic results than MagicClothing.The comparison also indicates compatibility with community adapters for greater diversity and controllability.
Ablation Studies
The ablations show that the image encoder branch and hybrid attention jointly improve garment-feature capture and image fidelity. Varying garment strength controls the balance between text-driven editing and similarity to the input garment, while ControlNet-Inpaint enables a virtual try-on application.
- Effectiveness of Each Component: The image encoder branch and hybrid attention together improve quantitative results and garment fidelity over the base setting.The base lacks both components, the image encoder branch improves all metrics, and adding hybrid attention further improves results.
- Garment Strength: Increasing garment strength λ makes the generated garment more similar to the input, while smaller λ values favor text-prompt adherence.λ balances original editing and generation capabilities against additional garment features.
- Potential Application: Combining IMAGDressing-v1 with ControlNet-Inpaint and garment-area masking achieves high-fidelity virtual try-on.The result demonstrates a potential application of the model beyond freely controlled virtual dressing.
Conclusion
The paper introduces virtual dressing for editable garment-centered human-image synthesis and proposes IMAGDressing-v1 to support garment fidelity with textual scene control. Experiments report state-of-the-art controlled human-image synthesis, alongside plugin support and a large paired dataset.
- Conclusion: IMAGDressing-v1 achieves state-of-the-art performance in controlled human image synthesis.The conclusion summarizes the method's performance across the controlled conditions studied.
- Conclusion: The method combines a garment UNet and hybrid attention to integrate garment features while enabling scene control through text.It also supports extensions such as ControlNet and IP-Adapter.
- Conclusion: The released IGPair dataset contains over 300,000 clothing-and-dressed-image pairs for data assembly and research.The paper presents the dataset as a resource accompanying the proposed system.
Text Prompt
The text-prompt examples show IMAGDressing-v1 synthesizing person and cartoon images from clothing under different textual prompts. Additional examples include garments with logos and optional faces and poses.
- Person Images: IMAGDressing-v1 synthesizes person images from garments under different text prompts.Figure 10 presents additional examples of clothing-conditioned generation.
- Optional Conditions: The examples include garments with logos together with optional faces and poses.Figure 11 highlights additional controllable conditions for garment-centered synthesis.
- Cartoon Images: The model generates cartoon images conditioned on clothing and different text prompts.Figure 12 extends the examples beyond realistic person images.