BooW-VTON: Boosting In-the-Wild Virtual Try-On via Mask-Free Pseudo Data Training

1. Introduction

• Masking removes person-image content and spatial information, disrupts foreground/background coherence, and can produce artifacts with complex poses and occlusions.Figure 2 contrasts accurate shop masks with less accurate in-the-wild masks under complex interference.Source: §1 · p. 2, Figure 2 · p. 2
• Prior fixes remain limited: TPD’s refined masks target shop scenarios, while PFDM distillation retains constraints of mask-based models that hinder wild-scene generalization.The introduction also notes that mask refinement does not restore information from the masked region.Source: §1 · p. 2
• BooW-VTON fine-tunes a pretrained latent diffusion model using pseudo-wild data, foreground/background augmentation, and a localization loss that suppresses garment features outside try-on areas.The pseudo-data is generated through a refined shop try-on pipeline and augmented with generated backgrounds and foregrounds.Source: §1 · p. 2
• BooW-VTON significantly outperforms the baseline and other state-of-the-art methods across multiple challenging try-on benchmarks.The paper also reports superior performance across diverse try-on situations.Source: §1 · p. 2

2.1. Mask-Based Virtual Try-On

• Mask-based methods use binary masks to define inpainting regions; earlier methods warp garments with pose parsing [4] [13] [20], while diffusion methods align latents before inpainting [5] [19] [21] [24] [37] [40].These approaches lose original image content and rely on pose parsers; TPD [40] reduces mask area, but its predicted masks are unstable.Source: §2.1 · pp. 2–3

2.2. Mask-free Virtual Try-On

• Mask-free methods [10] [14] [18] distill a student from a mask-based teacher, but inherit teacher flaws and have limited wild-scenario generalization.The paper contrasts this with its optimized inference, data augmentation, and loss-function approach.Source: §2.2 · p. 3

2.3. In-the-wild Virtual Try-On

• Wild-scene diffusion methods [5] [19] preserve backgrounds but still alter accessories, skin, and other content inside masked garment regions.StreetVITON [6] also faced challenges matching bodies and clothing with UV mapping.Source: §2.3 · p. 3

3. Preliminary

• A latent diffusion model encodes image x as latent features z0 = E(x) and performs denoising in the autoencoder’s latent space.The autoencoder is denoted E.Source: §3 · p. 3
• The forward diffusion process applies a predefined variance schedule βt to the latent representation over T steps.The passage defines αt := 1 − βt and ᾱt as the cumulative product of αs through step t.Source: §3 · p. 3
• The simplified latent-diffusion training objective uses a U-Net denoiser ϵθ(·) with condition y.The supplied passages identify the objective as the model’s training loss.Source: §3 · p. 3

4. Method

• The pipeline combines pseudo-triplet training, in-the-wild augmentation, and a localization loss to learn try-on regions while preserving non-try-on content.The method is presented as a three-part training pipeline.Source: §4 · p. 3

4.1. Mask-free Try-On Diffusion Model

• The try-on U-Net is trained on pseudo-triplets {P ′, G, P}, using P ′ instead of masked-person images to remove mask reliance.Garment features enter through cross-attention and Reference Net features through self-attention; only the try-on U-Net is unfrozen.Source: §4.1 · pp. 3–4, Figure 3 · p. 4

4.2. High-Quality Pseudo Data Prepare

• A two-stage inference process uses a coarse mask to create Pmid, then a refined mask to generate pseudo-data retaining more non-try-on content.The refined mask combines the garment regions from Pmid and the original person image.Source: §4.2 · p. 4

4.3. In-the-Wild Data Augmentation

• Augmentation stacks background, person, and foreground images from bottom to top, with foreground occlusions confined to non-try-on areas.Backgrounds are synthesized by inpainting, while transparent foreground objects are generated from prompts; random image selection and transformations are also used.Source: §4.3 · pp. 4–5, Figure 4 · p. 5

4.4. Try-On Localization Loss

• Try-on localization loss constrains garment-token attention in non-try-on regions, while LDM loss across all regions supports garment deformation and smooth image consistency.The attention maps are intended to focus on try-on areas rather than spread across the image.Source: §4.4 · p. 5

5. Experiments

• Models are trained separately on VITON-HD and DressCode, then evaluated cross-dataset on the in-the-wild StreetVTON and WildVTON datasets.The experiments also include paired and unpaired testing on VITON-HD and DressCode and unpaired testing on the wild datasets.Source: §5 · p. 5, Table 1 · p. 5

5.1. Qualitative Results

• On VITON-HD complex poses, StreetVTON garment changes, tattoos, foreground details, and untrained anime-style inputs, BooW-VTON preserves person and style details while rendering garments.On DressCode, it preserves muscle details and fabric textures and reproduces skirt features despite the original pants’ fit.Source: §5.1 · pp. 6–7, Figure 6 · p. 6, Figure 7 · p. 6

5.2. Quantitative Results

• BooW-VTON outperforms existing methods across all metrics on VITON-HD and DressCode, and has significant advantages over most methods on StreetVTON and WildVTON.DCI-VTON performs better on StreetVTON KID; the authors say its mask improves non-try-on pixel preservation but yields inferior visual results.Source: Table 2 · p. 7, Table 3 · p. 7, §5.2 · p. 7

5.3. Ablation Study

• In-the-wild data augmentation improves wild-scene try-on and lowers StreetVTON and WildVTON FID and KID, with a slight drop in in-shop performance.Adding try-on localization loss improves attention focus on try-on regions and reduces visual artifacts while better preserving foreground details.Source: §5.3 · pp. 7–8, Figure 8 · p. 8

5.4. Application

• Adjusting prompts controls try-on style for specified garments, enabling varied renderings useful for fashion design.The method also produces realistic full-body try-on results without full-body try-on training.Source: §5.4 · p. 8, Figure 9 · p. 8

• Missing lower-garment reference can make the lower part of a person wearing a dress randomly generated when trying on a T-shirt.The resulting lower outfit may not align with the desired outfit style.Source: §6 · p. 8
• A similar lack of user control also arises when trying on lower garments.Source: §6 · p. 8
• Matching upper and lower garments or accessories remains a key challenge for enabling users to achieve desired outfits.Source: §6 · p. 8

• BooW-VTON uses person-image pairs differing only in garment to train a mask-free try-on model and avoid damage from try-on masks.Source: §7 · p. 8
• In-the-wild data augmentation and try-on localization loss further boost performance in real-world scenarios where accurate try-on areas are difficult to capture.Source: §7 · p. 8
• The model surpasses existing methods across diverse try-on situations in qualitative and quantitative experiments.Source: §7 · p. 8

1.1. High Quality Pseudo Data Details

• Optimizing masks preserves more information unrelated to try-on, reducing mask-based methods’ negative impact on pseudo-data quality.The coarse mask is generated using the same method as the masks in IDM-VTON [5].Source: §1.1 · p. 11

1.2. In-the-Wild Augmentation Details

• Training augmentation independently adds backgrounds with probability 0.5 and foregrounds with probability 0.7, with foregrounds randomly scaled and rotated before overlay.Foregrounds are scaled to 0.25–0.6 times and rotated from -45 to 45 degrees; the person’s placement in a background is random.Source: §1.2 · p. 11
• The augmentation uses 50 foreground images at 1024 × 768 resolution and 40 background images at 1024 × 1024 resolution.Source: §1.2 · p. 11

2.2. Study on the Effectiveness of Attention Layers

• Attention focuses on clothing distribution in early denoising stages and garment details later, with greater activity near the network’s middle stages.The analysis sampled DDIM steps 1, 5, 10, 15, 20, 25, and 30, and attention blocks 1, 25, 35, and 70.Source: §2.2 · p. 11

2.3. Attention Visualization in Ablation Study

• A mask-free approach alone disperses attention across the image, while in-the-wild augmentation shifts it toward try-on areas and helps retain non-try-on content.Source: §2.3 · p. 11
• Try-on localization loss explicitly regularizes attention maps, enabling more accurate identification of try-on regions.Source: §2.3 · p. 11

• VITON-HD has 11,647 training pairs and 2,032 test pairs; DressCode has 29,478 dress, 15,363 top, and 8,951 bottom samples, with 1,800 test pairs per type.Source: §3 · p. 11
• StreetVTON contains 2,089 complex person images, while WildVTON contains 1,224 internet-collected lifestyle portraits with complex foregrounds and backgrounds.Source: §3 · p. 11
• Figures document high-quality pseudo-data generation in VITON-HD and DressCode and augmentation of training samples.Source: Figure 11 · p. 12, Figure 12 · p. 13, Figure 13 · p. 14
• Foreground-image generation prompts cover varied objects, and a separate prompt set targets background images.Source: Table 4 · p. 19, §3 · p. 19, Table 5 · p. 20
• The ablation caption reports that pseudo data improves skin and accessory retention on VITON-HD, while in-the-wild augmentation improves foreground and background retention on WildVTON.Source: Figure 18 · p. 21