MF-VITON: High-Fidelity Mask-Free Virtual Try-On with Minimal Input

1. Introduction

• VITON must align garments across body shapes and poses, preserve fine details, and generate artifact-free images, while precise masks and scarce diverse datasets complicate development.Source: §1 · p. 2
• The challenges include precise mask delineation, predicting mask placement for each body, and generating high-quality, diverse datasets while generalizing across bodies, poses, and garments.Source: §1 · p. 2
• MF-VITON reduces reliance on predefined masks, creates garment-person training data with Mask-based models and text-guided inpainting, and uses OFI training to improve robustness and generalization.The framework also offers plug-and-play integration with ReferenceNet-based VITON models.Source: §1 · p. 2
• Up to a 34.7% reduction in FID and KID scores versus prior Mask-based methods is reported across VITON-HD, DressCode, and VITON-HD In-the-Wild.Source: §1 · p. 2

2. Related Works

• Early pose-guided person image synthesis aligned human poses with target clothing, addressing pose transfer and adaptation to individual body shapes.Source: §2 · p. 3
• Mask-based GAN and diffusion VITON methods rely on precise masks; diffusion approaches improve garment fidelity, but intricate logos and textures remain difficult to preserve across poses and lighting.Source: §2 · p. 3
• Localized-control research and synthetic-data approaches, including BooW-VTON’s mask-free pseudo-data augmentation, motivate generative data creation for robust VITON.MF-VITON targets mask precision, garment detail preservation, and dataset generation while leveraging localized control mechanisms.Source: §2 · p. 3

3. Preliminary

• Stable Diffusion learns a denoising process in a lower-dimensional latent space and uses an encoder-decoder, denoiser, and noise scheduler for image generation.Source: §3 · p. 3
• Conditional flow matching learns data transport trajectories rather than performing standard diffusion noise regression, with the passage reporting 2.1× faster convergence and Lipschitz continuity.Source: §3 · p. 3
• The final image is reconstructed through Euler-Maruyama integration.Source: §3 · p. 3

4. Method

• The two-stage framework first trains Mask-based VITON and generates a dataset, then fine-tunes on Mask-Free data to generalize with minimal input constraints.It is designed to work with ReferenceNet-based models such as IDM-VTON and TED-VITON.Source: §4 · p. 3

4.1. Mask-based & MF-VITON pipeline

• ReferenceNet encodes fine-grained garment details, while the Adapter extracts semantic garment information such as structure, style, and material properties.The Adapter focuses on low-frequency garment attributes to mitigate sensitivity to poses, garment deformations, and environmental factors.Source: §4.1 · pp. 3–4
• TryonNet processes noised latents and selectively uses either Mask-based or Mask-Free conditions and their corresponding text prompts.The Mask-based Context combines person, garment-mask, masked-person, and pose features; the Mask-Free Context combines person-image representations.Source: Figure 2 · p. 4, §4.1 · p. 4
• The Adapter fuses joint and image embeddings with decoupled cross-attention, with λ controlling their relative contributions.The formulation is described as supporting robustness to pose variations, garment structures, and environmental shifts.Source: §4.1 · p. 4

4.2. Mask-Free Dataset Preparation

• If the unmasked condition already shows the target garment, the model can copy it instead of learning garment transformation, so dataset conditions must remain consistent except for clothing.Mask-based TryonNet outputs are used to create Mask-Free conditions while ReferenceNet and the Adapter process the target garment.Source: §4.2 · p. 5

4.3. In-the-Wild Dataset Augmentation

• Flux.1-fill-dev inpaints blank backgrounds, and Mask-Free training pairs combine the resulting background with a condition image showing different clothing.The paired images preserve the same background while varying garments, adding diverse In-the-Wild scenarios.Source: §4.3 · p. 5

4.4. Output-for-Input Training Strategy

• The OFI strategy uses Mask-based VITON outputs as Mask-Free training inputs, aiming to avoid interference from low-quality Mask-based outputs.The authors report that errors in synthesized inputs do not significantly degrade performance and that the strategy yields more natural, artifact-free results.Source: §4.4 · p. 5

5. Experiment

• The evaluation combines quantitative and qualitative analyses of mask-related artifacts, garment alignment, detail preservation, and OFI’s training effect.Figure 3 depicts dataset generation using FLUX.1-Fill-dev for background-filled images and a Mask-based VITON model for garment-swapped samples.Source: §5 · pp. 5–6, Figure 3 · p. 6

5.1. Experiment Setup

• Evaluation uses VITON-HD, DressCode, and an augmented VITON-HD In-the-Wild dataset, with paired fidelity metrics and unpaired realism metrics.VITON-HD and its In-the-Wild variant use 11,647 training pairs and 2,032 test pairs; DressCode uses 1,800 upper-body test pairs. Unpaired evaluation uses FID and KID, while paired evaluation uses SSIM, LPIPS, and CLIP-I.Source: §5.1 · pp. 6–7

5.2. Qualitative Results

• MF-VITON removes segmentation-mask reliance and improves garment blending, texture preservation, identity consistency, and artifact-free clothing integration.The qualitative comparison also identifies over-masking and mask leakage as sources of appearance inconsistencies and artifacts, especially when background variation complicates mask accuracy.Source: §5.2 · pp. 7–8, Figure 4 · p. 7

5.3. Quantitative Results

• MF-enhanced models show lower LPIPS, FID, and KID than non-MF counterparts and existing SOTA methods on VITON-HD and DressCode Upper-body.The reported comparisons also cover VITON-HD In-the-Wild, where background-aware training improves generalization; the supplied passages do not provide table cell values.Source: Table 1 · p. 6, §5.3 · p. 8, Table 2 · p. 8

5.4. Discussion

• OFI improves clothing-region attention alignment and garment placement, while training without it leaves Mask-Free performance constrained by the Mask-based data generator.The discussion reports that models without OFI perform worse than TED-VITON and that OFI improves generalization in challenging cases.Source: §5.4 · p. 8, Table 3 · p. 8, Figure 5 · p. 8

• MF-VITON removes mask-input requirements through a Mask-Free dataset creation strategy and training process.Source: §6 · pp. 8–9
• Across VITON-HD, DressCode, and VITON-HD In-the-Wild, MF-VITON with OFI significantly improves performance while reducing input requirements.Source: §6 · p. 9
• MF-VITON achieves superior visual fidelity with fewer input requirements than previous Mask-based VITON approaches.The authors characterize it as a practical and scalable solution for real-world VITON applications.Source: §6 · p. 9