PaperlayerView exact revision on arXiv

Source-linked AI summary

DS-VTON: An Enhanced Dual-Scale Coarse-to-Fine Framework for Virtual Try-On

Xianbing Sun, Yan Hong, Jiahui Zhan, Jun Lan, Huijia Zhu, Weiqiang Wang, Liqing Zhang, Jianfu Zhang

arXiv:2506.00908v2cs.CV

TL;DR

Virtual try-on methods still struggle to align garments accurately while preserving fine-grained textures and patterns. DS-VTON uses a mask-free dual-scale coarse-to-fine framework with blend-refine diffusion, and reports state-of-the-art results across VITON-HD and DressCode. The authors also describe the paradigm as scalable, with potential for higher resolutions and broader generation tasks.

  • Problem

    Virtual try-on methods struggle to achieve accurate garment-body alignment and high-fidelity detail reconstruction together.

    Source: §1 · p. 2

  • Method

    DS-VTON uses a mask-free dual-scale framework that aligns structure at low resolution and restores texture at high resolution with blend-refine diffusion.

    Source: §1 · p. 2

  • Results

    DS-VTON demonstrates state-of-the-art performance on VITON-HD and DressCode, with reported improvements in visual quality and robustness over existing methods.

    Source: §1 · p. 3, §5 · p. 10

  • Takeaways & Limitations

    The authors describe the paradigm as inherently scalable, with potential extensions to higher resolutions and generation tasks such as personalized image synthesis.

    Source: §5 · p. 10

    Diffusion-based virtual try-on methods still face garment fragmentation from imprecise segmentation and poor rendering of fine patterns such as flowers or text.

    Source: §2 · p. 3

Abstract

from arXiv · show

Despite recent progress, most existing virtual try-on methods still struggle to simultaneously address two core challenges: accurately aligning the garment image with the target human body, and preserving fine-grained garment textures and patterns. These two requirements map directly onto a coarse-to-fine generation paradigm, where the coarse stage handles structural alignment and the fine stage recovers rich garment details. Motivated by this observation, we propose DS-VTON, an enhanced dual-scale coarse-to-fine framework that tackles the try-on problem more effectively. DS-VTON consists of two stages: the first stage generates a low-resolution try-on result to capture the semantic correspondence between garment and body, where reduced detail facilitates robust structural alignment. In the second stage, a blend-refine diffusion process reconstructs high-resolution outputs by refining the residual between scales through noise-image blending, emphasizing texture fidelity and effectively correcting fine-detail errors from the low-resolution stage. In addition, our method adopts a fully mask-free generation strategy, eliminating reliance on human parsing maps or segmentation masks. Extensive experiments show that DS-VTON not only achieves state-of-the-art performance but consistently and significantly surpasses prior methods in both structural alignment and texture fidelity across multiple standard virtual try-on benchmarks.

1 Introduction

Virtual try-on must align garments to the body while preserving fine-grained texture, challenges that single-stage methods can struggle to address together. DS-VTON separates structural alignment from texture restoration in a dual-scale, mask-free framework and reports state-of-the-art performance on two benchmarks.

1 Introduction

  • Virtual try-on must fit garments accurately to human bodies while preserving fine-grained garment textures.Existing GAN and diffusion approaches face difficulties addressing both requirements together.Source: §1 · p. 2
  • DS-VTON separates global structure alignment from fine-grained texture restoration through low- and high-resolution stages.The high-resolution stage uses blend-refine diffusion to restore fine textures and correct detail errors from the first stage.Source: §1 · p. 2
  • The framework is mask-free, eliminating reliance on human parsing maps or segmentation masks.It leverages semantic priors embedded in pretrained diffusion models.Source: §1 · p. 2
  • Experiments on VITON-HD and DressCode demonstrate state-of-the-art performance, both qualitatively and quantitatively.The paper presents these results as validating the method’s effectiveness.Source: §1 · p. 3

2 Related works

Prior virtual try-on work has used both GAN-based warping-and-generation pipelines and diffusion-based synthesis strategies. Despite these advances, precise alignment and fine-pattern rendering remain difficult.

2 Related works

  • GAN-based methods typically warp garments to the target body before integrating them with the person image, but integration can introduce artifacts or detail loss.Examples include ACGPN’s TPS warping, PFAFN’s learned appearance flows, and VITON-HD’s specialized normalization and generator design.Source: §2 · p. 3
  • Diffusion-based methods range from two-stage warping and blending to single-process synthesis with conditioning strategies such as textual inversion, GarmentNet, and Leffa loss.FitDiT instead uses a DiT-based architecture with a rectangular mask for alignment issues in mask-based pipelines.Source: §2 · p. 3
  • Diffusion-based methods still face garment fragmentation from imprecise segmentation and poor rendering of fine patterns such as flowers or text.Source: §2 · p. 3

3 Methods

DS-VTON uses two resolutions to separate structural alignment from texture refinement, with a shared architecture across stages. Its high-resolution blend-refine process starts from a noise–coarse-result mixture and denoises toward the final try-on image.

3.1 Overview

  • The low-resolution representation suppresses fine textures to capture body shape, garment category, and coarse pattern placement for structural alignment and later refinement.The two stages share the same network architecture.Source: §3.1 · p. 4, Figure 2 · p. 4

3.4 High-resolution stage

  • High-resolution generation initializes its latent with a mixture of Gaussian noise and the upsampled low-resolution result, then denoises toward the final image.The person and garment are used as in the low-resolution stage; the latent initialization and denoising process differ.Source: §3.1 · p. 4, §3.4 · p. 5

3.2 Network architecture

  • A dual U-Net uses a reference U-Net for garment features and a main denoising U-Net; the framework initializes its U-Net weights from Stable Diffusion 1.5.The paper retains SD1.5 to favor a lightweight framework, despite SDXL's greater generative power.Source: §3.2 · p. 5

3.3 Low-resolution stage

  • The low-resolution stage downsamples person and garment images, then uses their encoded features and noise to generate a coarse try-on result.The downsampling ratio σ is selected from {1, 2, 4}; σ = 2 is chosen because σ = 4 sacrifices structural detail, while σ = 1 defeats the intended lower-resolution modeling.Source: §3.3 · p. 5

3.4.1 Reformulating the denoising process with blend-refine

  • Blend-refine reformulates diffusion as a transition from the low-resolution result to the high-resolution result, extending a prior initialization with two coefficients to control their relative influence.The model predicts a noise target combining noise and the low-resolution result; the remaining denoising process follows DDPM.Source: §3.4.1 · p. 6

3.4.2 Controlling noise-structure balance

  • The blend-refine coefficients balance noise and structural guidance: excessive β can suppress fine-detail recovery, while excessive α can disrupt structure.The paper reports α = β = 0.5 as an empirical balance between fidelity and flexibility.Source: §3.4.2 · p. 6

4 Experiments

Experiments evaluate DS-VTON on VITON-HD and DressCode, compare its outputs with recent methods, and ablate the dual-scale design and high-resolution initialization.

4.1 Experimental setup

  • The experiments use paired VITON-HD and DressCode data, with additional training pairs synthesized using IDM-VTON; evaluations use the unpaired setting, FID, KID, and a user study.Both datasets are evaluated at 768 × 1024; VITON-HD covers upper-body garments, while DressCode also includes lower-body garments and dresses.Source: §4.1 · pp. 6–7

4.2 Quantitative and qualitative results

  • DS-VTON consistently aligns garments across poses and garment types while preserving fine textures and tonal fidelity.Compared methods show issues including misalignment, missing patterns, artifacts, body-shape distortion, or tonal inconsistency.Source: §4.2 · pp. 7–8, Figure 3 · p. 7
  • DS-VTON achieves substantial improvements across VITON-HD and DressCode, with mask-free outputs that are robust and reproducible without dependence on mask quality.Table 1 compares the datasets; FitDiT is jointly trained on VITON-HD, DressCode, and CVDD, while the other methods are trained individually.Source: §4.2 · p. 8, Table 1 · p. 8

4.3 Ablation study

  • Among the tested variants, the full dual-scale pipeline with σ = 2 performs best overall; σ = 4 loses information in complex cases, while single-stage generation performs worst.The σ = 1 high-resolution variant has poor structural results, and adding refinement without reliable low-resolution guidance still leaves failures.Source: §4.3 · pp. 8–9, Table 2 · p. 8, Figure 4 · p. 9
  • With σ = 2 fixed, α = β = 1/2 produces generally good results; overly high α distorts structure, while α = 1/3, β = 2/3 leaves minor texture distortion.The authors adopt α = β = 1/2 as the default, supported by Table 3.Source: Table 3 · p. 8, §4.3 · p. 9, Figure 5 · p. 9

5 Conclusions

DS-VTON combines dual-scale generation with blend-refine denoising and a mask-free strategy. The authors report improvements in visual quality and robustness, with potential extension to higher resolutions and broader generation tasks.

5 Conclusions

  • DS-VTON uses blend-refine denoising to bridge low- and high-resolution generation.The conclusion presents this as the basis for a more effective coarse-to-fine process.Source: §5 · p. 9
  • Combined with a mask-free strategy, the framework achieves significant improvements in visual quality and robustness over existing methods.Source: §5 · p. 10
  • The authors describe the paradigm as inherently scalable and generalizable, with potential for higher resolutions and tasks such as personalized image synthesis.Source: §5 · p. 10

A.1.1 Additional training data generation and associated challenges

Mask-free training uses synthesized same-identity person images wearing a different garment, with a low-resolution try-on output also used to train the high-resolution stage. Synthesis can introduce changes outside the garment, though most images remain identity-consistent.

A.1.1 Additional training data generation and associated challenges

  • The low-resolution stage uses a garment image, a person wearing it, and a synthesized image of the same person wearing a different garment.The different-garment image is created by sampling another garment of the same category and synthesizing a new person image with IDM-VTON.Source: §A.1.1 · p. 12
  • High-resolution training adds the corresponding low-resolution try-on output and the ground-truth high-resolution image.The low-resolution output is generated using the target garment and the synthesized different-garment person image.Source: §A.1.1 · pp. 12–13
  • Synthesized person images may change hair, background, or accessories, but most remain visually consistent with the original identity.The authors identify clean, fully disentangled ground-truth data as a way these artifacts could be avoided.Source: §A.1.1 · p. 13

A.1.2 More implementation details

Experiments use two training stages on VITON-HD and DressCode, with inference on a single A6000 taking about five seconds per sample across both stages.

A.1.2 More implementation details

  • Both stages use AdamW with a 1e−6 learning rate, while training runs on 8 NVIDIA A6000 GPUs.For both datasets, batch sizes are 8 for the low-resolution stage and 2 for the high-resolution stage.Source: §A.1.2 · p. 13
  • VITON-HD training runs for 15,000 low-resolution and 30,000 high-resolution steps; DressCode runs for 20,000 and 30,000 steps, respectively.The reported training times are approximately 5 and 24 hours for VITON-HD, and 7 and 24 hours for DressCode.Source: §A.1.2 · p. 13
  • Inference uses 20 DDIM steps per stage; on one A6000, low- and high-resolution stages take about 1 and 4 seconds per sample.The method is reported slower only than CatVTON, comparable to FitDiT, and faster than the other compared methods.Source: §A.1.2 · p. 13

A.1.3 Additional Comparisons with Other Methods

The paired-setting comparison clarifies that competing methods require additional spatial or descriptive inputs, while DS-VTON uses paired images of the same person wearing different garments. Two DS-VTON evaluation variants are reported, with the train-way procedure adding artifacts.

A.1.3 Additional Comparisons with Other Methods

  • All competing methods require an agnostic mask; Leffa also uses DensePose, FitDiT keypoints, and IDM-VTON garment descriptions plus DensePose.DS-VTON is trained only on paired images of the same person wearing different garments.Source: §A.1.3 · p. 13
  • Because DS-VTON does not mask the original garment region, paired-setting inputs and expected outputs are identical, so two evaluation variants are included.The direct variant uses the garment and person images to generate the result.Source: §A.1.3 · p. 13
  • The train-way variant first uses IDM-VTON to synthesize a person wearing another garment, then re-dresses the original garment, adding artifacts that may understate DS-VTON’s capability.Quantitative results are reported in Table 4, but the supplied passage gives no values.Source: §A.1.3 · p. 13

A.2 More ablation studies

The appendix reports ablations, comparisons, limitations, and supplementary examples. It also discusses the refinement mechanism’s relationship to alternative approaches and potential practical use.

A.2.1 Ablation on removing cross-attention layers in U-Net

  • Adding cross-attention to both U-Nets does not improve garment-detail preservation and introduces noticeable distortions.The variant injects CLIP-encoded garment features through cross-attention; the comparison is shown in Figure 6 and Table 5.Source: §A.2.1 · p. 14, Figure 6 · p. 14, Table 5 · p. 15

A.3 Discussions

  • Both SD3- and SD3.5-based dual-DiT variants failed to converge, so the authors did not investigate them further.The authors speculate that directly applying DiT to mask-free try-on may be suboptimal.Source: §A.3 · p. 14
  • 8.24/0.31 FID/KID for DS-VTON compares with 8.98/0.65 for the SDXL variant, whose overall performance is reported inferior.Figure 7 provides a qualitative comparison; the passage does not identify the individual FID and KID values separately.Source: §A.3 · p. 14, Figure 7 · p. 15
  • Unlike DCI-VTON’s two-branch refinement, DS-VTON uses one branch and alters the noise formulation to explicitly bridge low- and high-resolution distributions.The authors describe the bridge as a controllable transition between the two complex distributions.Source: §A.3 · p. 15
  • DS-VTON’s multi-resolution outputs are described as suited to e-commerce platforms and devices needing different resolutions.The authors also note intellectual-property and personal-privacy concerns associated with generative technologies.Source: §A.3 · p. 16
  • Synthesized-person data may alter non-garment regions, and fixed α and β coefficients may make high-resolution refinement overly rigid.The authors identify improved disentanglement and identity preservation, and adaptive coefficient scheduling, as future-work directions.Source: §A.3 · p. 16

A.4 The Use of Large Language Models

  • The LLM was used only to refine text and improve clarity and readability, not for research ideation or experimental design.Source: §A.4 · p. 16

A.5 More experiment results

  • Supplementary results include DressCode qualitative comparisons for dresses, lower, and upper categories, plus VITON-HD comparisons and in-the-wild examples.The DressCode and VITON-HD captions distinguish low-resolution outputs from final high-resolution results.Source: §A.5 · p. 16, Figure 8 · p. 17, Figure 9 · p. 18

Related papers

Loading 2506.00908v2…