High-Resolution Virtual Try-On with Misalignment and Occlusion-Handled Conditions

1 Introduction

• Image-based virtual try-on uses a garment and person image, and prior frameworks commonly warp the garment and predict a segmentation map to guide image generation.The segmentation map separates regions to generate from those to preserve, and becomes more important at higher resolutions.Source: §1 · p. 2
• Separate warping and segmentation modules lack information exchange, causing garment-map misalignment and artifacts that especially harm high-resolution results.A prior attempt to alleviate misalignment does not solve it completely.Source: §1 · p. 2
• Lack of information exchange also causes excessive garment warping near body-occluded regions, producing pixel-squeezing artifacts that limit usable poses.These artifacts make real-world application more difficult.Source: §1 · pp. 2–3
• The framework achieves state-of-the-art results on a 1024×768 high-resolution dataset while handling misalignment and body-part occlusion.The authors report quantitative and qualitative results.Source: §1 · p. 3
• The unified try-on condition generator simultaneously predicts the warped garment and segmentation map, keeping them aligned and naturally handling body-part occlusion.The architecture combines warping and segmentation generation, making it inherently misalignment-free.Source: §1 · p. 3
• Discriminator rejection filters incorrect segmentation-map predictions that can produce unnatural results and helps assure virtual try-on performance.The authors present this assurance as important for real-world applications.Source: §1 · p. 3

2.1 Image-based Virtual Try-On

• TPS-based warping in VITON and CP-VTON has limited freedom, while appearance flow computes pixel-wise deformation; VITON-HD normalization does not naturally fill misaligned regions.The paper proposes generating warped clothes without misaligned regions.Source: §2.1 · pp. 3–4
• Segmentation maps of people wearing target clothes disentangle appearance from shape and guide spatially coherent synthesis, becoming more important at higher image resolutions.High-resolution methods VITON-HD and generally include a segmentation generation module.Source: §2.1 · p. 4

2.2 Rejection Sampling

• Prior rejection methods use classifier-based sampling or discriminator rejection [1], but discriminator rejection has not been commonly used for image-conditional generation; this paper applies it at test time.The discriminator filters low-quality samples; under strict assumptions, discriminator rejection sampling allows exact sampling from the data distribution.Source: §2.2 · p. 4

3.1 Try-On Condition Generator

• The condition generator jointly predicts clothing appearance flow and the worn-clothing segmentation by exchanging flow and segmentation features, aligning the warped garment and map.Body-part occlusion handling removes occluded regions from the warped clothing to reduce pixel-squeezing artifacts; losses supervise segmentation and clothing fit, with total-variation regularization applied to the final flow.Source: §3.1 · pp. 5, 7

3.2 Try-On Image Generator

• The image generator fuses the clothing-agnostic image, warped clothing, and pose map, using the predicted segmentation to guide SPADE-normalized residual blocks.The generator is trained with the losses used in SPADE and pix2pixHD.Source: §3.2 · p. 7

3.3 Discriminator Rejection

• At test time, discriminator rejection filters low-quality segmentation maps by rejecting predictions whose acceptance probability falls below a threshold.The acceptance probability is derived using the discriminator, with its normalizing constant estimated from the training dataset.Source: §3.3 · p. 8

4.1 Training

• Experiments use VITON-HD, with 13,679 frontal-view woman and top-clothing image pairs split into 11,647 training and 2,032 test pairs.Images have original resolution 1024×768 and are bicubically downsampled when needed.Source: §4.1 · p. 8

4.2 Qualitative Results

• Compared with baselines, HR-VITON produces more photo-realistic images, preserves target-clothing details, and generates necklines and body shapes more naturally.The results emphasize that condition quality is crucial for perceptually convincing synthesis; VITON-HD retains artifacts in misaligned regions, which HR-VITON avoids with misalignment-free segmentation maps and warped clothing.Source: §4.2 · pp. 8, 10, Figure 4 · p. 9, Figure 5 · p. 10
• Without body-part occlusion handling, excessive clothing deformation squeezes garment textures and removes patterns, while handling preserves high-frequency details.The comparison identifies logos and stripes as examples of squeezed patterns.Source: §4.2 · p. 10, Figure 7 · p. 11
• Discriminator rejection filters low-quality segmentation maps, which often result from preprocessing errors such as incorrect clothing masks.Rejected examples have considerably impaired segmentation maps; the method is presented as a simple, effective filter for low-quality outputs.Source: §4.2 · pp. 10, 12, Figure 8 · p. 12

4.3 Quantitative Results

• Evaluation uses paired reconstruction and unpaired clothing-change settings, with SSIM and LPIPS for paired results and FID and KID for unpaired results.The paper describes KID as more descriptive than FID when the sample count is small.Source: §4.3 · pp. 12–13
• HR-VITON outperforms CP-VTON [28], ACGPN [32], VITON-HD [2], and PF-AFN across metrics, especially at 1024×768; CP-VTON and ACGPN struggle with high-resolution unpaired try-on.The comparison also reports that HR-VITON surpasses VITON-HD despite using a similar try-on image generator, which the authors attribute to higher-quality conditions.Source: Table 2 · p. 13, §4.3 · p. 13, §4.4 · p. 14
• The feature fusion block and condition aligning provide largely additive benefits, and removing both yields suboptimal results that underscore the need for information exchange.The ablation is evaluated in the unpaired setting.Source: Table 1 · p. 12, §4.3 · p. 13

4.4 Comparison with Parser-free Virtual Try-on Methods

• Against parser-free PF-AFN, HR-VITON avoids failures to remove original clothing regions and outperforms it by a large margin, supporting segmentation prediction for convincing high-resolution try-on.The comparison argues that segmentation maps help distinguish regions to generate from those to preserve.Source: Figure 9 · p. 13, Table 2 · p. 13, §4.4 · p. 14

Limitation of Discriminator Rejection

• Existing image-based try-on methods assume test data matches training data, but real-world inputs may show different camera views or contain no humans.Such out-of-distribution inputs can lead to low-quality segmentation predictions, which discriminator rejection can filter.Source: §5 · p. 14
• The authors believe discriminator rejection can enhance user experience in virtual try-on applications by filtering out-of-distribution inputs.Source: §5 · p. 14

• At 1024×768 resolution, extensive experiments show the method outperforms existing virtual try-on methods.Source: §6 · p. 14
• The try-on condition generator jointly performs clothing warping and segmentation generation while exchanging information between the two stages.Source: §6 · p. 14
• By handling occlusion from body parts, the condition generator eliminates misaligned regions and solves pixel-squeezing.Source: §6 · p. 14
• The condition generator’s discriminator filters impaired segmentation results, which is practically helpful for real-world virtual try-on applications.Source: §6 · p. 14

A. Implementation Details

• The try-on condition generator has two encoders, four feature-fusion blocks, and five residual blocks per encoder.It concatenates the encoders’ final features and applies a 3×3 convolution to produce the first flow map; the segmentation encoder’s final feature passes through two residual blocks into the segmentation pathway.Source: A. Implementation Details · p. 17, Figure 10 · p. 17
• Condition-generator predictions are made at 256×192 and its segmentation map and appearance flow are upscaled to 1024×768 for image generation.Its discriminator inputs are downsampled by a factor of 2, and discriminator dropout is used to stabilize training.Source: A. Implementation Details · pp. 17–18
• The condition generator uses λCE=10, λVGG=10, and λTV=2, batch size 8, 100,000 training iterations, and a generator and discriminator learning rate of 0.0002.Two multi-scale discriminators are used for its conditional adversarial loss.Source: A. Implementation Details · pp. 17–18
• The image generator uses residual blocks with upsampling layers, two multi-scale discriminators, and spectral normalization on all convolutional layers.Its detailed architecture is shown in Fig. 11.Source: A. Implementation Details · p. 18, Figure 11 · p. 18
• The image generator’s objective combines conditional adversarial, perceptual, and feature-matching losses, following SPADE and pix2pixHD.The objective is expressed as a weighted sum of these losses.Source: A. Implementation Details · p. 18
• The image generator and discriminator learning rates are 0.0001 and 0.0004, respectively; both use Adam with β1=0.5 and β2=0.999.The image generator is trained with batch size 4 for 100,000 iterations.Source: A. Implementation Details · pp. 18–19

Results on Different Resolutions

• Qualitative comparisons include results at different resolutions, including baseline comparisons at 512×384.The paper points to Figs. 12 and 13 for the cross-resolution comparisons.Source: B. Additional Experiments · p. 19, Figure 13 · p. 20

Comparison with the Variant of VITON-HD

• At 1024×768, our model consistently outperforms VITON-HD* on all evaluation metrics and avoids its misalignment and pixel-squeezing artifacts.VITON-HD* replaces VITON-HD’s clothes warping module with Clothflow’s, which reduces misalignment but does not eliminate the artifacts; greater warping freedom can worsen pixel squeezing without occlusion handling.Source: B. Additional Experiments · pp. 19, 22, Table 3 · p. 21, Figure 14 · p. 21

User Study

• In a study with 21 participants judging 30 test-set examples, our model had the highest average selection rate for photo-realism and clothing-detail preservation.Participants chose among synthesized results from the model and baselines at 1024×768.Source: B. Additional Experiments · p. 22, Figure 15 · p. 22

Effectiveness of Multi-Scale L1/VGG Losses

• Multi-scale L1/VGG losses help the model learn meaningful intermediate flow at coarse scales, supporting coarse-to-fine appearance-flow generation.Without these losses, the model has difficulty learning coarse-scale flow estimation.Source: B. Additional Experiments · pp. 22–23, Figure 16 · p. 22

Additional Results

• Additional examples show combinations of different clothes and people, as well as high-resolution synthesis at 1024×768.These results appear in Figs. 17–20.Source: B. Additional Experiments · p. 23, Figure 17 · p. 24, Figure 18 · p. 25