Wear-Any-Way: Manipulable Virtual Try-on via Sparse Correspondence Alignment

1 Introduction

• Users can customize sleeve rolling, coat opening, and tuck style through simple clicks and drags using a single model in one pass.The interactions provide point-based control over specific garment locations and wearing styles.Source: Figure 1 · p. 2, §1 · p. 3
• Existing methods struggle with detailed garment patterns, complex poses and backgrounds, model-to-model try-on, multi-garment inputs, and complicated scenarios.They also generally lack control over how garments are worn, despite styling variations being important in fashion applications.Source: §1 · p. 2
• Wear-Any-Way supports high-quality standard virtual try-on across diversified inputs and scenarios, including model-to-model, multi-garment, and complex settings.Its dual-branch pipeline uses a main denoising U-Net, a reference U-Net for garment features, self-attention injection, and human-pose guidance.Source: §1 · p. 3
• Sparse correspondence alignment matches specified garment points to target person points by injecting permutable point embeddings into both U-Nets.Condition dropping, zero-initialization, and point-weighted loss improve the learnability and controllability of this alignment.Source: §1 · p. 3
• The framework is presented as a strong baseline that reaches state-of-the-art performance while preserving fine garment patterns and enabling customizable wearing styles.The contributions combine standard try-on quality with flexible point-based manipulation.Source: §1 · p. 3

GAN-based virtual try-on

• GAN-based try-on methods often degrade on complex poses and intricate backgrounds, while large spatial clothing transformations remain difficult for conditional GANs.Many GAN pipelines first deform garments and then integrate them with a try-on generator.Source: §2 · p. 4

Diffusion-based virtual try-on

• Diffusion-based try-on improves generation quality but existing approaches can require extensive pose data, inaccurately warp garments, or lose fine details.These methods generally produce rigid static images without flexible control over wearing style.Source: §2 · p. 4

Point-based image editing

• Text-guided editing typically provides broad modifications rather than the precision required for detailed garment manipulation.Point-based editing addresses this limitation, but earlier GAN-based approaches inherit constraints from their underlying models.Source: §2 · pp. 4–5
• Wear-Any-Way combines diffusion models with point-based control to enable fine-grained, context-aware alterations of virtual try-on results.The approach is intended to improve both the granularity and adaptability of image editing.Source: §2 · p. 5

3.1 Preliminaries

• Stable Diffusion progressively adds Gaussian noise to latent representations and learns to predict that noise before decoding denoised latents into images.The framework uses this latent diffusion process as its generative foundation.Source: §3.1 · p. 5

3.2 Virtual Try-on Pipeline

• Wear-Any-Way uses a dual-branch diffusion pipeline in which the main U-Net generates the try-on image while the reference U-Net extracts detailed garment features.The main branch receives masked-person latents, garment masks, and noise; reference features are fused through attention.Source: §3.2 · p. 6

3.3 Sparse Correspondence Alignment

• Sparse correspondence alignment matches marked garment points to target person points, enabling users to manipulate wearing style through multiple controls.Point embeddings are injected into both U-Nets and modify attention to align garment features with target positions.Source: §3.3 · pp. 7–8
• The method represents control points with randomly assigned, corresponding labels and supports up to K = 24 points at arbitrary locations.Random label assignment makes the representation permutable and decouples point semantics from locations.Source: §3.3 · p. 7

3.4 Training strategies

• Training improves controllability through condition dropping, zero-initialized point injection, and point-weighted loss around sampled person-image points.These strategies respectively encourage point reliance, stabilize optimization, and emphasize correspondence supervision.Source: §3.4 · p. 8

3.5 Training Points Collection

• Training point pairs are obtained by matching person and garment diffusion features with cosine similarity, then sampling queries inside and along clothing-mask boundaries.The person-to-garment direction is chosen because complex poses can leave some garment points unmatched on the body.Source: Figure 3 · p. 8, §3.5 · p. 9

3.6 Inference with manipulation

• Users can click paired garment and person points or convert a drag into such a pair, allowing garment locations to be aligned with desired positions in the generated image.Drag endpoints become garment and person points, while parsed clothes serve as the garment image.Source: §3.6 · pp. 9–10, Figure 4 · p. 9

4.1 Implementation Details

• The experiments cover standard virtual try-on and point control, using VITON-HD, Dresscode, and a 1,000-sample benchmark spanning upper, down, and coat garments.The protocol detects fashion landmarks and measures Euclidean distance between target and generated landmarks.Source: §4.2 · p. 10, §4.1 · p. 10

4.3 Ablations Studies

• The reference U-Net preserves fine-grained garment patterns, texts, and logos better than CLIP, DINOv2, and ControlNet feature extractors.The comparison uses the same training settings across feature extractors.Source: §4.3 · p. 11, Figure 5 · p. 11
• Attention-layer point-embedding injection and stepwise enhancement strategies are evaluated using landmark distance to improve point-based control.Table 1 compares injection locations, while Table 2 adds the proposed enhancement strategies incrementally.Source: §4.3 · p. 11, Table 1 · p. 12, Table 2 · p. 12

4.4 Comparisons with Existing Alternatives

• Wear-Any-Way achieves the best FID and KID on VITON-HD and Dresscode, with competitive SSIM and LPIPS.Table 4 reports quantitative comparisons against existing state-of-the-art try-on solutions on both datasets.Source: §4.4 · p. 13, Table 4 · p. 14
• Wear-Any-Way qualitatively outperforms prior try-on methods in detail preservation and generation quality, while drag-based alternatives struggle with instruction following or structural integrity.Comparisons include DCI-VTON, LaDI-VTON, KGI, AnyDoor, StableVITON, DragDiffusion, and DragonDiffusion.Source: Figure 6 · p. 12, §4.4 · p. 13, Figure 7 · p. 13

4.5 Qualitative Analysis

• Users can assign arbitrary numbers of control points across coats, T-shirts, pants, and hoodies to create continuous coat opening, rolled-up sleeves, and varied tuck styles.The qualitative analysis demonstrates point-based manipulation across multiple garment types and wearing styles.Source: §4.5 · pp. 13–14, Figure 8 · p. 14

• Wear-Any-Way reaches state-of-the-art performance in classical virtual try-on and enables users to customize wearing styles by assigning control points.Source: §5 · p. 14
• The pipeline uses a dual U-Net design and sparse correspondence alignment to make virtual try-on customizable.Source: §5 · p. 15
• Artifacts may remain in fine details such as human hands, especially when hands occupy a small image region.The paper suggests higher resolutions and larger diffusion models as possible improvements.Source: §5 · p. 15