Source-linked AI summary
FlowVVTON: Flow-Guided Mask-Free Video Virtual Try-On
Shengyao Chen, Xianbing Sun, Liqing Zhang, Jianfu Zhang
TL;DR
Video try-on lacks explicit motion supervision, leaving open whether flow can guide mask-free temporal consistency. FlowVVTON uses optical-flow-guided latent supervision during training without preprocessing masks. It achieves a 5.7× VFID-R improvement over SwiftTry.
Problem
Video try-on lacks explicit pixel-level motion information, raising the question of whether optical flow can enable effective, robust mask-free try-on.
Source: Introduction · p. 2Method
FlowVVTON trains mask-free video try-on with a per-layer flow-warped latent loss for multi-scale temporal supervision, using optical flow only during training.
Source: Introduction · p. 2Results
5.7× VFID-R improvement over SwiftTry was achieved by FlowVVTON.
Source: Introduction · p. 2Takeaways & Limitations
At inference, FlowVVTON generates temporally consistent video without computing optical flow.
Source: Introduction · p. 2FlowVVTON depends on optical-flow quality during training.
Source: Conclusion · p. 7
Abstract
from arXiv · showhide
Video virtual try-on aims to transfer a target garment onto a moving person across video frames. Current methods rely on human parsing masks or pose keypoints that frequently fail under large motions and occlusions, causing boundary artifacts and temporal inconsistency. A further limitation is that most approaches rely solely on attention mechanisms for temporal modeling, providing no explicit motion supervision. We propose FlowVVTON, a mask-free framework that eliminates parsing mask dependency entirely. Optical flow is used solely as a training-time supervision signal: a flow-warped latent loss, applied across all layers of the generation model, enforces multi-scale temporal consistency by aligning adjacent-frame features under explicit physical motion constraints. A two-stage training strategy establishes mask-free spatial alignment before introducing flow-guided temporal supervision. Experiments on TikTokDress show that FlowVVTON outperforms baselines by substantial margins, particularly in temporal consistency (5.7$\times$ VFID-R improvement over SwiftTry), while requiring no segmentation masks, pose keypoints, or region annotations at any stage.
Introduction
FlowVVTON addresses video try-on’s need for stable garment appearance across moving frames by removing reliance on brittle parsing and pose estimates and using optical flow for training-time temporal supervision. It reports improved reconstruction quality and temporal consistency, including a 5.7× VFID-R improvement over SwiftTry.
Motivation
- Parsing and pose estimates can fail under fast motion, self-occlusion, or challenging lighting, while attention-only temporal modeling lacks explicit motion supervision and can break down under rapid movement.These failures can propagate into jagged garment edges, background texture bleeding, and temporal instability.Source: Introduction · p. 1
- Optical flow provides dense pixel-level correspondence and motion information, motivating its use as training-time supervision rather than brittle semantic masks.Video try-on must keep the transferred garment stable across frames despite movement, rotation, and limb occlusion.Source: Introduction · pp. 1–2
Method
- The mask-free framework takes raw video frames and a garment image, using a per-layer flow-warped latent loss to align adjacent-frame features during training.Optical flow is used only as a training loss signal; inference requires no flow computation.Source: Figure 1 · p. 1, Introduction · p. 2
Results
- FlowVVTON significantly outperforms existing baselines in reconstruction quality and temporal consistency, achieving a 5.7× VFID-R improvement over SwiftTry.Experiments use synthetic paired training data prepared from TikTokDress with off-the-shelf video try-on models.Source: Introduction · p. 2
Motivation and contribution
- By eliminating parsing masks, FlowVVTON avoids mask-induced boundary artifacts and is designed to handle large body motions more robustly.It requires no parsing masks, pose keypoints, or region annotations.Source: Introduction · p. 2
Training strategy
- Training first establishes mask-free spatial alignment on video frames, then adds flow-guided temporal supervision to enforce cross-frame consistency.The two-stage pipeline uses the flow-warped latent loss in its second stage.Source: Introduction · p. 2
Related Work
Related work moves from image-based garment transfer toward video generation, mask-free methods, and optical-flow guidance. FlowVVTON builds on mask-free image try-on and uses flow as explicit geometric supervision for temporal consistency.
Image Virtual Try-On
- Image try-on progressed from controlled-condition GAN warp-and-blend pipelines to dominant diffusion models; mask-free image methods remove parsing dependence, and FlowVVTON adopts a pretrained mask-free model.GAN methods include Han et al. 2018, Wang et al. 2018, Han et al. 2019, Ge et al. 2021, and Choi et al. 2021; mask-free methods include Zhang et al. 2025, Wan et al. 2025, Jo, Park, and Kang 2025, Guo et al. 2025, and Sun et al. 2025.Source: Image Virtual Try-On · p. 2
Video Virtual Try-On
- Video try-on evolved from independently processed, post-smoothed frames to end-to-end video diffusion, but these approaches rely on parsing masks or pose keypoints.The cited video-diffusion approaches include SVD-based methods (Fang et al. 2024b; Xu et al. 2024b; He et al. 2024; Li et al. 2025a,c; Nguyen et al. 2025) and DiT-based architectures (Zheng et al. 2024; Chong et al. 2025b; Li et al. 2025b; Zuo et al. 2025; He et al. 2026; Zeng et al. 2025; Zheng et al. 2026).Source: Video Virtual Try-On · p. 2
Mask-Free Try-On
- Video mask-free methods replace dense masks with sparse keypoints, coarse boxes, or pose-driven animation, yet retain spatial annotations or mask-based teachers; FlowVVTON uses raw frames and a garment image.FlowVVTON uses a flow-warped latent loss for temporal consistency.Source: Mask-Free Try-On · p. 2
Optical Flow for Video Generation
- Prior video-generation work uses optical flow for losses, conditioning, or model-internal injection; FlowVVTON instead applies per-layer flow-warped latent loss to align adjacent-frame UNet features under motion constraints.The cited approaches use flow losses (Wu, Ota, and Kanezaki 2025; Bhowmik et al. 2025), conditioning (Wang et al. 2026; Nam et al. 2025; Liao et al. 2025; Liang et al. 2024), or internal injection (Guo et al. 2024; Xue et al. 2025; Koroglu et al. 2025; Gökmen et al. 2025).Source: Optical Flow for Video Generation · pp. 2–3
Method
FlowVVTON uses garment-feature injection and temporal attention in a mask-free video-generation model, with optical flow providing training-only, multi-scale feature consistency supervision. A two-stage pipeline separates spatial garment alignment from flow-guided temporal training.
Method
- The framework uses a pretrained-VAE latent diffusion backbone, operating on compact latent codes for video frames and garment images.Source: Method · p. 3
Model Architecture
- A garment-reference UNet injects multi-scale features into a video-generation UNet through mutual self-attention, while temporal attention models frame motion and spatial weights stay frozen.Only temporal attention layers are trained, preserving spatial quality from image pretraining.Source: Model Architecture · p. 3
- The main UNet receives noisy latents concatenated with the raw source video, learning garment regions from feature contrast and injected garment features without masks or pose keypoints.Optical flow is used only as a training loss signal, not computed at inference.Source: Model Architecture · p. 3
Why feature space?
- During training, the flow-warped latent loss matches noise-free features from the main UNet across layers by backward-warping adjacent-frame features, supervising semantic representations rather than pixels.Flows are computed directly on subsampled frames at strides 1, 2, and 4, with weights α1=1.0, α2=0.5, and α4=0.25.Source: Why feature space? · p. 3
Data Preparation
- Training pairs come from synthesized garment transfers on TikTokDress videos, with the original videos serving as ground truth and the source of cleaner flow supervision.The dataset comprises 693 training and 124 test videos.Source: Data Preparation · p. 4
Training Strategy
- Training first establishes mask-free image-level garment alignment, then freezes spatial weights and trains temporal attention with flow guidance across the main UNet layers.The total objective combines diffusion denoising with the flow-warped latent loss; denoising uses Min-SNR weighting with γ = 5.0.Source: Training Strategy · p. 4
Experiments
On TikTokDress, FlowVVTON outperforms SwiftTry in paired and unpaired evaluation, while ablations show the importance of flow-based temporal supervision and multi-scale alignment.
Datasets and Metrics
- Evaluation uses the TikTokDress test set in paired and unpaired settings; paired metrics are SSIM, LPIPS, and VFID with I3D and ResNeXt backbones, while unpaired evaluation reports both VFID variants.VFIDI3D captures spatiotemporal dynamics, whereas VFIDRN captures spatial texture quality.Source: Datasets and Metrics · p. 5, Quantitative Comparison · p. 5
Implementation Details
- The model uses a frozen pretrained VAE and reference UNet, with only temporal attention layers in a 3D-inflated main UNet trained.Training first fine-tunes the image try-on model for 50K steps, then trains temporal layers for 100K; RAFT uses 32 updates and forward-backward filtering (τ = 3.0), and inference uses 25-step DDIM with 24-frame windows and 4-frame overlap.Source: Implementation Details · p. 5
Qualitative Comparison
- FlowVVTON retains sharper textures, clean boundaries, and stable garment adherence than SwiftTry, especially during rapid turns or arm crossings; additional results show generalization across garments and motion patterns.Source: Qualitative Comparison · p. 5, Figure 5 · p. 5, Figure 6 · p. 6
Quantitative Comparison
- FlowVVTON achieves the best results across all four paired metrics, with a 6× VFID-R improvement over SwiftTry (0.71 vs. 4.30), and a 5.7× unpaired VFID-R gain (0.60 vs. 3.42).Source: Quantitative Comparison · pp. 5–6, Table 1 · p. 6, Table 2 · p. 6
Ablation Study
- Removing flow-warped loss raises VFID-I from 30.99 to 32.48 and VFID-R from 0.60 to 1.24; removing confidence weighting yields VFID-R 0.67 and occlusion-boundary artifacts.The results support explicit flow supervision and confidence filtering of unreliable flow regions.Source: Ablation Study · p. 6, Figure 7 · p. 7
- Stride-1 flow gives VFID-I 30.95 and VFID-R 0.72; strides {1, 2, 4} reach VFID-R 0.60, a 17% improvement, while stride-8 degrades it to 0.67.The full multi-scale configuration combines short-, intermediate-, and long-range constraints; Figure 7 reports the best temporal smoothness for strides {1, 2, 4}.Source: Table 3 · p. 6, Ablation Study · p. 7, Figure 7 · p. 7
- Reducing RAFT iterations from 32 to 16 worsens VFID-I from 30.99 to 31.46 and VFID-R from 0.60 to 0.61; 48 iterations yield VFID-I 30.92 with identical VFID-R.At 16 iterations, flow is noisy and misaligned at motion boundaries; 48 iterations offer diminishing returns while increasing training time.Source: Ablation Study · p. 7, Table 4 · p. 7, Figure 8 · p. 7
Conclusion
FlowVVTON removes preprocessing mask dependencies and uses optical flow only as training-time supervision to improve temporal consistency and reduce boundary artifacts. Its approach has limitations in flow quality, architecture, and resolution, but the authors identify potential applications beyond try-on.
Conclusion
- FlowVVTON uses a per-layer flow-warped latent loss, with multi-scale strides, to improve temporal consistency and reduce boundary artifacts, particularly under large motions where mask-based methods degrade.The experiments identify the flow-warped loss and multi-scale strides as essential.Source: Conclusion · p. 7
- The approach depends on optical-flow quality during training, is UNet-specific, and faces resolution constraints, while its mask-free paradigm and flow-guided loss may extend to video inpainting and motion-guided stylization.Source: Conclusion · p. 7