Dynamic Try-On: Taming Video Virtual Try-on with Dynamic Attention Mechanism

• Dynamic Try-On repurposes the DiT backbone as a garment encoder and combines DFFM with LDAM to preserve clothing details and body-motion consistency.DFFM stores and integrates garment features, while LDAM focuses denoising on limb regions.Source: §3 · p. 2

• Video try-on must preserve clothing, identity, and motion, but prior methods often struggle with temporal coherence during rapid human movements.Source: §1 · p. 2
• LDAM selects limb-related tokens and enforces their temporal consistency, targeting failures caused by limited cross-frame attention during rapid movements.Source: Figure 2 · p. 3, §1 · p. 3
• Dynamic Try-On uses DFFM to store and integrate garment features without a bulky garment encoder, reducing the computational burden of video try-on.Source: §1 · p. 3
• Dynamic Try-On outperforms existing methods in both quantitative and qualitative video-generation evaluations.Source: §1 · p. 3
• The framework is designed for consistent spatio-temporal generation in videos with complex human motions.Source: §1 · p. 3

2.1 Video Virtual Try-on.

• Dynamic Try-On targets complicated human postures and long video sequences with high-quality spatiotemporal consistency, unlike prior demonstrations limited to simpler movements and frame counts.ViViD handles camera movements and preserves clothing textures, but its demonstrations primarily use product images, simple movements, and limited frame counts.Source: §2.1 · p. 4
• GAN-based methods use garment warping and fusion but often suffer garment-person misalignment when optical-flow estimation is inaccurate.FW-GAN warps preceding frames for temporal coherence, while ClothFormer [16] integrates garment and person features through a dual-stream transformer.Source: §2.1 · p. 4

2.2 Image Animation.

• A two-stage pipeline that applies image try-on to the first frame before animation can lose detailed garment information when no in-shop clothing reference is available.Comparative experiments in Sec. 4.4 are reported to favor the integrated Dynamic Try-On approach.Source: §2.2 · p. 4

3 Method

• Dynamic Try-On is a video virtual try-on framework built upon diffusion transformers (DiT) [28].The paper introduces latent diffusion models and DiT before presenting the overall architecture.Source: §3 · p. 4
• The architecture includes a dynamic feature fusion module, DFFM, and a limb-aware dynamic attention module, LDAM.DFFM and LDAM are introduced in Sec. 3.3 and Sec. 3.4, respectively.Source: §3 · p. 4
• The method section proceeds from background concepts to the overall architecture and then to detailed descriptions of DFFM and LDAM.Latent Diffusion Models and DiT are reviewed in Sec. 3.1, followed by the architecture in Sec. 3.2.Source: §3 · p. 4

3.1.1 Latent Diffusion Models (LDMs)

• Figure 3 depicts Dynamic Try-On extracting garment features in Step 1 and delivering them with human-pose information in Step 2.The overview connects garment processing and pose-conditioned generation within the proposed pipeline.Source: Figure 3 · p. 5
• LDMs reduce computational difficulty by encoding high-dimensional data into a compact latent space before applying diffusion.The latent representation is decoded after denoising to recover the original image or video.Source: §3.1.1 · pp. 4–5
• An LDM combines an encoder, a forward noise process, a learned reverse denoising network, and a decoder.The encoder maps x to z, diffusion progressively adds noise, the denoiser predicts the reverse process, and the decoder reconstructs the output.Source: §3.1.1 · p. 5

3.1.2 Diffusion Transformers (DiT)

• DiT retains the diffusion formulation but uses a transformer instead of a U-Net to learn the denoising function.The proposed framework uses a modified Spatio-Temporal DiT as its backbone.Source: §3.1.2 · p. 5

3.2.1 Formulation of Video Try-On Task

• The task places target clothing c onto a reference video using a cloth-agnostic frame, pose skeleton frame, and inpainting mask frame.Video virtual try-on is treated as a video inpainting problem, with the inputs visualized in Figure 3.Source: §3.2.1 · p. 6
• A trainable replica of the denoising backbone’s first block adds a residual that preserves the person’s pose, identity, and background.The replica is denoted C and is added only to the first denoising DiT block.Source: §3.2.1 · p. 6

3.2.2 Step 1: Garment Feature Extraction

• In Step 1, the garment image is encoded and passed through N ST-DiT blocks, with intermediate garment features stored in a feature bank.Temporal self-attention and limb-aware dynamic attention are skipped because this pass processes a single garment image without pose information.Source: §3.2.2 · p. 6

3.2.3 Step 2: Dynamic Attention Mechanism

• In Step 2, each ST-DiT block retrieves the corresponding garment feature through DFFM while LDAM enhances denoising features using the human pose sequence.The two modules jointly integrate clothing information and pose-aware structure during generation.Source: §3.2.3 · p. 6

3.3 Dynamic Feature Fusion Module

• DFFM uses two backbone passes: one stores garment features in a feature bank, and the other combines them with denoising features during video generation.This design provides a lightweight alternative to parallel garment encoders used by prior methods.Source: §3.3 · pp. 6–7
• The input video is encoded into latent z0 with h = H/8 and w = W/8 before noise addition and patchification into denoising features.The latent representation has shape f×h×w×4, where f is the number of frames.Source: §3.3 · p. 6
• DFFM duplicates the garment feature across frames, applies cross-attention with the denoising feature, and adds the output back through a residual connection.The garment feature is retrieved from the bank and matched to the denoising feature before fusion.Source: §3.3 · p. 7
• Using the denoising backbone itself as the garment encoder improves garment-feature perception while decreasing computational demands.The module is intended to preserve texture details without a bulky additional encoder.Source: §3.3 · pp. 6–7

3.4 Limb-aware Dynamic Attention Module

• LDAM addresses limb inconsistency during rapid motion, overlap, or temporary disappearance, where basic spatial-temporal attention struggles.The module is proposed as an efficiency-conscious alternative to computationally overwhelming 3D full attention.Source: §3.4 · p. 7, Table 1 · p. 8
• Pose-derived limb masks dynamically index, group, and model tokens from the denoising feature so each limb remains consistent across the generated video.Figure 4 visualizes the limb-aware dynamic attention mechanism.Source: §3.4 · p. 7, Figure 4 · p. 7
• LDAM retrieves masked limb features, pads them to a common length n, computes masked self-attention, and writes the updated features back to the denoising representation.The limb features have shape L×n×d and use an attention mask for the self-attention calculation.Source: §3.4 · p. 7
• LDAM is described as a plug-and-play complement to spatial and temporal attention for enhancing temporal consistency efficiently.Further implementation details are deferred to the supplementary materials.Source: §3.4 · pp. 7–8
• The module uses L = 4 limbs and n = 12 compared with f×s = 192×36 in the stated 192×256 training setting.The comparison fixes the number of limbs and reports the much smaller limb-token length used in the theoretical analysis.Source: Table 1 · p. 8

4.1 Datasets

• The evaluation uses the VVT dataset and a custom in-shop video dataset designed to include complex human poses and occlusions.VVT contains 791 paired person videos, while the custom dataset contains 9,100 video-image pairs with separate training and testing splits.Source: §4.1 · p. 8

4.2 Implementation Details

• Training proceeds in three stages, progressively enabling spatial and cross-attention, the ST-DiT block, and finally LDAM.Experiments use 192×256 inputs for VVT comparisons and 384×512 inputs for demonstrations, with AdamW training on eight A100 GPUs.Source: §4.2 · p. 8

4.3 Qualitative Results

• Dynamic Try-On preserves clothing details and maintains inter-frame consistency better than baselines across close viewpoints and complex motion.Compared with ClothFormer, StableVITON, OOTDiffusion, and ViViD, it avoids reported misalignment, blur, texture discrepancy, and temporal jitter.Source: §4.3 · pp. 8–9, Figure 5 · p. 9

4.4 Quantitative Results

• Dynamic Try-On outperforms other methods on the VVT dataset, while image-based and two-stage animation pipelines show weaker video performance.The quantitative comparison evaluates frame-wise quality and video quality or temporal consistency using SSIM, LPIPS, VFID, and FVD.Source: §4.4 · p. 9, Table 2 · p. 10

4.5 Ablation Study

• DFFM reduces training-memory growth while preserving clothing details, and LDAM improves all evaluated metrics and handles overlapping limbs better than tested 3D full attention.The ablations compare memory cost across backbone sizes and evaluate DFFM and LDAM on the VVT dataset.Source: §4.5 · p. 10, Table 3 · p. 10, Table 4 · p. 10

• Dynamic Try-On uses dynamic attention to recover clothing details and maintain consistent body movements in videos with diverse clothing and complex motions.The conclusion states that the framework outperforms previous methods in all aspects.Source: §5 · p. 10