PaperlayerView exact revision on arXiv

Source-linked AI summary

JewelTry: Mask-Free Scale Aware Jewelry Virtual Try-On

Xinlei Niu, Peixia Li, Jun Wang, Chenchen Xu, Jiayu Yang, Jing Zhang, Pulak Purkait, Hongdong Li

arXiv:2609.16626v1cs.CV

TL;DR

Jewelry virtual try-on must preserve fine detail while placing items at plausible scales, but existing benchmarks and mask-free scale guidance are limited. This paper introduces JVTO-Bench and JewelTry, a mask-free scale-aware diffusion framework; experiments report a balance across visual fidelity, background preservation, object consistency, and scale accuracy.

  • Problem

    Jewelry VTON requires preserving fine details while rendering items at plausible scales, but suitable jewelry-specific benchmarks and explicit scale guidance for mask-free methods are limited.

    Source: Introduction · pp. 1–2

  • Method

    The paper introduces JVTO-Bench and JewelTry, a mask-free diffusion framework that encodes product dimensions as a scale token and uses attention mechanisms to preserve jewelry details.

    Source: Introduction · p. 2, Conclusion · p. 7

  • Results

    JewelTry achieves a balance among visual fidelity, background preservation, object consistency, and scale accuracy.

    Source: Conclusion · p. 7

  • Takeaways & Limitations

    JVTO-Bench provides training and test splits for model development and evaluation, while JewelTry establishes a baseline for mask-free, scale-aware jewelry VTON.

    Source: Introduction · p. 2

    JewelTry provides relative rather than precise absolute scale control, and extreme jewelry dimensions may not render at their exact perceptual scale.

    Source: Discussion and limitation · p. 7

Abstract

from arXiv · show

Virtual try-on (VTON) enables customers to visualize how fashion products appear when worn and has become an important technology for online shopping. While recent advances have substantially improved garment VTON, jewelry remains a challenging and underexplored category due to its small size, rigid structure, and sensitivity to fine-grained visual details. Realistic jewelry VTON requires not only faithful appearance transfer but also accurate scale and placement relative to the wearer. Existing jewelry VTON methods typically rely on mask guidance, whereas mask-free approaches lack explicit guidance for modeling the product scale. To bridge this gap, we introduce JVTO-Bench, a benchmark dataset for scale-faithful jewelry VTON, providing reference source target triplets with real-world product-scale annotations across four major jewelry categories. Building upon this benchmark, we propose JewelTry, a mask-free diffusion framework for scale-aware jewelry VTON. JewelTry incorporates a scale adapter that encodes product dimensions into a scale token, enabling the model to learn scale relationships between jewelry items and surrounding human anatomy in-context. To further improve jewelry consistency, we introduce a single-directional condition attention mechanism and an attention refinement loss that preserve both coarse geometry and fine-grained structural details of the reference jewelry. Extensive experiments show that JewelTry achieves a balance among visual fidelity, background preservation, object consistency and scale accuracy, establishing a strong baseline for mask-free, scale-aware jewelry virtual try-on.

Introduction · Related Work

Jewelry virtual try-on must preserve fine detail while placing small, rigid items at plausible scales and locations, but existing benchmarks and methods leave this mask-free task under-supported. The paper introduces JVTO-Bench and JewelTry to address these gaps.

Introduction

  • Jewelry VTON is challenging because small, rigid, detailed objects require accurate scale, precise placement, and faithful geometry, while existing jewelry methods often use masks and general frameworks infer scale implicitly (Miao et al. 2025; Feng et al. 2025).Source: Introduction · p. 1
  • JVTO-Bench provides reference-source-target triplets with real-world product-scale annotations across four jewelry categories, plus training and test splits for mask-free development and evaluation.Source: Introduction · p. 2
  • JewelTry encodes product dimensions in inches into a scale token, helping the model learn scale relationships between jewelry and nearby anatomy without masks or explicit geometric supervision.The relevant anatomical regions include ears, fingers, wrists, and necks.Source: Introduction · p. 2
  • Single-directional condition attention protects reference jewelry structure from noisy latent features, while attention refinement loss guides condition-token interactions toward the target try-on region.Together, these mechanisms target coarse geometry and fine-grained appearance consistency.Source: Introduction · p. 2
  • The paper reports extensive experiments and presents JewelTry as a state-of-the-art baseline for mask-free, scale-aware jewelry VTON.Source: Introduction · p. 2

Related Work

  • Unlike general object-guided editing, virtual try-on must satisfy stronger geometric alignment, scale consistency, and body-region interaction constraints so wearable objects appear correctly positioned and proportioned.Source: Related Work · p. 2
  • Garment VTON progressed from U-Net reconstruction to diffusion methods and has shifted toward mask-free pipelines; omni-style VTON extends unified mask-free approaches to accessories and other object categories.The passage describes specialized adapters, semantic feature injection, transformer approaches, and inpainting strategies along this progression.Source: Related Work · p. 2
  • ShiningYourself uses masks to specify jewelry placement and scale, SparklingTogether extends mask-guided try-on to multiple accessories, and omni-style methods support mask-free jewelry without dedicated scale-faithful control (Miao et al. 2025; Xu et al. 2026b; Feng et al. 2025; Wu et al. 2025).Source: Related Work · p. 2

JVTO-Bench Dataset · Method

JewelTry introduces JVTO-Bench, a scale-annotated jewelry virtual try-on benchmark, and a mask-free framework with scale conditioning and structural-consistency mechanisms. The method combines scale tokens, asymmetric condition attention, and attention-based training losses.

JVTO-Bench Dataset

  • JVTO-Bench contains over 23K diverse images across rings, earrings, necklaces, and bracelets, organized as reference, try-off source, and try-on target triplets with scale in inches.The dataset addresses the lack of publicly available jewelry VTON data with reliable scale annotations.Source: JVTO-Bench Dataset · pp. 2–3
  • The dataset pipeline collects roughly 3M raw items across the UK, US, and India, then extracts product measurements from scale-informed images with Claude-Sonnet-4 and human validation.Sellers’ scale information may be noisy or inaccurate.Source: JVTO-Bench Dataset · p. 3
  • Cleaning filters for isolated reference jewelry and clearly visible, matched try-on items, while post-processing crops references, generates captions, and synthesizes try-off images refined using object removal and SSIM difference maps.Automatic filtering is followed by expert manual verification; synthesized sources help address the typical absence of source images without jewelry.Source: JVTO-Bench Dataset · p. 3

Method

  • JewelTry has three components: a scale adapter that projects numerical prompts into scale tokens, single-directional condition attention, and an attention refinement loss for fine-grained detail.The attention mechanism is designed to preserve jewelry structure consistency.Source: Method · p. 3

Scale Adapter

  • The scale adapter appends a token encoding physical measurements in inches to VLM text tokens, encouraging in-context learning of relative scale between jewelry and nearby anatomy.It uses semantic scale-aware guidance from a VLM rather than requiring explicit camera recovery, geometric alignment, or absolute-scale supervision.Source: Scale Adapter · p. 3

Single Directional Condition Attention

  • Single-directional attention blocks noisy latent tokens from updating jewelry tokens while retaining jewelry-to-latent guidance, protecting reference structure from condition collapse.The method targets jewelry-token interference from noisy latents, unlike masking approaches focused on spatial control, condition injection, or controllable generation.Source: Single Directional Condition Attention · p. 4

Training Loss

  • The attention refinement loss aligns an attention-derived jewelry soft mask with the ground-truth mask to improve spatial correspondence, placement, scale fidelity, and structural detail.The mask is extracted from reference-query and noisy-latent-key attention; later MMDiT blocks’ maps highlight the target jewelry region.Source: Training Loss · pp. 4–5
  • Training combines velocity prediction loss, an object-region loss weighted to jewelry, and global velocity-prediction supervision to preserve the person and overall scene.The global loss covers identity, skin tone, clothing, and background, while jewelry-specific losses focus on structure and detail.Source: Training Loss · p. 5

Experiment and Results

The section evaluates image fidelity, background preservation, jewelry consistency, and scale and placement, then compares JewelTry with baselines on JVTO-Bench and OmniTry-Bench’s jewelry subset.

Evaluation metrics

  • FID measures generated-image realism, while DINOp, LPIPSp, and SSIMp assess masked-out non-jewelry-region preservation against target images.FID compares generated and ground-truth image distributions.Source: Evaluation metrics · p. 5
  • Object consistency uses DINORef and CLIPRef against reference jewelry and DINOTar and CLIPTar against target jewelry; scale and placement use jewelry-region IoU and bounding-box log-ratio scale error.Higher IoU indicates better scale and spatial alignment, while lower scale error indicates greater scale faithfulness.Source: Evaluation metrics · p. 5

Comparison

  • The comparison uses four zero-shot baselines—Qwen-Image-Edit (Wu et al. 2025), OmniTry (Feng et al. 2025), Any-to-any TryOn, and InsertAnything —plus trained Qwen-JVTON.Qwen-JVTON is LoRA-fine-tuned on JVTO-Bench with scale information embedded in the text prompt; the other trained-on-bench baseline is evaluated on the same dataset as JewelTry.Source: Comparison · p. 5
  • Evaluation covers the JVTO-Bench test split and the jewelry subset of OmniTry-Bench.JVTO-Bench has 375 scale-annotated samples across rings (59), earrings (104), necklaces (106), and bracelets (106); the OmniTry-Bench subset has 300 paired samples without ground truths or scale annotations.Source: Comparison · p. 5

Ablation study

Ablations show that JewelTry’s components improve reference and target consistency, scale placement, and object preservation, with the full model best on object consistency while maintaining background preservation. Its scale control is relative rather than precisely absolute, and repeated generations may vary slightly.

Ablation study

  • The full JewelTry model achieves the best object-consistency metric while maintaining background preservation and improving placement IoU.Qualitatively, it preserves both coarse geometry and fine-grained jewelry details better than variants without attention refinement, single-directional condition attention, or object consistency loss.Source: Ablation study · p. 7
  • SDAttn yields the largest single DINOref gain, the scale adapter increases IoU, and Lattn sharpens alignment with the target try-on region.The scale adapter brings small decreases in DINOref and increases in DINOtar, described as expected inference variance.Source: Ablation study · p. 7

Discussion and limitation

  • Repeated inference with the same jewelry, person, and scale input may produce slight variations in generated jewelry scale.The paper notes that repeated generations converge to similar relative scales.Source: Discussion and limitation · p. 7, Figure 8 · p. 7
  • JewelTry provides relative rather than precise absolute scale control, so extreme requested dimensions may not render at their perceptually exact size.Increasing the scale token consistently enlarges jewelry, but mapping physical dimensions to image-space geometry is difficult without explicit geometric supervision; rare extreme scales in real-world jewelry data may also contribute.Source: Discussion and limitation · p. 7, Figure 9 · p. 7

Conclusion

The paper introduces JVTO-Bench and JewelTry for mask-free, scale-aware jewelry virtual try-on, targeting accurate product scale and preservation of fine-grained structure. Experiments show a balance across visual fidelity, background preservation, object consistency, and scale accuracy.

Conclusion

  • JewelTry balances visual fidelity, background preservation, object consistency, and scale accuracy, establishing a strong baseline for mask-free, scale-aware jewelry virtual try-on.The paper positions this baseline as a foundation for future research.Source: Conclusion · p. 7
  • JVTO-Bench provides triplet samples and product-scale annotations across four major jewelry categories.Source: Conclusion · p. 7
  • JewelTry uses a scale adapter to incorporate product scale, plus single-directional condition attention and attention refinement loss to improve jewelry fidelity.These designs preserve jewelry appearance and structure while rendering it at scale-aware without masks.Source: Conclusion · p. 7

1 JVTO-Bench

JVTO-Bench is a roughly 23k-sample jewelry virtual try-on dataset spanning four categories, with try-on triplets and real-world product-scale annotations. It also specifies category-specific scale dimensions and includes a selectively refined try-off image pipeline.

1 JVTO-Bench

  • JVTO-Bench contains approximately 23k samples across rings, earrings, necklaces, and bracelets, with try-on, try-off, and reference-jewelry images and product annotations.Annotations include real-world 2D scale in inches, product type, interaction captions, and target-object bounding boxes; samples also include object masks.Source: §1 · p. 10
  • The test set contains 59 ring products, 104 earring products, 106 necklace products, and 106 bracelet products, with rings retained separately because their scale annotations are more heterogeneous.Products are reserved before constructing the training split to prevent product-level overlap; the ring subset may be biased, while the other categories are approximately balanced around 100 samples each.Source: §1 · p. 10
  • JVTO-Bench uses a unified one-dimensional H scale: earring height, pendant height, or maximum ring and bracelet band width, while also providing perpendicular W dimensions.The category-specific 2D physical dimensions reflect commonly listed ecommerce measurements.Source: §1 · pp. 10–11, Figure 1 · p. 11
  • To improve try-off images, the pipeline uses object removal followed by SSIM-based changed-region masking and inpainting, applying this refinement manually only to visibly degraded Qwen results.The SSIM-based mask is not robust for all images.Source: §1 · p. 11

2 Implementation details

Implementation details specify JewelTry’s training and inference protocols, stochastic sampling, and scale-specific evaluation metric.

2.1 Training

  • JewelTry fine-tunes Qwen-Image-Edit-2511 (Wu et al. 2025) with rank-16, alpha-16 LoRA in self-attention for 200k iterations on eight NVIDIA H200 GPUs.Training uses batch size 8 and learning rate 5×10−5.Source: §2.1 · p. 11
  • Training encodes product scale as a 256-dimensional embedding combined with a jewelry-category embedding, projects it to 3584 dimensions, and drops the scale token randomly 25% of the time.Attention refinement extracts cross-attention maps from MMDiT blocks 51–60, averages across heads and feature dimensions, normalizes to [0, 1], and uses a timestep binary gate.Source: §2.1 · pp. 11–12

2.2 Inference

  • Inference uses CFG 3.0 and 50 denoising steps for the Qwen models, with benchmark-specific prompts, scale-token handling, and mask adjustments for comparisons.InsertAnything uses DINO/SAM masks, with JVTO-Bench person masks converted to enlarged bounding boxes; for OmniTry-Bench, the simplified prompt omits scale information and JewelTry drops its scale token.Source: §2.2 · p. 12

2.3 Seeds and randomness statement

  • All inference results use random seeds rather than a fixed seed to reflect stochastic inference conditions.Source: §2.3 · p. 12

2.4 Evaluation metrics

  • ScaleErr is the absolute log-ratio of predicted and ground-truth jewelry’s dominant bounding-box dimensions; zero indicates a perfect size match, and the measure is symmetric.Grounded DINO and SAM detect the jewelry region used for masking or metric calculation.Source: §2.4 · p. 12

3 Quantitative comparison across jewelry categories

On JVTO-Bench, JewelTry performs best in three jewelry categories and remains close to the top method on earrings. The category-wise results support its robustness across different jewelry scales, attachment regions, and structures.

3 Quantitative comparison across jewelry categories

  • JewelTry ranks first on rings, necklaces, and bracelets and second on earrings, with a DINOp gap of −0.0309 to the best method.Figure 5 reports DINOp for person/background preservation and CLIPTar for object consistency against ground-truth try-on images.Source: §3 · p. 12
  • Results indicate generalization across jewelry types, from localized rings and bracelets to necklaces requiring placement around complex neck and upper-body regions.Earrings remain challenging because of tiny size, complex structure, scale sensitivity, and frequent occlusion by hair or face contours, but JewelTry remains highly competitive; the results support its scale-aware conditioning and conditional fidelity design.Source: §3 · p. 12

4 Result variance and significance test · 5 Attention Refinement Loss

Significance tests report that JewelTry significantly outperforms zero-shot and trained-on-benchmark baselines on measures of object consistency, background preservation, and scale faithfulness. The attention refinement loss is motivated by block-wise condition–latent attention findings, with attention maps examined across transformer blocks.

4 Result variance and significance test

  • Against zero-shot baselines, JewelTry significantly outperforms on object consistency, background preservation, and scale-faithfulness metrics.For DINOTar, p-values versus OmniTry, Qwen-image-edit, InsertAnything, and Any2AnyTryOn are 2.81 × 10−3, 1.1 × 10−2, 3.10 × 10−22, and 3.87 × 10−47; for IoU they are 1.19 × 10−13, 1.46 × 10−15, 1.49 × 10−23, and 1.52 × 10−49; for LPIPSp they are 8.16 × 10−32, 7.76 × 10−21, 9.61 × 10−31, and 3.87 × 10−47, respectively.Source: §4 · p. 13
  • JewelTry significantly outperforms Qwen-JVTON on DINORef, LPIPSp, and IoU for object consistency, background preservation, and scale-faithfulness, respectively.The corresponding p-values are 8.64 × 10−3, 4.78 × 10−7, and 2.19 × 10−2.Source: §4 · p. 13
  • For jewelry scale faithfulness, JewelTry significantly outperforms every baseline tested, including Qwen-JVTON and four zero-shot methods.The p-values are 1.24 × 10−4 (Qwen-JVTON), 2.03 × 10−12 (OmniTry), 9.7 × 10−7 (Qwen-image-edit), 3.05 × 10−10 (InsertAnything), and 1.61×10−50 (Any2AnyTryOn).Source: §4 · p. 13
  • The significance tests indicate more balanced performance on object consistency and background preservation alongside better scale-aware ability.Source: §4 · p. 13

5 Attention Refinement Loss

  • Section 5 provides further technical details for the proposed attention refinement loss.Source: §5 · p. 13

5.1 Attention refinement loss and block-wise cross attention

  • The attention refinement loss is motivated by the empirical effectiveness of block-wise attention between conditions and latent in the MMDiT transformer.Figures 6 and 7 visualize attention maps across transformer blocks for person and reference conditions, averaged over diffusion time steps; the passage notes maps from later blocks, such as the last two rows, but does not specify their observed pattern.Source: §5.1 · p. 13

5.2 Timestep binary gate · 6 Person-conditioned Classifier-Free Guidance

JewelTry gates attention-map supervision to low-noise timesteps and uses person-conditioned classifier-free guidance to balance jewelry-related control with preservation of the target person. Qualitative results show that guidance strength affects scale responsiveness and jewelry realism.

5.2 Timestep binary gate

  • The attention refinement loss uses a binary timestep gate to supervise cross-attention maps only when the noise level σb is below τ = 0.5.The threshold corresponds to the lower half of the noise schedule; samples with σb ≥ τ contribute zero attention loss. Supervision combines BCE and Dice objectives against the ground-truth object mask.Source: §5.2 · p. 13

6 Person-conditioned Classifier-Free Guidance

  • At inference, JewelTry treats scale, text, and reference jewelry tokens as unified conditioning and applies classifier-free guidance with a shared guidance value.The scale, text, person-image, and reference-jewelry conditions are denoted cs, ctxt, cP, and cJ, respectively; w is the guidance strength.Source: §6 · p. 13
  • The guided noise prediction adds w times the difference between fully conditioned and person-only predictions to the fully conditioned prediction.The equation retains the person condition in the negative sample while dropping scale, text, and reference jewelry conditions.Source: §6 · p. 13
  • Person-conditioned CFG retains the target person structure while encouraging the model to follow the scale, text, and reference jewelry conditions.The person condition remains in the negative sample; the jewelry-related conditions are dropped there.Source: §6 · p. 13
  • CFG = 1 provides insufficient conditional guidance, making generated try-on results less responsive to the specified product scale.Figure 8 compares inference results under different CFG values.Source: §6 · p. 13
  • CFG = 10 or 15 can overamplify scale, text, and reference jewelry conditions, producing unrealistic results or jewelry structural collapse.Figure 8 indicates that a moderate CFG value balances scale controllability, jewelry fidelity, and visual realism.Source: §6 · p. 13

7 More study on single direction condition attention mechanism

This section tests whether blocking reverse attention paths to both person and jewelry tokens improves try-on consistency. It reports better object consistency but weaker background preservation and a noticeable skin-tone shift.

7 More study on single direction condition attention mechanism

  • JewelTry’s single-direction conditional attention blocks noisy latent tokens from attending to jewelry condition tokens during training and improves coarse-level jewelry consistency at inference.This section further examines the mechanism under different block-wise masking strategies.Source: §7 · p. 13
  • The study tests whether blocking reverse information flow to both person and jewelry tokens can improve background preservation and object consistency, based on a condition-collapse hypothesis.The intended try-on result should preserve the person image’s identity and appearance while remaining consistent with the reference jewelry.Source: §7 · pp. 13–14
  • The M P&J mask blocks reverse attention paths from noisy latent tokens to both person and reference jewelry tokens.The ablation integrates the mask into both Qwen-JVTON and JewelTry.Source: §7 · p. 14
  • Blocking reverse attention to both person and jewelry tokens improves object consistency, but slightly worsens background preservation with higher LPIPSp and causes a noticeable skin-tone shift.The authors attribute the shift to reduced access to person-specific appearance cues, weakening preservation of local color and illumination.Source: Table 3 · p. 13, §7 · p. 14

8 More visualization

Additional visualizations show JewelTry preserving fine-grained jewelry details and producing jewelry with stronger structural fidelity, texture preservation, and detail consistency in qualitative comparisons.

8 More visualization

  • Figure 10 shows JewelTry preserving fine-grained textual details in generated earrings and necklaces.The passage notes that these items require preserving complex topology and fine-grained textual details.Source: §8 · p. 14, Figure 10 · p. 20
  • Across JVTO-Bench and OmniTry-Bench, comparisons with OmniTry, Qwen-JVTON, Qwen-image-edit, and InsertAnything show stronger structural fidelity, texture preservation, and fine-grained detail consistency for JewelTry.OmniTry-Bench does not provide product-scale annotations; the comparisons also show faithful reference geometry and appearance with natural integration into the wearer.Source: §8 · p. 14
  • Figure 5 reports superior object consistency and balanced background preservation for JewelTry across four JVTO-Bench jewelry categories.The categories are earrings, bracelets, rings, and necklaces.Source: Figure 4 · p. 15, Figure 5 · p. 16
  • Earring and necklace visualizations show person and reference attention maps across attention blocks, averaged over inference time.The figures visualize Qnoise·KT attention maps for the person and reference conditions.Source: Figure 6 · p. 17, Figure 7 · p. 18
  • Figure 9 shows that JewelTry with M P &J improves object consistency but harms background preservation, shifts skin tone, and loses fine details in input source images.Source: Figure 9 · p. 19

Related papers

Loading 2609.16626v1…