PaperlayerView exact revision on arXiv

Source-linked AI summary

TryOnReward: Learning Foveated Consistency for Reinforcement Fine-Tuning of Virtual Try-On

Xueheng Li, Yong Liu, Xiaolong Fu, Wen Xue, Chengjun Xie, Yipeng Sun, Yan Li, Simiu Gu

arXiv:2609.13259v1cs.CVcs.AI

TL;DR

Virtual try-on scoring must capture human preferences while preserving fine-grained garment and person details that existing metrics and generic judges often miss. TryOnReward grounds three quality dimensions in relevant regions and combines preference ranking with score regression. It reports improved human-preference alignment and human-preferred RFT outputs across baselines.

Abstract

from arXiv · show

Virtual Try-On (VTON) aims to dress a person with the reference garment, producing visually reasonable results aligned with human preferences. Turning this preference-oriented goal into an actionable objective relies on a scoring function aligned with human taste. However, classic fidelity metrics exhibit weak correlation with human judgments, and generic VLMs fail to provide the discriminative granularity demanded by try-on quality evaluation, which hinges on faithfully preserving garment and person details. This shortcoming is further exacerbated in the reinforcement fine-tuning (RFT) optimization and leads to severe reward hacking. To this end, we present TryOnReward, a fine-grained reward model tailored for VTON. Built on a vision-language backbone, it adopts a foveation calibration objective that grounds each quality dimension in the relevant region to avoid global shortcut learning. Meanwhile, TryOnReward jointly optimizes pairwise preferences and per-dimension quality scores via margin-aware supervision, leveraging both relative and absolute quality signals. For model training and evaluation, we build TryOnReward-100K, a human-annotated per-dimension rating dataset, alongside TryOn-Bench and TryOnRewardBench, two benchmarks covering diverse real scenarios. Extensive experiments confirm that TryOnReward significantly outperforms generic judges in human preference alignment, and when serving as the RFT reward function, it consistently yields human-preferred try-on results across multiple baselines.

Introduction

VTON quality scoring must capture human preferences while preserving fine-grained garment and person details, but existing metrics and general-purpose judges often miss those details. TryOnReward addresses this with dimension-specific visual grounding and joint preference-ranking and score supervision.

Introduction

  • Human-aligned scoring is central to both benchmarking try-on generations and supplying the optimization signal for reinforcement fine-tuning.The paper frames preferred outputs, rather than mere plausibility, as the advancing frontier for VTON.Source: Introduction · p. 1
  • Holistic scorers miss region-level garment and identity details, while generic rewards can favor globally attractive outputs with garment mismatch or identity drift during RFT.Classic fidelity metrics correlate poorly with human judgment, and general-purpose reward models and VLM judges miss subtle try-on failures.Source: Introduction · pp. 1–2
  • Pairwise learning captures preference direction but not the quality gap, whereas pointwise regression conveys absolute quality but is sensitive to annotation subjectivity and imposes no explicit ordering.The two signals motivate a unified formulation.Source: Introduction · p. 2
  • TryOnReward combines dimension-specific foveation calibration with pairwise preference learning and pointwise score regression to model ranking, rating semantics, and preference strength.It evaluates garment consistency, identity consistency, and visual quality, and grounds each dimension in its relevant region.Source: Introduction · p. 2
  • TryOnReward aligns better with human preferences than general judges and, as an RFT reward, yields human-preferred try-on results across baselines.The paper builds TryOnReward-100K, TryOn-Bench, and TryOnRewardBench to support training and systematic evaluation.Source: Introduction · p. 2

Related Work

Related work covers progress in virtual try-on generation and general reward modeling, while identifying a need for VTON-specific, fine-grained evaluation and supervision.

Virtual Try-On

  • Recent VTON work spans diffusion-based garment integration, newer transformer and unified frameworks, and RFT post-training, but lacks a dedicated fine-grained reward model for task-specific consistency.Source: Related Work · p. 2

Reward Model

  • General reward models address image generation and editing, but VTON requires multi-input reasoning and region-level evaluation of garment and identity consistency.The paper positions its VTON-specific model as combining task-aware supervision with unified pairwise–pointwise learning and foveation calibration.Source: Related Work · p. 2

Dataset Curation and Construction

TryOnReward-100K is built from curated try-on candidates and human preference annotations, while TryOn-Bench and TryOnRewardBench cover complementary try-on settings.

Dataset Curation and Construction

  • The four-stage TryOnReward-100K pipeline collects and filters source data, generates candidates, constructs pairs, and annotates preferences.Candidates are generated with four image-editing models and filtered before pairing.Source: Dataset Curation and Construction · pp. 2–3, Figure 2 · p. 3
  • 99,286 high-quality preference pairs form TryOnReward-100K, after candidates receive per-dimension ratings from at least nine annotators and discrepant pairs are removed.The three dimensions are garment consistency, identity consistency, and visual quality; ratings use a 1–5 scale.Source: Dataset Curation and Construction · p. 3
  • TryOnRewardBench adds multi-model generations and human preferences to assess reward-model alignment, alongside the complementary TryOn-Bench.The benchmarks cover Cloth2Model and Model2Model settings.Source: Dataset Curation and Construction · p. 3
  • Model2Model comprises 7,045 pairs with source garments depicted on another person, making it more challenging than Cloth2Model because models must disentangle garment appearance from the source wearer and perform cross-person appearance transfer.Source: Dataset Curation and Construction · p. 3

Methods

TryOnReward learns garment consistency, identity consistency, and visual quality from human ratings and pairwise preferences. Its architecture grounds dimension-specific rewards in visual regions and combines uncertainty-aware ranking with absolute and margin-aware score supervision.

Methods

  • Each training pair includes garment-person inputs and human ratings for two candidates across garment consistency, identity consistency, and visual quality.The ratings supervise relative preference ordering, absolute rating semantics, and preference strength.Source: Methods · p. 3

Model Architecture

  • A shared VLM representation feeds separate reward and score heads for each dimension, with a Gaussian reward distribution and five-level rating predictions.At inference, the reward mean supplies the dimension-wise reward; score heads are used only in training and add no ranking or online RFT overhead.Source: Model Architecture · pp. 3–4

Foveation Calibration Objective

  • Mask priors target garment, person, or full-image regions by dimension, with spatial supervision used only during training.The objective increases attention to relevant regions without fixing its distribution, and averages losses across final-layer attention heads.Source: Foveation Calibration Objective · p. 4

Multi-Dimensional Pairwise Ranking

  • Pairwise preferences are learned as dimension-wise probabilities by marginalizing over the preferred and rejected candidates’ latent reward distributions.This incorporates both expected reward differences and prediction uncertainty; the loss is the negative log-likelihood of the observed preference.Source: Multi-Dimensional Pairwise Ranking · p. 4

Margin-Aware Pointwise Regression

  • A five-level rating head anchors absolute quality, while ordinal score-gap regression supplies a preference-strength-proportional margin.The rating distribution uses cross-entropy; Smooth L1 is applied to the normalized residual between predicted and annotated score gaps.Source: Margin-Aware Pointwise Regression · p. 5

Experiments

Experiments assess human-preference alignment, reward-guided RFT, and out-of-domain transfer; reported results show stronger preference alignment, improved RFT policy scores, and competitive transfer to unseen benchmarks.

Implementation Details

  • The model is fine-tuned across all parameters from Qwen3VL-8B-Instruct, using 672×672 inputs with original aspect ratios preserved.Training uses a 2 × 10−6 learning rate and total batch size of 16.Source: Implementation Details · pp. 5–6

Evaluation Setting

  • Preference alignment is evaluated on TryOnRewardBench with pairwise and strict K-way accuracy; online DiffusionNFT RFT is evaluated on TryOn-Bench, with transfer tested on held-out DressCode and VITON-HD.The RFT study uses Qwen-Image-Edit-2511 and fine-tuned VTON-SFT policies, with GPT-5 as an independent automatic evaluator.Source: Evaluation Setting · pp. 6–7

Main Results

  • 74.61% and 73.39% overall pairwise accuracy on Cloth2Model and Model2Model, with average K-way accuracy of 43.61% and 44.62%, respectively, surpass proprietary baselines.The reported K-way scores substantially exceed the strongest proprietary baselines.Source: Main Results · p. 7
  • TryOnReward improves both policy models on TryOn-Bench; Qwen-Image-Edit-2511’s Model2Model overall score rises from 3.047 to 3.564.This gain is particularly pronounced for cross-person try-on.Source: Main Results · p. 7
  • On held-out DressCode and VITON-HD, the optimized model remains competitive with methods trained on those benchmarks, indicating transfer to unseen data distributions.Neither benchmark was used during post-training.Source: Table 3 · p. 6, Main Results · p. 7

Ablations

  • Preference-supervised reward learning improves human alignment; the score head adds pointwise quality semantics and preference strength, while FCO improves pairwise and K-way accuracy but slightly lowers Visual Quality accuracy.The reported FCO gains mainly strengthen garment and identity consistency through localized cross-image grounding.Source: Table 4 · p. 6, Ablations · p. 7

Qualitative Results

  • TryOnReward better preserves garment structure and person identity while maintaining natural quality; its FCO attention focuses on dimension-relevant regions and cross-image correspondences.The compared general-purpose reward models tend to improve visual plausibility at the expense of VTON-specific consistency.Source: Figure 4 · p. 6, Qualitative Results · p. 7

Conclusion

TryOnReward grounds garment consistency, identity consistency, and visual quality in relevant image regions while combining pairwise preference learning with pointwise score regression. Experiments report improved human-preference alignment and effective reward signals for reinforcement learning with try-on models.

Conclusion

  • TryOnReward decomposes human preferences into garment consistency, identity consistency, and visual quality, grounding reward predictions in task-relevant regions through dimension-specific foveation calibration.The approach also unifies pairwise preference learning and pointwise score regression to capture ranking consistency, rating semantics, and preference strength.Source: Conclusion · p. 7
  • TryOnReward-100K, TryOn-Bench, and TryOnRewardBench support training and systematic evaluation of try-on reward models.Source: Conclusion · p. 7
  • Experiments demonstrate better human-preference alignment and effective reward signals for reinforcement learning with try-on models.Source: Conclusion · p. 7
Loading 2609.13259v1…