PaperlayerView exact revision on arXiv

Source-linked AI summary

Oxygen-TryOn: Fashion-Native Foundation Model for Any-item Virtual Try-On

Yong Liu, Xiaolong Fu, Zihang Xu, Wen Xue, Xueheng Li, Lin Song, Yuan Zhang, Chuyang Zhao, Haoyang Huang, Nan Duan, Yipeng Sun, Yan Li, Simiu Gu

arXiv:2607.21694v1cs.CV

TL;DR

Existing virtual try-on methods often remain limited to one garment category or studio settings, while unconstrained references and faithful preservation of item appearance and subject identity remain challenging. Oxygen-TryOn treats try-on as multi-reference, understanding-driven generation and reports state-of-the-art consistency and realism on single-item try-on while leading multi-item composition.

  • Problem

    Many existing methods are limited to a single garment category and narrow studio settings, while unconstrained references and faithful item and subject consistency remain challenging.

    Source: §1 · p. 5

  • Method

    Oxygen-TryOn reframes try-on as multi-reference, understanding-driven generation, supported by a dedicated data engine and CPT–SFT–RL training recipe.

    Source: §1 · p. 5, §8 · p. 28

  • Results

    Across public benchmarks and Oxygen-TryOn Bench, it achieves state-of-the-art consistency and realism on single-item try-on and leads multi-item composition, outperforming proprietary and open-source systems.

    Source: §8 · p. 28

  • Takeaways & Limitations

    The authors conclude this is, to their knowledge, the first system to deliver any-item, multi-reference try-on at this fidelity.

    Source: §8 · p. 28

    The model is pretrained for at most four reference images, and five or more items increase the risk of item confusion, dropped items, and implausible layering.

    Source: §7 · p. 27

Abstract

from arXiv · show

We present Oxygen-TryOn, a unified foundation model for any-item virtual try-on. Rather than repurposing a general-purpose image editor, Oxygen-TryOn is fashion-native, built for try-on through a dedicated data engine and try-on-specific training. Given one or more reference items (clean product shots or in-the-wild worn-on photos) and a single target subject image, it synthesizes a photorealistic image of the subject wearing the items across virtually any fashion category. Prior systems handle a single garment category in a studio setting, and recent multi-reference methods remain garment-centric; in contrast, Oxygen-TryOn supports diverse items and scenarios, including full- and half-body views, a variable number of references, and free multi-item composition, while faithfully preserving both subject identity and item appearance. Instead of mask-based inpainting, we reformulate try-on as a multi-reference, understanding-driven generation task. We build a data engine that collects, manufactures, annotates, and filters high-quality try-on data at scale, and design a three-stage recipe of continued pre-training (CPT), supervised fine-tuning (SFT), and reinforcement learning (RL). The RL stage uses a hybrid reward combining an in-house try-on reward model with a proprietary, rubric-guided general-purpose model, jointly supervising fine-grained consistency and instruction-level quality. It also follows general editing instructions (e.g., pose changes) in the same pass. Across public benchmarks and our in-house Oxygen-TryOn Bench, it achieves state-of-the-art consistency and realism on single-item try-on and leads on multi-item try-on, matching or surpassing both leading proprietary systems (Nano Banana Pro, GPT-Image-2, Seedream5 Lite) and open-source models (FLUX.2).

1 Introduction

Oxygen-TryOn reformulates virtual try-on as multi-reference, understanding-driven generation and combines a dedicated data engine with staged training. It supports diverse items and scenarios, and achieves leading consistency and realism across single-item and multi-item try-on.

1 Introduction

  • Existing systems are limited by narrow garment categories and studio settings, difficulty handling unconstrained worn-on references, and persistent item- and subject-consistency challenges.The introduction also notes that the closest systems are proprietary and that open-source models lack a comparable, extensible full pipeline.Source: §1 · p. 5
  • The model treats try-on as multi-reference, understanding-driven generation rather than mask-based inpainting, with variable item counts and optional editing instructions in one pass.This formulation requires no explicit garment-agnostic mask and supports holistic reasoning about layering, occlusion, and item-subject interaction.Source: §1 · p. 5
  • Oxygen-TryOn achieves state-of-the-art consistency and realism across public and in-house benchmarks, outperforming proprietary and open-source baselines in single-item and multi-item try-on.The comparisons cover Nano Banana Pro, GPT-Image-2, Seedream5 Lite, and FLUX.2.Source: §1 · pp. 5–6
  • Oxygen-TryOn supports arbitrary combinations of garments, accessories, shoes, and bags from product shots or worn-on photos, across full-body and half-body views.The system is described as preserving fine-grained item appearance and subject identity.Source: §1 · pp. 5–6
  • Built on JoyAI-Image-Edit, Oxygen-TryOn combines a dedicated high-quality data engine with a three-stage CPT–SFT–RL recipe documented as a reproducible training route.The data engine collects, manufactures, annotates, and filters try-on data at scale.Source: §1 · pp. 5–6

2 Data Engine for Any-item Virtual Try-On

Oxygen-TryOn addresses the scarcity of aligned try-on triplets with a scalable data engine that converts heterogeneous fashion images into unified training examples. The resulting dataset spans diverse wearable items, subjects, and scenarios through collection and filtering, structured annotation, and constrained try-on pairing.

2 Data Engine for Any-item Virtual Try-On

  • The data engine converts heterogeneous fashion images into item–subject–result triplets through collection and filtering, annotation and captioning, and try-on pairing.Each triplet can use one or multiple item references and a full-body subject image or local body region.Source: §2 · p. 6

2.1 Data Collection & Filtering

  • The pipeline combines e-commerce, open-domain, and synthetic images, then filters more than 50 million raw images for quality, content, matching, and triplet consistency.Synthetic data expands long-tail categories, local body regions, rare wearing combinations, and challenging pose-view configurations.Source: §2.1 · pp. 6–7

2.2 Annotation and Captioning

  • Structured annotations capture product categories and attributes, subject characteristics, and compatible body regions, with consistency checks producing unified metadata and bilingual descriptions.Product attributes include color, material, texture, design, and style; subject labels include visible regions, pose, viewpoint, and body shape.Source: §2.2 · p. 8

2.3 Try-on Pairing

  • Four complementary pairing pipelines combine real and synthetic products and subjects, while constrained multi-item sampling models plausible co-occurrence, layering, and outfit compatibility.The pipelines range from real products paired with real dressed-person images to fully synthetic compositions for open-ended scenarios.Source: §2.3 · p. 8

2.4 Dataset Overview and Statistics

  • The dataset covers 207 wearable-item category mappings and diverse full-body, half-body, and local-region subjects, supporting single- and multi-item try-on across varied poses and scenes.Its unified triplet data combines real supervision with synthetic augmentation to support cross-category, cross-body-region, and open-domain generalization.Source: §2.4 · pp. 9–10

3 Model and Training

Oxygen-TryOn adapts the pretrained JoyAI-Image-Edit foundation model for any-item virtual try-on, combining an MLLM, VAE, and MMDiT. The section describes its multi-reference adaptation and three-stage training recipe.

Model and Training

  • The model combines an MLLM, VAE, and MMDiT, as illustrated in Figure 6.It is built on JoyAI-Image-Edit [35] and initialized from that model’s pretrained weights.Source: §3 · p. 10
  • Initializing from JoyAI-Image-Edit [35] lets Oxygen-TryOn inherit a general-purpose multimodal prior rather than train a task-specific model from scratch.The inherited prior supports scene understanding, instruction parsing, and high-fidelity synthesis, which the model is specialized to use for any-item try-on.Source: §3 · p. 10
  • The section covers adaptation to the multi-reference setting and a three-stage recipe for turning the foundation model into an any-item try-on model.It first describes the architecture and multi-reference adaptation, then presents the training recipe.Source: §3 · p. 10

3.1 Model Architecture

Oxygen-TryOn combines multimodal understanding, latent image encoding, and diffusion generation, with conditioning designed to handle heterogeneous references and a variable number of items.

3.1.1 Inherited Architecture

  • The inherited pipeline uses an MLLM to interpret the subject, references, and instruction, a VAE to encode image details, and an MMDiT to fuse semantics, image tokens, and noise for synthesis.The MLLM resolves item-to-body-region relationships; the VAE preserves details such as textures, prints, and logos.Source: §3.1.1 · p. 11

3.1.2 Multimodal Large Language Model

  • Qwen3-VL-8B-Instruct [1] parses and aligns the try-on inputs, and its final-layer hidden states condition generation while resolving heterogeneous references and physical layering.This supports distinguishing clean product shots from worn-on photos and associating each item with the correct body region.Source: §3.1.2 · p. 11

3.1.3 Variational Auto-Encoder and Multimodal Diffusion Transformer

  • A Wan-2.1-VAE [37] preserves high-frequency garment details, while a 16B-parameter MMDiT attends across shared-sequence target and reference latents during denoising.The shared token sequence supports an arbitrary number of reference items.Source: §3.1.3 · p. 11

3.1.4 Multi-Reference Conditioning

  • Multi-reference conditioning places noisy target latents and clean reference latents in separate temporal MRoPE intervals, using natural-language reference positions and instructions—not dedicated tags or category signals—to convey identity and placement.MLLM semantic tokens convey item identity and wearing intent, separating item appearance in VAE tokens from semantics and placement in MLLM tokens.Source: §3.1.4 · p. 11

3.2 Training

Oxygen-TryOn is trained in three stages: continued pre-training adapts a general-purpose foundation to try-on, supervised fine-tuning builds broad try-on capability, and reinforcement learning targets consistency, realism, and instruction following.

3.2.1 Continued Pre-Training

  • CPT continues JoyAI-Image-Edit training on equal shares of general-purpose and try-on data, adapting to try-on while retaining broad generation and editing ability.The try-on data teaches reference-item appearance, body-region placement, and wearing relations; the resulting checkpoint initializes SFT.Source: §3.2.1 · p. 12, §3.2 · p. 12

3.2.2 Supervised Fine-Tuning

  • SFT specializes the CPT checkpoint on paired data spanning product-to-person, person-to-person, multi-item, accessory, shoe, bag, and mask-free try-on.Structured supervision maps references and the subject to the dressed result, teaching item-to-region correspondence while preserving identity and background and supporting plausible item–subject interactions. Dynamic sequence packing isolates samples with a blockdiagonal mask to reduce padding waste and improve throughput.Source: §3.2.2 · pp. 12–13

3.2.3 Reinforcement Learning

  • RL prompts are curated and balanced across instruction, reference type, scene complexity, and reference count to avoid overfitting to head categories or easy cases.Human annotation and model-based filtering select high-quality samples, including prompts with or without editing directives and clean-product or worn-on references.Source: §3.2.3 · p. 13
  • The in-house 8B VLM, built on Qwen3-VL-Instruct [1], averages three reward heads for item consistency, face identity consistency, and visual quality.It is fine-tuned on about 100k human-annotated pairwise comparisons using an uncertainty-aware ranking loss that models reward means and variances to account for subjective, near-tie, or noisy preferences.Source: §3.2.3 · p. 13
  • Gemini 3.1 Pro rates six anchored criteria across consistency and visual quality, aggregating within aspects by harmonic mean and across aspects by geometric mean.The criteria cover instruction, identity, and item consistency, plus wearing plausibility, aesthetics, and artifact absence; the aggregation penalizes imbalance and helps curb reward hacking.Source: §3.2.3 · p. 14
  • RL combines a try-on-specific reward model with a rubric-guided Gemini judge to supervise fine-grained consistency and holistic instruction-level quality.The two rewards are fused by weighted average; policy optimization uses DiffusionNFT [46] and group-relative rewards, improving consistency and stability under challenging poses, occlusions, and dense multi-item combinations.Source: §3.2.3 · pp. 13–14

3.3 Inference and Prompt Design

Oxygen-TryOn uses lightweight prompt design to make reference interpretation and composition predictable without changing the model’s network or weights. Structured instructions map items to body regions and can combine try-on with editing directives in one generation pass.

3.3 Inference and Prompt Design

  • At inference, the textual instruction, subject image, and reference set drive generation, with wording governing how references are interpreted, mapped to body regions, and composed.The prompt design aims to expose the model’s capabilities predictably without changing its network or weights.Source: §3.3 · p. 14
  • The structured instruction separates wearing intent from optional edits and specifies each reference item’s target region or layering order for unambiguous composition.An editing directive, such as changing the pose, is applied in the same generation pass.Source: §3.3 · p. 14
  • Prompts use [garment_image], [subject_image], [item_type], and optional [edit_directive] slots, repeated for each item and bound to its source by image index.Item types include tops, pants, dresses, shoes, bags, and accessories.Source: §3.3 · p. 14
  • Templates preserve specified subject attributes unless an explicit editing directive overrides them; an enabled prompt enhancer predicts task and image roles before rewriting the prompt canonically.Preserved attributes can include facial features, hairstyle, body shape, posture, background, and lighting.Source: §3.3 · p. 15
  • Natural-language references such as “Picture 1” disambiguate multi-item placement, while ordering product or garment shots first and worn-on references last best matches training and stabilizes placement.Placement is controlled by the instruction rather than reference order alone.Source: §3.3 · p. 15

4 Experiments

Across academic, public multi-reference, and real-world benchmarks, Oxygen-TryOn delivers strong try-on quality, with especially high deployment usability and human-rated subject consistency. Qualitative comparisons show it handles diverse items and scenes while more reliably preserving identity, item fidelity, and plausible placement than proprietary editing systems.

4.1 Evaluation Benchmark

  • Oxygen-TryOn is evaluated using four complementary benchmarks, progressing from constrained academic settings to real-world conditions, including diverse public TStars-VTON scenarios and a 1,000-test-sample real-world benchmark evenly split between two reference scenarios.Source: §4.1 · p. 15

4.2 Quantitative Results

  • Oxygen-TryOn leads paired DressCode and VITON-HD metrics and all five re-evaluated TStars-VTON single-item dimensions, scoring 9.36 overall versus 8.77 for the next-best system.On paired DressCode, it scores FID 1.932, KID 0.387, SSIM 0.936, and LPIPS 0.034; on VITON-HD, FID 3.941, KID 0.491, SSIM 0.914, and LPIPS 0.050. On unpaired comparisons, it leads DressCode at FID 5.155 / KID 0.945 and VITON-HD FID 9.145, but ranks second on VITON-HD KID (1.171 versus FastFit*’s 0.973).Source: §4.2 · pp. 16–17, Table 2 · p. 17, Table 3 · p. 18
  • On TStars-VTON multi-item try-on with up to four references, Oxygen-TryOn leads four of five dimensions, including overall quality at 8.41 versus 8.19 for Seedream5 Lite.Item fidelity is the exception: Oxygen-TryOn scores 8.03, below Nano Banana Pro (8.50), Seedream5 Lite (8.49), and GPT-Image-2 (8.33); five-or-more-reference layering remains challenging.Source: §4.2 · pp. 18–19, Table 4 · p. 18
  • Oxygen-TryOn ranks first on 8 of 10 Oxygen-TryOn Bench metrics, with usability rates of 86.79% for Cloth-to-Model and 85.43% for Model-to-Model.These rates exceed GPT-Image-2’s 80.35% and 77.58%, respectively; aesthetics is the exception, where Oxygen-TryOn ranks second on Cloth-to-Model and third on Model-to-Model.Source: Table 5 · p. 19, §4.2 · p. 19
  • SFT establishes broad try-on competence, while RL further improves consistency and usability, raising Model-to-Model usability by 4.99 points to 85.43%.SFT increases usability from 67.95% to 85.95% on Cloth-to-Model and from 70.18% to 80.44% on Model-to-Model; RL then improves item consistency and overall scores on both splits.Source: §4.2 · p. 19, Table 6 · p. 20

4.3 Human Evaluation

  • In a 985-sample human study, Oxygen-TryOn achieves the best subject-consistency score (3.7492) and overall score (3.5502), narrowly ahead of GPT-Image-2 (3.5375).GPT-Image-2 leads item consistency (3.2951 vs. 3.2250) and aesthetics (3.8240 vs. 3.8025); Nano Banana Pro trails on every dimension.Source: §4.3 · p. 20, Table 7 · p. 20

4.4 Qualitative Results

  • Qualitative examples show Oxygen-TryOn handling diverse items, subjects, visual domains, and multi-item outfits while preserving appearance and resolving layering and occlusion.The demonstrations include product shots and worn-on references, real people and artwork, dense multi-item compositions, and edit-augmented instructions; matched comparisons show more reliable identity preservation, item fidelity, and plausible placement than proprietary systems.Source: §4.4 · pp. 21, 26

5 Applications and Capabilities

Oxygen-TryOn extends beyond single-item try-on to support free outfit composition, instruction-based editing in the same pass, and applications to non-standard subjects. These capabilities preserve item and subject properties while broadening the settings in which try-on can be used.

5.1 Free Multi-Item Composition

  • Oxygen-TryOn composes an arbitrary number of mixed-category items into one outfit, resolving layering and occlusion while preserving item appearance and the subject’s identity, pose, and background.This enables full-outfit OOTD-style exploration rather than item-by-item replacement.Source: §5.1 · p. 26

5.2 Built-in General Editing

  • During try-on generation, Oxygen-TryOn can follow editing instructions such as pose adjustments in the same pass, without a separate editing model or second pass.The model’s initialized general-purpose foundation model and editing-augmented try-on training support this built-in capability.Source: §5.2 · p. 26

5.3 Cross-Domain Generalization

  • Oxygen-TryOn applies garments and accessories to stylized 3D avatars, illustrated characters, and statues while respecting their original artistic style and geometry.The paper attributes this cross-domain capability to generalizable representations of items and wearing relations rather than a narrow human-pose prior.Source: §5.3 · p. 26

6 Related Work

Oxygen-TryOn reframes virtual try-on as mask-free, understanding-driven multi-reference generation for arbitrary fashion items, specializing a general-purpose model and aiming to make strong try-on quality reproducible from the open side.

6.1 Image-Based Virtual Try-On

  • Prior try-on progressed from GAN warping and compositing [21] [38] to diffusion fusion methods [9] [10] [24] [26] [43] [47], but remains largely garment-centric.Wear-Any-Way and Tunnel Try-on [6] [44], and Any2AnyTryon and FastFit [11] [20], relax control or reference constraints but do not jointly handle heterogeneous references, non-garment items, and arbitrary outfits. Oxygen instead uses mask-free, understanding-driven multi-reference generation to reason about item identity, placement, layering, and optional edits.Source: §6.1 · pp. 26–27

6.2 General-Purpose Generation and Editing Models

  • General-purpose generators provide photorealistic synthesis and semantic understanding, but lack try-on’s fine-grained consistency; Oxygen specializes one with fashion data, multi-reference conditioning, SFT, and RL.Examples include Qwen-Image, FLUX.2, and HunyuanImage [5] [41], and Nano Banana Pro, GPT-Image-2, and Seedream5 Lite; strong general models degrade when several references must be honored.Source: §6.2 · p. 27

6.3 Toward Open and Reproducible Try-On

  • Oxygen-TryOn aims to match closed-source quality while releasing weights, data-engine design, training recipe, and inference protocol for reproducibility and auditing.Proprietary systems are important comparators but disclose little about their training or evaluation, while open-source task-specific systems remain confined to narrow settings.Source: §6.3 · p. 27

7 Limitations and Future Work

Oxygen-TryOn delivers high-fidelity try-on across single- and multi-item scenarios, but its current capabilities have limits that future work will address.

Scaling to more reference items

  • The model is pretrained for at most four reference images and becomes more prone to item confusion, dropped items, and implausible layering with five or more.Future work will target dense, high-cardinality outfit data, stronger multi-reference conditioning, and a larger-capacity base model.Source: §7 · p. 27

Distilled, accelerated variant

  • Multiple inference denoising steps limit throughput, so the authors plan a distilled, accelerated variant that preserves item and subject consistency with minimal quality loss.The variant will markedly reduce sampling steps for faster, cheaper try-on.Source: §7 · pp. 27–28

8 Conclusion

Oxygen-TryOn reframes virtual try-on as multi-reference, understanding-driven generation, using a dedicated data engine and CPT–SFT–RL training to preserve item appearance and subject identity. It achieves leading results across single- and multi-item try-on and retains instruction-based editing in the same generation pass.

8 Conclusion

  • The model treats try-on as multi-reference, understanding-driven generation, synthesizing photorealistic results from product shots or worn-on photos while preserving item appearance and subject identity.It supports single- and multi-item, full- and half-body scenarios, and combines an MLLM with an MMDiT initialized from JoyAI-Image-Edit pretrained weights.Source: §8 · p. 28
  • A dedicated data engine and three-stage CPT–SFT–RL recipe enable try-on, while Oxygen-TryOn also follows general editing instructions such as pose changes in the same generation pass.The hybrid RL reward combines an in-house try-on reward model with a rubric-guided Gemini 3.1 Pro judge.Source: §8 · p. 28
  • Oxygen-TryOn achieves state-of-the-art consistency and realism on single-item try-on and leads multi-item composition across public benchmarks and Oxygen-TryOn Bench.It surpasses Nano Banana Pro, GPT-Image-2, Seedream5 Lite, and FLUX.2; the authors describe it as the first system, to their knowledge, to deliver any-item, multi-reference try-on at this fidelity.Source: §8 · p. 28

Related papers

Loading 2607.21694v1…