PaperlayerView exact revision on arXiv

Source-linked AI summary

Oxygen-TryOn: Fashion-Native Foundation Model for Any-item Virtual Try-On

Yong Liu, Xiaolong Fu, Zihang Xu, Wen Xue, Xueheng Li, Lin Song, Yuan Zhang, Chuyang Zhao, Haoyang Huang, Nan Duan, Yipeng Sun, Yan Li, Simiu Gu

arXiv:2607.21694v1cs.CV

TL;DR

Existing virtual try-on methods often remain restricted to single garment categories, studio settings, or pristine references. Oxygen-TryOn reformulates try-on as multi-reference generation and achieves state-of-the-art consistency and realism while leading multi-item composition.

  • Problem

    Existing virtual try-on methods are limited in category, setting, reference type, and free multi-item composition, while preserving item and subject consistency remains challenging.

  • Method

    Oxygen-TryOn reformulates try-on as multi-reference, understanding-driven generation for heterogeneous items and uses a dedicated data engine with CPT–SFT–RL training.

  • Results

    Oxygen-TryOn achieves state-of-the-art consistency and realism on single-item try-on, surpasses proprietary and open-source baselines, and leads multi-item composition.

  • Takeaways & Limitations

    Oxygen-TryOn delivers any-item, multi-reference try-on across heterogeneous items and viewing scenarios at reported high fidelity.

  • Takeaways & Limitations

    Multi-item use is focused on up to four references; five or more can cause item confusion, dropped items, and implausible layering.

Abstract

from arXiv · show

We present Oxygen-TryOn, a unified foundation model for any-item virtual try-on. Rather than repurposing a general-purpose image editor, Oxygen-TryOn is fashion-native, built for try-on through a dedicated data engine and try-on-specific training. Given one or more reference items (clean product shots or in-the-wild worn-on photos) and a single target subject image, it synthesizes a photorealistic image of the subject wearing the items across virtually any fashion category. Prior systems handle a single garment category in a studio setting, and recent multi-reference methods remain garment-centric; in contrast, Oxygen-TryOn supports diverse items and scenarios, including full- and half-body views, a variable number of references, and free multi-item composition, while faithfully preserving both subject identity and item appearance. Instead of mask-based inpainting, we reformulate try-on as a multi-reference, understanding-driven generation task. We build a data engine that collects, manufactures, annotates, and filters high-quality try-on data at scale, and design a three-stage recipe of continued pre-training (CPT), supervised fine-tuning (SFT), and reinforcement learning (RL). The RL stage uses a hybrid reward combining an in-house try-on reward model with a proprietary, rubric-guided general-purpose model, jointly supervising fine-grained consistency and instruction-level quality. It also follows general editing instructions (e.g., pose changes) in the same pass. Across public benchmarks and our in-house Oxygen-TryOn Bench, it achieves state-of-the-art consistency and realism on single-item try-on and leads on multi-item try-on, matching or surpassing both leading proprietary systems (Nano Banana Pro, GPT-Image-2, Seedream5 Lite) and open-source models (FLUX.2).

1 Introduction

Oxygen-TryOn is a unified, fashion-native foundation model that reformulates virtual try-on as multi-reference, understanding-driven generation for arbitrary fashion items and compositions. Built with dedicated data and CPT–SFT–RL training, it achieves leading consistency and realism across diverse try-on scenarios.

  • Any-item try-on: Oxygen-TryOn accepts a target subject and heterogeneous references—clean product shots or worn-on photos—for garments, accessories, shoes, and bags.It supports full-body and half-body views while preserving item appearance and subject identity.
  • Motivation: Existing methods often target one garment category in studio settings and assume pristine flat-lay references, limiting accessories, unconstrained inputs, and multi-item composition.These limitations motivate a professional any-item try-on system.
  • Unified formulation: Variable reference count enables single-item try-on and free multi-item composition, while optional instructions such as pose changes execute in the same generation pass without explicit garment-agnostic masks.The formulation enables holistic reasoning about layering, occlusion, and item interactions.
  • Training: The model combines JoyAI-Image-Edit with a dedicated data engine and three-stage continued pre-training, supervised fine-tuning, and reinforcement learning recipe.The data engine collects, manufactures, annotates, and filters high-quality try-on data at scale.
  • Results: Oxygen-TryOn achieves state-of-the-art consistency and realism across single-item and multi-item try-on, surpassing Nano Banana Pro, GPT-Image-2, Seedream5 Lite, and FLUX.2.Evaluations cover diverse garments, accessories, shoes, and bags under full-body and half-body views, including in-the-wild conditions.

2 Data Engine for Any-item Virtual Try-On

Oxygen-TryOn’s data engine converts heterogeneous fashion imagery into unified item–subject–result triplets because real-world aligned try-on data are scarce. It combines broad wearable-item and subject coverage with large-scale filtering, structured annotation, constrained pairing, and iterative refinement.

  • Motivation and Pipeline: Real-world aligned item–subject–result triplets are scarce, making large-scale supervised try-on data covering diverse categories and scenarios difficult to obtain.The engine addresses this scarcity by transforming heterogeneous fashion images into unified triplets containing one or more item references, a full-body or local-region target, and the resulting try-on image.
  • Coverage and Representation: The dataset supports full-body, half-body, and local-region subjects while unifying single-item, multi-item, real-supervision, and synthetic-augmentation samples in one triplet representation.This broadens try-on beyond conventional garment-focused datasets and supports diverse wearing tasks.
  • Collection and Filtering: The engine collects e-commerce, open-domain, and model-generated synthetic imagery, then progressively filters more than 50 million raw images for quality, content, matching, and triplet consistency.These sources provide high-fidelity product appearance, diverse identities and conditions, and expanded coverage of long-tail categories, local regions, and rare combinations.
  • Annotation and Pairing: Structured annotation combines VLMs, detection models, and rule-based verification to describe product attributes and subject metadata, constrain pairing, and remove semantic inconsistencies.Product annotations include category, compatible body region, color, material, texture, design, and style; subject annotations include visible regions, pose, viewpoint, apparent demographics, body shape, and background complexity.
  • Item Distribution: 207 valid category mappings cover 58 coarse-grained product groups, 151 e-commerce third-level categories, and 104 fine-grained fashion-style categories.The taxonomy spans garments and accessories including tops, bottoms, outerwear, full-body clothing, shoes, bags, hats, glasses, jewelry, and scarves.

3 Model and Training

Oxygen-TryOn is initialized from the pretrained JoyAI-Image-Edit foundation model and specialized for any-item virtual try-on. The section covers its multi-reference architecture adaptation and three-stage training recipe.

  • Model and Training: Oxygen-TryOn builds on pretrained JoyAI-Image-Edit weights, inheriting general-purpose multimodal capabilities before specialization for any-item virtual try-on.The inherited capabilities include scene understanding, instruction parsing, and high-fidelity synthesis.
  • Model and Training: The section first presents the model architecture and its adaptation to the multi-reference setting, then introduces the three-stage training recipe.These topics correspond to Sections 3.1 and 3.2, respectively.

3.1 Model Architecture

Oxygen-TryOn uses an understanding-driven, three-stage architecture that combines multimodal reasoning, latent detail preservation, and conditional diffusion. Its variable-reference design organizes heterogeneous items through shared latent tokens and temporal positions, while language specifies reference identity and category.

  • Multimodal Understanding: The MLLM jointly encodes subject and reference images with the wearing instruction to resolve item placement and relationships before generation.Qwen3-VL-8B-Instruct provides the primary conditioning signal to the MMDiT through final-layer hidden states.
  • Latent Encoding: The VAE compresses subject and reference images into latent tokens while preserving fabric textures, prints, and logos needed for faithful reconstruction.Wan-2.1-VAE supplies the latent representations used by the generative core.
  • Conditional Generation: A 16B-parameter MMDiT fuses MLLM semantic priors, VAE image tokens, and noise tokens for high-fidelity multi-condition synthesis.Its dual-stream architecture jointly models multimodal conditioning and latent tokens, attending across the encoded conditions.
  • Variable-Reference Tokenization: Oxygen-TryOn places the target and every reference in one shared token sequence, using distinct functional regions to support an arbitrary number of items.The target region begins as Gaussian noise, while reference and target latents are encoded together so the MMDiT can attend across conditions.
  • Variable-Reference Tokenization: Temporal intervals in MRoPE distinguish the target and references, while natural-language positions and instructions convey reference identity and item category without dedicated tagging tokens.The design deliberately avoids segment, index, region-type, reference-tagging, and image-identifier tokens.

3.2 Training

Oxygen-TryOn is trained through continued pre-training, supervised fine-tuning, and reinforcement learning, progressively adapting the foundation model, specializing any-item try-on, and improving consistency, realism, and instruction following. Its RL objective combines task-specific try-on preferences with rubric-guided multimodal judging.

  • Training pipeline: The three-stage pipeline uses CPT to adapt JoyAI-Image-Edit, SFT to acquire broad try-on capability, and RL to improve consistency, realism, and instruction following.CPT uses a balanced mixture of general-purpose and try-on data.
  • Supervised fine-tuning: SFT specializes the CPT checkpoint into an any-item try-on expert using high-quality paired data, targeting item fidelity, pose and shape adaptation, identity and background preservation, and plausible interactions.The conditioning includes the text instruction, subject image, and one or more reference item images.
  • Supervised fine-tuning: Dynamic sequence packing greedily combines short single- and multi-reference samples while blockdiagonal masking isolates packed subsequences, reducing padding waste without cross-sample attention.This handles heterogeneous training-example lengths.
  • Reinforcement learning: RL optimizes item–subject consistency, item fidelity, identity and background preservation, and physical plausibility under complex poses and layering, using curated, balanced prompts filtered by humans and models.The RL paradigm builds on DiffusionNFT and related reward-based diffusion optimization methods.
  • Reinforcement learning: The hybrid reward fuses an 8B in-house try-on VLM trained on about 100k human-annotated preference pairs with Gemini 3.1 Pro’s six rubric-guided criteria through a weighted average.The in-house model emphasizes item, identity, and visual quality, while Gemini provides broader instruction-level assessment and open-world generalization.

3.3 Inference and Prompt Design

Oxygen-TryOn uses structured textual prompts to control how reference items are interpreted, mapped to body regions, and composed with a subject image. The template supports optional editing directives and recommends a reference ordering that improves placement stability.

  • Structured instruction template: Structured prompts separate wearing intent from an optional editing directive, making item-to-region mappings explicit for single- and multi-item composition.Each reference item can specify its target body region or layering order.
  • Structured instruction template: The prompt template uses garment image, subject image, item type, and optional edit directive slots, repeated for each reference in multi-item try-on.Image indices bind each item to its corresponding source image.
  • Structured instruction template: Single-item prompts preserve the subject’s posture, face, and body shape while transferring the specified item from the reference image.The template explicitly instructs the model to place the item from Picture 1 onto the subject in Picture 2.
  • Reference ordering: Natural-language reference positions such as “Picture 1” and “Picture 2” disambiguate item-to-image binding without special identifier tokens.A single instruction therefore controls which item goes where rather than relying on reference order alone.
  • Reference ordering: For stable placement, product or garment shots should precede worn-on subject references, matching the training-data convention.This ordering best fits the training distribution and yields the most stable placement.

4 Experiments

Experiments evaluate Oxygen-TryOn from standard studio benchmarks to diverse real-world try-on, where it achieves strong reconstruction, consistency, usability, and human-preference results. Ablations and qualitative comparisons further show the contributions of supervised fine-tuning and robust multi-item constraint handling.

  • Evaluation setup: Four complementary benchmarks progress from constrained academic settings to unconstrained real-world conditions, including standard garment-transfer, diverse-reference, and deployment-oriented evaluation.Oxygen-TryOn Bench covers garments, jewelry, shoes, and bags in complex backgrounds, poses, layering, and full- or half-body views.
  • Standard academic benchmarks: Oxygen-TryOn ranks first on every paired DressCode and VITON-HD metric, while leading the re-evaluated comparison on DressCode and most VITON-HD unpaired metrics.It ranks second on VITON-HD unpaired KID at 1.171, behind FastFit*’s 0.973; specialized methods can report lower unpaired FID/KID in their original papers.
  • Multi-item try-on: Oxygen-TryOn leads the re-evaluated multi-item comparison on four of five dimensions for samples with up to four reference images.Five or more references under complex layering remain challenging and are identified as a primary target for future work.
  • Oxygen-TryOn Bench: 86.79% and 85.43% usability rates on Cloth-to-Model and Model-to-Model exceed GPT-Image-2’s 80.35% and 77.58%, respectively.Oxygen-TryOn ranks first on 8 of 10 metrics across the two splits, including subject consistency, item consistency, overall score, and usability rate.
  • Effect of the training recipe: 85.95% and 80.44% usability after SFT rise from CPT’s 67.95% and 70.18% on Cloth-to-Model and Model-to-Model, respectively.The ablation evaluates CPT, SFT, and RL checkpoints under both reference styles, with SFT providing the largest single jump.
  • Human preference: Oxygen-TryOn attains the best subject consistency at 3.7492 and overall score at 3.5502, narrowly ahead of GPT-Image-2’s 3.5375 in a 985-sample human evaluation.GPT-Image-2 leads item consistency at 3.2951 versus 3.2250 and aesthetics at 3.8240 versus 3.8025, while Nano Banana Pro trails on every dimension.

5 Applications and Capabilities

Oxygen-TryOn’s unified, understanding-driven design supports practical capabilities beyond standard single-item try-on. It composes arbitrary item combinations, follows editing directives during generation, and generalizes to non-standard subjects while preserving relevant visual properties.

  • Multi-item composition: Oxygen-TryOn composes an arbitrary number of mixed-category items into one coherent outfit while resolving layering and occlusion relationships.It preserves each item’s appearance and the subject’s identity, pose, and background.
  • Instruction-based editing: Oxygen-TryOn follows additional instruction-based editing directives, such as pose adjustments, during try-on generation without a separate model or second pass.This capability comes from general-foundation-model initialization and editing-augmented try-on training.
  • Non-standard subjects: Oxygen-TryOn applies garments and accessories to stylized 3D avatars, illustrated characters, and statues while respecting their artistic style and geometry.This generalization suggests representations of items and wearing relations beyond a narrow human-pose prior.

6 Related Work

Prior virtual try-on evolved from single-garment warping and compositing toward diffusion-based, controllable, and multi-reference systems, but remains largely garment-centric. Oxygen-TryOn instead formulates try-on as understanding-driven multi-reference generation for arbitrary fashion items and targets reproducible open development alongside strong proprietary comparators.

  • Classical and diffusion-based try-on: Classical systems recombine a target person with one reference garment using warping and compositing under assumptions about pose, category, and input layout.VITON and CP-VTON established this dominant recipe, while diffusion methods improved realism and detail preservation.
  • Recent multi-reference try-on: Recent methods improve controllability, video support, versatility, and multi-reference try-on, yet remain largely garment-centric.They do not jointly cover heterogeneous references, non-garment items, and arbitrary multi-item outfits.
  • Oxygen-TryOn positioning: Oxygen-TryOn casts try-on as understanding-driven, multi-reference generation over arbitrary fashion items rather than inpainting or warping.One generation pass jointly reasons about item identity, body-region placement, layering, and optional edits.
  • General-purpose generators: General-purpose generators offer scalable, photorealistic synthesis and instruction following, but are not tuned for try-on’s fine-grained identity, item, layering, and occlusion consistency.Try-on requires preserving the person’s identity, pose, body shape, and background while reproducing each item’s texture, print, structure, and logo.
  • Open reproducibility: Oxygen-TryOn targets strong closed-source quality while releasing weights, data-engine design, CPT–SFT–RL recipe, and inference and prompt-enhancer protocols for reproducible, auditable development.Proprietary systems remain indispensable comparators because their training data, architecture, optimization, and evaluation protocols are undisclosed.

7 Limitations and Future Work

Oxygen-TryOn still has limitations despite high-fidelity any-item try-on across single- and multi-item scenarios. Future work targets scaling beyond four references and improving inference efficiency through distillation.

  • Future directions: Oxygen-TryOn’s current limitations motivate future work on reference-item scaling and faster inference.The model already supports high-fidelity any-item try-on across diverse single- and multi-item scenarios.
  • Scaling to more reference items: Beyond four reference images, complex layering and occlusion increase item confusion, dropped items, and implausible layering.The foundation model was pretrained for at most four references, so current multi-item use and evaluation focus on up to four.
  • Distilled, accelerated variant: A distilled, accelerated variant will reduce sampling steps while preserving item and subject consistency.The goal is faster and cheaper try-on with minimal quality loss for interactive and large-scale deployment.

8 Conclusion

Oxygen-TryOn is a unified foundation model that reformulates virtual try-on as multi-reference, understanding-driven generation, preserving item appearance and subject identity. It achieves state-of-the-art single-item consistency and realism while leading multi-item composition across public benchmarks and Oxygen-TryOn Bench.

  • Model and formulation: Oxygen-TryOn is a unified foundation model for any-item virtual try-on using heterogeneous references and a single target subject image.References may be clean product shots or in-the-wild worn-on photos.
  • Model and formulation: The method reformulates try-on as multi-reference, understanding-driven generation rather than mask-based inpainting.It synthesizes photorealistic results while faithfully preserving item appearance and subject identity.
  • Results: Oxygen-TryOn attains state-of-the-art consistency and realism on single-item try-on and leads multi-item composition across public benchmarks and Oxygen-TryOn Bench.It surpasses Nano Banana Pro, GPT-Image-2, Seedream5 Lite, and FLUX.2.
  • Results: The report describes Oxygen-TryOn as the first system, to the authors’ knowledge, to deliver any-item, multi-reference try-on at this fidelity.The system’s data engine and CPT–SFT–RL recipe are detailed as the foundation of this capability.

9 Authors

The paper is authored by Yong Liu, Xiaolong Fu, Zihang Xu, Wen Xue, Xueheng Li, Lin Song, Yuan Zhang, Chuyang Zhao, Haoyang Huang, Nan Duan, Yipeng Sun, Yan Li, and Simiu Gu.

  • Authors: Authors: Yong Liu, Xiaolong Fu, Zihang Xu, Wen Xue, Xueheng Li, Lin Song, Yuan Zhang, Chuyang Zhao, Haoyang Huang, Nan Duan, Yipeng Sun, Yan Li, and Simiu Gu.Yong Liu and Xiaolong Fu are marked with ∗; Yipeng Sun is marked with †.
Loading 2607.21694v1…