PaperlayerView exact revision on arXiv

Source-linked AI summary

OmniStyle: Filtering High Quality Style Transfer Data at Scale

Ye Wang, Ruiqi Liu, Jiang Lin, Fei Liu, Zili Yi, Yilin Wang, Rui Ma

arXiv:2505.14028v1cs.CV

TL;DR

Style-transfer methods face limitations in arbitrary-style generalization, output control, and access to large paired datasets. The paper introduces a paired dataset, a quality-filtering framework, and an end-to-end DiT-based model for instruction- and image-guided transfer. The authors report superior performance to existing methods.

  • Problem

    Existing methods have limited ability to handle truly arbitrary styles and control stylization, while large-scale, high-quality paired datasets are lacking.

    Source: §1 · p. 2

  • Method

    The paper constructs OmniStyle-1M, filters triplets with OmniFilter across content, style, and aesthetic criteria, and trains an end-to-end DiT framework for instruction- and image-guided transfer.

    Source: §1 · p. 2

  • Results

    The authors report that OmniStyle achieves superior performance compared with existing style-transfer methods.

    Source: §1 · p. 2, §6 · p. 8

  • Takeaways & Limitations

    OmniStyle-1M, OmniFilter, and OmniStyle provide resources and methods for advancing high-quality style transfer.

    Source: §6 · p. 8

    The authors identify the lack of artist-annotated perfect paired data with ground-truth stylized images as a boundary for objective evaluation.

    Source: §6 · p. 9

Abstract

from arXiv · show

In this paper, we introduce OmniStyle-1M, a large-scale paired style transfer dataset comprising over one million content-style-stylized image triplets across 1,000 diverse style categories, each enhanced with textual descriptions and instruction prompts. We show that OmniStyle-1M can not only enable efficient and scalable of style transfer models through supervised training but also facilitate precise control over target stylization. Especially, to ensure the quality of the dataset, we introduce OmniFilter, a comprehensive style transfer quality assessment framework, which filters high-quality triplets based on content preservation, style consistency, and aesthetic appeal. Building upon this foundation, we propose OmniStyle, a framework based on the Diffusion Transformer (DiT) architecture designed for high-quality and efficient style transfer. This framework supports both instruction-guided and image-guided style transfer, generating high resolution outputs with exceptional detail. Extensive qualitative and quantitative evaluations demonstrate OmniStyle's superior performance compared to existing approaches, highlighting its efficiency and versatility. OmniStyle-1M and its accompanying methodologies provide a significant contribution to advancing high-quality style transfer, offering a valuable resource for the research community.

1. Introduction

The paper targets limited arbitrary-style generalization, unpredictable stylization, and the lack of large paired datasets that has made many methods computationally intensive. It introduces OmniStyle-1M, OmniFilter, and an end-to-end DiT framework for controlled, efficient style transfer.

1. Introduction

  • Existing methods struggle with truly arbitrary styles and controllable outputs, while scarce paired data has driven computationally intensive, non-end-to-end approaches.Some methods require hundreds or thousands of iterations or transfer style features within intermediate diffusion U-net blocks.Source: §1 · p. 2
  • OmniStyle-1M contains one million content-style-stylized triplets across 1,000 style categories, with textual descriptions and instructional prompts.Content images are generated in 20 categories, and six style-transfer models produce the stylized results.Source: §1 · p. 2
  • OmniFilter assesses content preservation, style consistency, and aesthetic appeal using CLIP and a multimodal large language model.The framework is intended to help control style-transfer outputs.Source: §1 · p. 2
  • OmniStyle is an end-to-end DiT-based framework for instruction- and image-guided transfer, and the authors report superior performance to existing methods.The framework is described as efficient and able to produce detailed, high-resolution results.Source: §1 · p. 2

2. Related Work

Prior style-transfer approaches can be computationally expensive, time-consuming, or limited to one guidance mode. The paper positions OmniStyle-1M as paired training data for an efficient end-to-end framework spanning diverse styles and tasks.

Style Transfer

  • Earlier optimization and diffusion-based methods can require extensive iterative optimization or DDIM inversion, and many support only image- or text-guided transfer.The paper presents these constraints as obstacles to large-scale or real-time use and multitask flexibility.Source: §2 · p. 3

Style-Related Datasets

  • OmniStyle-1M and IMAGStyle are identified as the only compared datasets with triplets; OmniStyle-1M spans 1,000 styles versus IMAGStyle’s 14.OmniStyle-1M uses six generation models and contains over one million triplets, five times IMAGStyle’s size; IMAGStyle uses B-LoRA [8].Source: §2 · p. 3
  • Style-image-only Style30K [19] covers 1,120 fine-grained categories but lacks paired data for style-transfer tasks.The comparison also describes UltraEdit [49] as having few content-stylized pairings and limited textual instructions.Source: §2 · p. 3

3. OmniStyle-1M

OmniStyle-1M combines diverse style-transfer triplets with prompts, and OmniFilter selects a smaller subset using complementary content, style, and aesthetic assessments. The section describes the dataset’s coverage, construction, and filtering process.

3.1. Dataset Overview

  • OmniStyle-1M covers 1,000 fine-grained styles and 20 content categories, with a balanced distribution of stylized images across content categories.Triplets include content, style, and stylized images alongside two prompt texts and instructions.Source: §3.1 · p. 3, Figure 2 · p. 4

3.2. Dataset Curation

  • The curation pipeline generates 2,000 content images from 2,000 prompts and creates stylizations using six models and 1,000 Style30K [19] images.Each content image is paired with 100 randomly selected styles and stylized by each of the six models, yielding one million triplets.Source: §3.2 · pp. 3–4

3.3. OmniFilter

  • OmniFilter evaluates content preservation through semantic consistency and structural integrity, using image-caption similarity and image embeddings as measures.The semantic score uses CLIP [28]; structural preservation uses DINOv2 [25] embeddings, and the two scores are combined with α set to 0.5.Source: §3.3 · pp. 4–5
  • Style consistency is scored by a CLIP [28] image encoder fine-tuned contrastively on Style30K [19] to group same-style images and separate different-style images.The paper notes that style is subjective and that existing style-loss metrics may not capture the range of styles robustly.Source: §3.3 · p. 5
  • Aesthetic scoring uses 40 visual attributes and InternVL-G [5] image and text features, with training on AVA followed by fine-tuning on BAID [41].The fine-tuning step is described as bridging the domain gap between natural and stylized images.Source: §3.3 · p. 5
  • OmniFilter selects OmniStyle-150K, a 150K-triplet subset spanning 1,000 categories, by choosing the highest-scoring of six outputs for each content-style pair.The total score weights content preservation, style consistency, and aesthetic appeal at 0.2, 0.6, and 0.2, respectively.Source: §3.3 · p. 6

4. OmniStyle

OmniStyle supports style transfer guided by either text instructions or reference images through a FLUX-dev-based architecture. Training fine-tunes the diffusion transformer while keeping other components frozen.

4.1. Architecture

  • For reference-guided transfer, OmniStyle spatially concatenates style and content VAE features with noisy latents and text tokens before MM-DiT.Instruction-guided transfer instead treats content features as text tokens and marks them with [img] and [/img] tags.Source: §4.1 · p. 6

4.2. Implementation Details

  • Training fine-tunes only the diffusion transformer on the style-transfer dataset, while all other components remain frozen.The model is trained on 8 NVIDIA H100 GPUs with batch size 1 per GPU and a learning rate of 1e-4.Source: §4.2 · p. 6
  • Table 2 compares state-of-the-art methods on instruction-guided style transfer.The supplied table passage gives its scope but no cell values.Source: Table 2 · p. 6

5. Experiments

OmniStyle is evaluated on instruction-guided and image-guided style transfer using quantitative metrics, qualitative comparisons, and a user study. The reported results favor OmniStyle across both tasks and in user preferences.

5.1. Quantitative Experiments

  • 0.5128 content preservation, 0.6441 style consistency, and 0.2873 style loss: OmniStyle exceeds DiffStyler on all three instruction-guided metrics.Its aesthetic appeal is 5.7512 versus DiffStyler’s 5.4551, while OmniStyle is competitive with HQ-Edit (5.7777).Source: §5.1 · pp. 6–7
  • 0.7483 style consistency, 5.7913 aesthetic appeal, and 0.1086 style loss: OmniStyle leads the reported image-guided comparisons.The reported style consistency and aesthetic scores exceed StyleShot (0.7347, 5.7818) and CSGO (0.7251, 5.7712).Source: §5.1 · p. 7, Table 3 · p. 7

5.2. Qualitative Experiments

  • Qualitative comparisons report that OmniStyle handles a broader range of styles with accurate, consistent results across instruction-guided and image-guided tasks.For image-guided transfer, the reported comparisons highlight style preservation, aesthetic appeal, and content preservation; instruction-guided examples contrast OmniStyle with methods that struggle with fine-grained styles or alter content.Source: §5.2 · pp. 7–8, Figure 6 · p. 8

5.3. User Study

  • In a study with 30 participants and 1,800 votes, users preferred OmniStyle’s stylized images in both instruction-guided and image-guided tasks.Participants ranked their top three results for style preservation, content preservation, and aesthetic appeal; the reported tables summarize first-place rankings.Source: §5.3 · p. 8, Table 4 · p. 8, Table 5 · p. 8

6. Conclusion

The conclusion presents OmniStyle-1M, OmniFilter, and OmniStyle as a dataset, quality-filtering framework, and transfer model for advancing style transfer. It identifies artist-annotated paired data as a possible direction for more objective evaluation.

6. Conclusion

  • OmniStyle-1M contains one million paired content-style-stylized triplets across 1,000 categories, enriched with prompts and descriptions for controlled, consistent, and diverse transfer.The conclusion describes OmniFilter as assessing content preservation, style consistency, and aesthetic appeal to filter stylized images.Source: §6 · p. 8
  • OmniStyle is an end-to-end DiT framework for instruction-guided and image-guided transfer, with high-resolution detailed stylizations and reported gains in quality and efficiency over existing methods.The conclusion describes the dataset, filtering framework, and model together as a foundation for future style-transfer research and applications.Source: §6 · p. 8
  • Artist-annotated perfect pairs are proposed as future work to provide ground-truth stylizations and support more objective evaluation.The paper notes that such targets could enable comparison-based measures of perceptual similarity or content preservation.Source: §6 · p. 9

1. More Details about OmniStyle

The supplementary details describe how OmniStyle-1M’s content and style images are assembled and show examples of the resulting triplets.

1.1. Content Images Generation

  • Content images are generated from 100 ChatGPT descriptions in each of 20 categories using FLUX, yielding 2,000 images at 1K resolution.The paper says this generative approach avoids copyright and privacy issues while ensuring high-quality data.Source: §1.1 · p. 11

1.2. Style Images Collections

  • The style-image collection spans 1,000 distinct, fine-grained categories from Style30K, broader than the traditional categories commonly emphasized by existing datasets.The examples shown in Figure 8 illustrate a subset of the collected style images.Source: §1.2 · p. 11

1.3. Examples of Triplet

  • Example content-style-stylized triplets use 1024×1024 content and stylized images, with reported high consistency between stylized results and their reference styles.Figure 9 arranges each triplet as content image, style image, then stylized result.Source: §1.3 · p. 11

2. More Details about OmniFilter

OmniFilter assesses content preservation, style consistency, and aesthetic appeal using distinct evaluation procedures. The described procedures combine pretrained visual encoders, a contrastively fine-tuned image encoder, and an aesthetic-score predictor.

Content Preservation

  • Content preservation combines CLIP similarity between the stylized image and content text with DINOv2 structural similarity between stylized and original content images.The paper states that this evaluation uses pretrained weights and requires no training or fine-tuning.Source: §2 · p. 11

Style Consistency

  • Style consistency is measured with a CLIP image encoder fine-tuned through contrastive learning on positive and negative Style30K pairs.In style retrieval on 500 held-out queries and 500 corresponding keys, the fine-tuned model outperforms original CLIP.Source: §2 · p. 12, Table 7 · p. 12
  • Figure 12 presents style-image pairs by row with similarity scores; higher scores indicate greater style consistency between the paired images.The figure’s scores illustrate the model’s style-similarity judgments.Source: §2 · pp. 12–13, Figure 12 · p. 13

Aesthetic Appeal

  • Aesthetic scoring uses InternVL2-generated visual-attribute descriptions and InternVL-G features, followed by a trained MLP score predictor.The described attribute prompts cover 40 visual-attribute categories, and the training data include AVA and BAID aesthetic-image datasets.Source: §2 · p. 13

3. More Experiments

Section 3 reports user-study and OmniFilter ablation results, then presents additional LoRA-based and qualitative style-transfer experiments. OmniStyle receives the highest user ratings in both tasks, while the filtering components improve Style Loss.

3.1. Quantitative Experiments

  • OmniStyle achieves the highest user ratings in both instruction-guided and image-guided style transfer tasks.Participants selected and ranked their top three results; Tables 9 and 10 report Rank 1 and Top 3 scores.Source: Table 9 · p. 14, Table 10 · p. 14, §3.1 · p. 14
  • Adding Style Consistency to Content Preservation reduces Style Loss from 0.1577 to 0.1434, and adding Aesthetic Appeal lowers it further to 0.1405.Lower Style Loss indicates better data quality.Source: §3.1 · p. 14, Table 11 · p. 14
  • StyleShot achieves the highest overall OmniFilter score; CAST leads on content preservation, while StyleShot leads on style similarity.CAST scores 0.6411 on content preservation; StyleShot scores 0.7431 on style similarity.Source: §3.1 · p. 14, Figure 13 · p. 14

3.2. Qualitative Experiments

  • A LoRA model trained on OmniFilter-selected OmniStyle-1M triplets produces high-quality style transfer results.The training adapts official FLUX-LoRA by encoding the content image with a VAE and concatenating it with noise latents before model input.Source: §3.2 · p. 15, Figure 14 · p. 15
  • The qualitative experiments provide additional style-transfer results across instruction-guided and image-guided tasks.The examples span diverse style categories; the section characterizes the outcomes as demonstrating superior performance.Source: §3.2 · p. 15, Figure 15 · p. 16, Figure 16 · p. 17

Related papers

Loading 2505.14028v1…