PaperlayerView exact revision on arXiv

Source-linked AI summary

A Style is Worth One Code: Unlocking Code-to-Style Image Generation with Discrete Style Space

Huijie Liu, Shuhao Cui, Haoxiang Cao, Shuai Ma, Kai Wu, Guoliang Kang

arXiv:2511.10555v5cs.CVcs.AI

TL;DR

Existing stylistic image-generation methods struggle to combine consistency, creativity, and reproducibility, and the code-to-style task has lacked open-source academic research. CoTyle learns style embeddings and an autoregressive style generator to turn numerical codes into styles; experiments report diverse, consistent, reproducible generation, while its diversity remains somewhat below Midjourney’s.

  • Problem

    Existing style-generation methods struggle to achieve high consistency, creativity, and reproducibility together, while code-to-style generation had primarily been explored in industry.

    Source: §1 · p. 2

  • Method

    CoTyle learns a discrete style codebook and an autoregressive generator that maps numerical codes to style embeddings used to condition a text-to-image diffusion model.

    Source: §1 · p. 2, §3.3 · p. 5

  • Results

    Experiments report that CoTyle generates diverse, consistent, reproducible styles from codes and produces target-style images without reference images, lengthy prompts, or fine-tuning.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    A numerical code serves as a compact style controller, and CoTyle also supports image-conditioned generation and style interpolation.

    Source: §1 · p. 3, §5 · p. 8

    CoTyle’s stylistic diversity remains somewhat lower than Midjourney’s, which the authors link likely to a limited dataset of curated style-paired images.

    Source: §9 · p. 12

Abstract

from arXiv · show

Innovative visual stylization is a cornerstone of artistic creation, yet generating novel and consistent visual styles remains a significant challenge. Existing generative approaches typically rely on lengthy textual prompts, reference images, or parameter-efficient fine-tuning to guide style-aware image generation, but often struggle with style consistency, limited creativity, and complex style representations. In this paper, we affirm that a style is worth one numerical code by introducing the novel task, code-to-style image generation, which produces images with novel, consistent visual styles conditioned solely on a numerical style code. To date, this field has only been primarily explored by the industry (e.g., Midjourney), with no open-source research from the academic community. To fill this gap, we propose CoTyle, the first open-source method for this task. Specifically, we first train a discrete style codebook from a collection of images to extract style embeddings. These embeddings serve as conditions for a text-to-image diffusion model (T2I-DM) to generate stylistic images. Subsequently, we train an autoregressive style generator on the discrete style embeddings to model their distribution, allowing the synthesis of novel style embeddings. During inference, a numerical style code is mapped to a unique style embedding by the style generator, and this embedding guides the T2I-DM to generate images in the corresponding style. Unlike existing methods, our method offers unparalleled simplicity and diversity, unlocking a vast space of reproducible styles from minimal input. Extensive experiments validate that CoTyle effectively turns a numerical code into a style controller, demonstrating a style is worth one code.

1. Introduction

Existing style-generation methods struggle to balance consistency, creativity, and reproducibility. CoTyle introduces numerical-code-driven style generation, using learned style embeddings and an autoregressive generator to create styles that can be reproduced from their codes.

1. Introduction

  • Existing methods cannot simultaneously achieve high consistency, creativity, and reproducibility because prompts vary visually, while image- and LoRA-based methods rely on existing styles and complex representations.Image and LoRA approaches also require sharing pixel-level references or complex weights for reproducible generation.Source: §1 · p. 2
  • Code-to-style generation uses a numerical style code to create novel styles, and CoTyle is presented as the first open-source framework for this task.The task had primarily been explored in industry, including Midjourney.Source: §1 · p. 2
  • CoTyle learns a discrete style codebook from paired style images, conditions a text-to-image diffusion model on its embeddings, and trains an autoregressive transformer to generate style-index sequences.At inference, the numerical code fixes the initial token, and the generated indices condition image generation.Source: §1 · p. 2
  • Experiments report that CoTyle generates target-style images without reference images, lengthy prompts, or fine-tuning, while supporting consistent, novel, and reproducible styles.The evaluation uses CSD [32] to assess style consistency and diversity.Source: §1 · p. 2
  • CoTyle’s numerical style code represents a style and can be modified to create unique styles without complex prompts, images, or LoRAs.The figure defines creativity as generating novel styles, consistency as producing multiple images in the same style, and reproducibility as using simple style definitions.Source: Figure 2 · p. 3
  • The paper also identifies image-conditioned generation and style interpolation as extensions of CoTyle.Its contributions describe a framework based on a discrete style codebook and an autoregressive style generator.Source: §1 · p. 3

2. Related Work

Related work covers conditioning methods for general image generation and approaches focused specifically on stylistic image generation. The latter rely on reference images or style LoRAs, while the cited discussion notes limitations in style novelty and representation.

2. Related Work

  • Conditioned image generation ranges from text-based methods to structural guidance, identity-specific generation, inversion, adapters, fine-tuning, and token concatenation.Qwen-Image [40] encodes visual information as prompt embeddings but is primarily designed for image editing.Source: §2 · p. 3
  • Stylistic image-generation methods use style images as visual conditions [38] [39] [41] [42] or train style LoRA modules to steer generation.The discussion says these signals require complex pixel-level maps or additional model parameters and lack the capability to create novel styles.Source: §2 · p. 3
  • Midjourney introduced style-code-based image generation, but the absence of technical reports has hindered its adoption and development in the open-source community.The cited methods using style images or LoRAs are described as relying on existing styles.Source: §2 · p. 3

3. Method

CoTyle combines a discrete style codebook, a style-conditioned text-to-image diffusion model, and an autoregressive generator of style indices. The codebook extracts style embeddings, which condition image generation, while the generator enables numerical codes to produce novel styles.

3.1. Style Codebook

  • The style codebook maps paired images with the same style to similar embeddings and different styles to distinct embeddings, rather than reconstructing images.Its objective combines contrastive, reconstruction, and vector-quantization losses; reconstruction helps avoid codebook collapse.Source: §3.1 · p. 4

3.2. T2I-DM Conditioned on Style Codebook

  • Quantized style embeddings condition a DiT through its textual branch, replacing image features while a text prompt guides generation of a target image.Training pairs a style image with another image of the same style and conditions generation on the first image's embedding and the second image's prompt.Source: §3.2 · p. 4

3.3. Code-to-Style Image Generation

  • The autoregressive generator learns style-index sequences from codebook indices, then a numerical code seeds index sampling to create a style embedding for image generation.Inference generates a fixed-length sequence, retrieves its codebook vectors, and decodes them into a style embedding.Source: Figure 3 · p. 4, §3.3 · p. 5
  • During inference, CoTyle suppresses logits for frequent codebook indices because sampling only those indices yields images without a specific style.The suppression coefficient depends on index frequency, a threshold, and a hyperparameter.Source: §3.3 · p. 5

4. Experiment

The experiments evaluate CoTyle’s code- and image-conditioned generation, compare style-injection choices, test training components, and examine style interpolation. CoTyle shows stronger style consistency than Midjourney, while its diversity is slightly lower.

4.1. Implementation Details

  • The codebook and diffusion model are trained separately, while the style generator is trained from scratch; each style reference is encoded as 196 discrete style indices.The codebook uses a vocabulary of 1,024 and 64-dimensional embeddings; the diffusion model starts from pretrained Qwen-Image, and the style generator uses Qwen2-0.5B architecture.Source: §4.1 · p. 5

4.2. Benchmark

  • The evaluation uses CSD for style consistency and diversity, with CLIP-T for text-image alignment and QualityCLIP for aesthetics.The code-to-style evaluation samples 500 codes and generates four images per code; the image-conditioned set contains 500 prompt-reference pairs.Source: §4.2 · p. 5

4.4. Results

  • CoTyle achieves significantly superior style consistency to Midjourney, though its diversity is slightly inferior and may be limited by training-data breadth.The qualitative comparison uses image sets generated from the same style code.Source: Figure 4 · p. 6, §4.4 · p. 6

4.5. Ablation

  • Textual-branch style injection better preserves semantic information than visual-branch injection, while the visual branch can miss semantic-style elements.The visual branch captures some color cues but misses details such as circular contours; textual injection can generate a human figure composed in the specified style.Source: Figure 5 · p. 7, §4.5 · p. 7
  • Contrastive loss improves style extraction and consistency, reconstruction loss aligns style embeddings with original ViT features, and suppressing high-frequency indices helps avoid low-diversity, photorealistic outputs.Sampling only high-frequency indices produces nearly style-less images, resembling outputs from the vanilla text-to-image diffusion model.Source: §4.5 · p. 7, Figure 7 · p. 8

4.6. Style Interpolation

  • Combining subsets of style indices from different styles, with adjustable sampling ratios, produces different degrees of stylistic interpolation.The approach also supports blending predefined style codes with styles extracted from user-provided pixel images.Source: Figure 8 · p. 8, §4.6 · p. 8

5. Conclusion

The conclusion presents code-to-style generation as a new task and CoTyle as its first open-source framework. It reports that CoTyle maps discrete codes to diverse visual styles and supports image-conditioned generation and style interpolation.

5. Conclusion

  • The paper introduces code-to-style image generation, in which numerical style codes condition stylistic image generation.CoTyle is presented as the first open-source framework for the task.Source: §5 · p. 8
  • CoTyle uses a style generator to produce novel style embeddings that condition a diffusion model to generate images in a specific style.Source: §5 · p. 8
  • The paper reports that CoTyle maps discrete codes to diverse visual styles and also supports image-conditioned generation and style interpolation.The conclusion says this work opens future research in discrete stylistic representations.Source: §5 · p. 8

6. Experimental Details

The experimental details describe the comparison data and evaluation settings for code-conditioned and image-conditioned generation. Baselines for image-conditioned generation use their official pretrained weights with specified generation settings.

6. Experimental Details

  • For code-to-style evaluation, the authors compare CoTyle with Midjourney and manually collect test data from its website because its code is not open-source.Source: §6 · p. 11
  • The image-conditioned comparison includes StyleStudio, CSGO, USO, Flux-Kontext, and InstantStyle, using official pretrained weights and method-specific generation settings.The settings include different classifier-free guidance values and denoising-step counts; Flux-Kontext uses the style image as a condition and prefixes the prompt with “Using this style, ...”.Source: §6 · p. 11
  • For image-conditioned methods, the authors set classifier-free guidance, denoising steps, conditioning scale, and style scale according to each method’s implementation or recommendation.Source: §6 · p. 11

7. Analysis of High-frequency Index

The analysis finds a long-tail distribution in codebook-index usage, with a small fraction of indices selected much more often than most others. Controlled ablations suggest these frequent indices act as placeholders rather than encoding specific stylistic features.

7. Analysis of High-frequency Index

  • Approximately 5–10% of codebook indices are selected far more frequently than the vast majority, indicating uneven token use.The analysis aggregates index occurrences after encoding a diverse dataset of stylistically varied images.Source: §7 · p. 11
  • Replacing dominant indices with random or shuffled values causes minimal degradation in reconstruction quality or perceptual style fidelity, and outputs from only the top-k indices remain stylistically coherent.Source: §7 · p. 11
  • The authors interpret high-frequency indices as generic placeholder tokens rather than semantically meaningful style features, citing possible overfitting or insufficient codebook capacity.They identify balanced, semantically disentangled discrete representations as a direction for future work.Source: §7 · p. 11

8. Implications and Insights

The discrete style representation supports style interpolation, while the authors advocate semantic text conditioning and present code-to-style generation as an open direction for academic research.

8. Implications and Insights

  • Blending discrete style codes enables smooth style interpolation, and token order is described as irrelevant to the learned style representation.The authors report minimal impact from random sampling versus evenly splitting tokens between styles.Source: §8 · p. 12, Figure A2 · p. 12
  • Text conditioning is advocated because the authors argue that style perception is inherently semantic rather than purely chromatic.Source: §8 · p. 12
  • The authors present code-to-style generation as academically unexplored and release model weights and source code to support further research.Source: §8 · p. 12

9. Limitations

CoTyle's limitations concern style diversity, loss of fine stylistic nuance through quantization, and the difficulty of navigating its broad code space; the authors describe the framework as an initial step for further work.

9. Limitations

  • Generated-style diversity remains somewhat lower than Midjourney's, possibly because the curated style-paired dataset does not capture the full variability of human artistic expression.The authors suggest broader artistic training data or mixture-of-experts architectures as possible avenues for improvement.Source: §9 · p. 12
  • Finite-codebook quantization can attenuate or omit intricate or hybrid nuances from reference images, potentially reducing output expressiveness.Source: §9 · p. 12
  • Finding aesthetically or artistically engaging styles currently requires stochastic search across a vast code space.The authors suggest more intuitive ways to navigate or modify the latent space could improve user agency and practical applicability.Source: §9 · p. 12
  • Despite areas for improvement, CoTyle provides an overall framework for the first open-source work on code-to-style generation.Source: §9 · p. 12

10. More Results

The additional results describe interpolation between styles encoded from two images and controlled code-to-style comparisons across prompts and style conditions.

10.1. Style Interpolation

  • Given two reference images, style interpolation randomly blends their codebook index sets at a specified mixing ratio to generate intermediate styles.Figure A3 places the source references at the ends and varies the blending ratio across intermediate outputs.Source: §10.1 · p. 12, Figure A3 · p. 14

10.2. Code-to-Style Generation

  • The additional code-to-style figures vary text prompts across columns and style codes across columns for direct comparison of style consistency and fidelity.Images in each row share a random seed; prompts are listed in Table A1, and the results are described as robust and semantically aligned.Source: §10.2 · p. 13, Table A1 · p. 13, Figure A4 · p. 15

Related papers

Loading 2511.10555v5…