Source-linked AI summary
Playground v2.5: Three Insights towards Enhancing Aesthetic Quality in Text-to-Image Generation
Daiqing Li, Aleks Kamko, Ehsan Akhgari, Ali Sabet, Linmiao Xu, Suhail Doshi
TL;DR
Playground v2.5 addresses aesthetic-quality challenges in text-to-image generation involving color and contrast, aspect-ratio diversity, and human-centric details. It refines training through noise-schedule analysis, balanced aspect-ratio bucketing, and human-preference alignment, then evaluates the result against competing models. The model surpasses the cited open- and closed-source systems in reported aesthetic-preference comparisons and overall FID, while future work identifies text-to-image alignment, variation, and architecture as remaining areas.
Problem
Text-to-image models face limitations in vibrant color and contrast, multiple aspect-ratio generation, and visual accuracy in human features.
Method
Playground v2.5 uses noise-schedule analysis, balanced bucket sampling across aspect ratios, and iterative human-in-the-loop training from user-rated data.
Results
Playground v2.5 surpasses SDXL, Playground v2, PIXART-α, DALL·E 3, and Midjourney 5.2 in reported aesthetic quality, with a 4.8x preference over SDXL.
Takeaways & Limitations
The study provides three empirically evaluated directions for improving aesthetic quality: color and contrast, multiple aspect ratios, and human-preference alignment for fine details.
Takeaways & Limitations
Future work remains to improve text-to-image alignment, variation capabilities, and explore new architectures.
Abstract
from arXiv · showhide
In this work, we share three insights for achieving state-of-the-art aesthetic quality in text-to-image generative models. We focus on three critical aspects for model improvement: enhancing color and contrast, improving generation across multiple aspect ratios, and improving human-centric fine details. First, we delve into the significance of the noise schedule in training a diffusion model, demonstrating its profound impact on realism and visual fidelity. Second, we address the challenge of accommodating various aspect ratios in image generation, emphasizing the importance of preparing a balanced bucketed dataset. Lastly, we investigate the crucial role of aligning model outputs with human preferences, ensuring that generated images resonate with human perceptual expectations. Through extensive analysis and experiments, Playground v2.5 demonstrates state-of-the-art performance in terms of aesthetic quality under various conditions and aspect ratios, outperforming both widely-used open-source models like SDXL and Playground v2, and closed-source commercial systems such as DALLE 3 and Midjourney v5.2. Our model is open-source, and we hope the development of Playground v2.5 provides valuable guidelines for researchers aiming to elevate the aesthetic quality of diffusion-based image generation models.
1 Introduction
Playground v2.5 retains Playground v2’s architecture while refining its training recipe to improve aesthetic quality, especially color, contrast, and style-prompt following. The model is evaluated against prior open-source systems and presented with open-source weights for research use.
- Playground v2.5 keeps Playground v2’s underlying architecture and instead focuses on analyzing and improving the training recipe.The stated goal is to push aesthetic quality to a new level without changing the model architecture.
- The work targets color and contrast, generation across multiple aspect ratios, and human-centric fine details.These issues are evaluated through user studies and benchmarks, including the proposed MJHQ-30K benchmark across 10 categories.
- The model’s weights are open-sourced on HuggingFace under a license intended to facilitate research-team use.Extensions for A1111 and ComfyUI are also planned.
- Playground v2.5 produces samples with more vibrant colors and contrast than SDXL.
- Compared with Playground v2, Playground v2.5 improves color, contrast, and the ability to follow style-related prompts.The comparison uses more complex prompts, with Playground v2 in the top row and v2.5 in the bottom row.
2 Methods
Playground v2.5 improves aesthetic quality through a revised noise schedule, balanced aspect-ratio bucketing, and human-preference alignment. These changes target muted colors, ratio-dependent composition failures, and malformed human details.
- Enhanced Color and Contrast: Latent diffusion models struggle with high color contrast and vibrant color ranges, partly because their noise scheduling leaves the terminal signal-to-noise ratio too high.SDXL may fail to produce pure black or white images and to place subjects on solid backgrounds.
- Enhanced Color and Contrast: Playground v2.5 trains from scratch with EDM, whose near-zero terminal signal-to-noise ratio removes the need for offset noise and addresses muted colors.The authors also describe EDM as supporting explicit training, sampling, and UNet-preconditioning choices.
- Generation Across Multiple Aspect Ratios: Balanced bucket sampling across aspect ratios helps Playground v2.5 avoid ratio bias and catastrophic forgetting when training on non-square images.The approach follows a bucketing strategy similar to SDXL while distributing data more evenly across ratios.
- Generation Across Multiple Aspect Ratios: Playground v2.5 generates high-quality images across portrait and landscape ratios without errors such as multiple objects or incorrect composition.The comparison with SDXL covers qualitative examples across both portrait and landscape settings.
- Human Preference Alignment: Human-preference alignment uses automatically curated, user-rated datasets and iterative human-in-the-loop selection to reduce visual errors in human features.The authors monitor progress using fixed-prompt image-grid comparisons.
- Human Preference Alignment: The alignment strategy improves facial detail, eye shape and gaze, hair texture, lighting, color, saturation, and depth-of-field relative to SDXL.The paper identifies these as four important human-centric categories where the model excels over SDXL.
3 Evaluations
The evaluations combine in-product pairwise user studies with the People-200 and MJHQ-30K benchmarks. Across aesthetic preference, aspect ratios, people-focused prompts, and FID, Playground v2.5 outperforms the reported baselines and competing models.
- User Study Interface: Thousands of product users evaluated blinded image pairs, with each pair shown to at least 7 users and wins requiring a 2-vote margin.The studies used fixed prompts, sampled images from two models, and asked users to choose according to an attribute such as aesthetic preference.
- Overall Aesthetic Preference against other SoTA models: Playground v2.5 was favored 4.8x over SDXL on the Internal-1K aesthetic-preference study, exceeding Playground v2’s prior 2.5x result.Internal-1K contains real user prompts, and the study involved thousands of users.
- Evaluation of Generation Across Multiple Aspect Ratios: Playground v2.5 outperformed SDXL by a large margin across commonly used aspect ratios ranging from 9:16 to 16:9.A separate user study was conducted for each aspect ratio with the same conditioning applied to both models.
- Evaluation on People-centric Prompts: In the People-200 study, Playground v2.5 outperformed SDXL and RealStock v2 by a large margin for people-related prompts at 3:2 resolution 1254x836.People-200 contains 200 high-quality prompts curated from real product user prompts.
- Automatic Evaluation Benchmark: MJHQ-30K computes FID on a high-quality dataset spanning 10 categories, and Playground v2.5 outperformed Playground v2 and SDXL in overall and every category FID.The strongest improvements were reported in people and fashion; all FID metrics were computed at 1024x1024, and the study indicates correlation with human preferences.
- Automatic Evaluation Benchmark: The MJHQ-30K benchmark was released publicly on HuggingFace for community benchmarking of aesthetic quality during pre-training and alignment.The benchmark uses images made on Midjourney 5.2 and includes 3K samples per category.
4 Conclusion
Playground v2.5 reports state-of-the-art aesthetic quality through improvements in color and contrast, aspect-ratio coverage, and human-preference alignment. The model is publicly released for community use and further development.
- Playground v2.5 demonstrates improved color and contrast, high-quality generation across aspect ratios, and stronger alignment with human aesthetic preferences.
- The model is available for use through the product website, with its weights open-sourced on HuggingFace.
- Future work targets better text-to-image alignment, stronger variation capabilities, and exploration of new architectures.
- Playground v2.5 is presented as a stepping stone toward a unified general-purpose vision system for generating and editing pixels.