PaperlayerView exact revision on arXiv

Source-linked AI summary

Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis

Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, Hongsheng Li

arXiv:2306.09341v2cs.CVcs.AIcs.DB

TL;DR

Existing metrics poorly capture human preferences for text-to-image outputs. This paper introduces a large preference dataset, trains HPS v2, and develops evaluation prompts and a benchmark; HPS v2 generalizes better across image distributions and detects algorithmic improvements.

  • Problem

    Existing text-to-image metrics do not correlate well with human preferences, making preference evaluation poorly tracked.

  • Method

    The paper builds HPD v2, fine-tunes CLIP into HPS v2, and designs evaluation prompts and a benchmark for text-to-image models.

  • Results

    HPS v2 generalizes better across image distributions and is sensitive to algorithmic improvements in text-to-image generation.

  • Takeaways & Limitations

    HPD v2 and HPS v2 support more aligned evaluation of text-to-image generation with human judgments.

  • Takeaways & Limitations

    The dataset may overlook topics, reflect annotator or prompt-cleaning bias, and does not study image-resolution effects on human preferences.

Abstract

from arXiv · show

Recent text-to-image generative models can generate high-fidelity images from text inputs, but the quality of these generated images cannot be accurately evaluated by existing evaluation metrics. To address this issue, we introduce Human Preference Dataset v2 (HPD v2), a large-scale dataset that captures human preferences on images from a wide range of sources. HPD v2 comprises 798,090 human preference choices on 433,760 pairs of images, making it the largest dataset of its kind. The text prompts and images are deliberately collected to eliminate potential bias, which is a common issue in previous datasets. By fine-tuning CLIP on HPD v2, we obtain Human Preference Score v2 (HPS v2), a scoring model that can more accurately predict human preferences on generated images. Our experiments demonstrate that HPS v2 generalizes better than previous metrics across various image distributions and is responsive to algorithmic improvements of text-to-image generative models, making it a preferable evaluation metric for these models. We also investigate the design of the evaluation prompts for text-to-image generative models, to make the evaluation stable, fair and easy-to-use. Finally, we establish a benchmark for text-to-image generative models using HPS v2, which includes a set of recent text-to-image models from the academic, community and industry. The code and dataset is available at https://github.com/tgxs002/HPSv2 .

1 Introduction

Existing metrics correlate poorly with human preferences for text-to-image outputs, motivating HPD v2 and HPS v2. The work also establishes prompt-based evaluation and shows HPS v2 can assess model improvements.

  • Motivation: Popular metrics including Inception Score, Fréchet Inception Distance, and CLIP Score do not correlate well with human preferences for generated images.Different random seeds can produce images with varying quality, making human preference an important evaluation signal.
  • HPD v2: HPD v2 contains 798k human-annotated pairwise comparisons from images generated by 9 text-to-image models and COCO Captions.It compares images generated from the same prompt and addresses image-source bias in previous datasets.
  • HPS v2: HPS v2 is trained on HPD v2 to score generated images by their likelihood of being preferred by humans.Experiments report better generalization than HPS v1, ImageReward, and PickScore.
  • Benchmarking: The authors establish a benchmark using HPS v2 and study prompt selection for stable, fair, and easy-to-use text-to-image evaluation.They find DrawBench is not large enough for stable evaluation and investigate which prompts and how many prompts to use.
  • Benchmarking: HPS v2 is sensitive to algorithmic improvements, including a diffusion-model test-time noise-blending trick and quantitative improvements from a prior method.These experiments support using HPS v2 to evaluate text-to-image generative models.

2 Related Work

Text-to-image generation has progressed through autoregressive, GAN, and diffusion architectures, with diffusion models producing high-quality samples efficiently. Existing evaluation and human-rating resources provide important foundations, but standard metrics do not correlate well with human preferences for recent text-to-image models.

  • Text-to-Image Generation: Text-to-image generation has explored autoregressive models, GANs, and diffusion models, with diffusion models demonstrating superior computational efficiency and high-quality samples.DALL·E was the first to achieve high-quality open-domain text-to-image generation.
  • Evaluation Metrics: IS and FID are widely used for evaluating image generative models but correlate poorly with human preferences for recent text-to-image outputs.LPIPS instead measures perceptual similarity between image pairs using a model fitted on human-perception data, while HYPE applies similar methodology to generated human faces.
  • Human Preference Data: AVA and Simulacra collect absolute human ratings for photographs and text-generated images, respectively, and have supported training the Aesthetic Score Predictor.The Aesthetic Score Predictor participates in Stable Diffusion’s data-cleaning pipeline.
  • Human Preference Data: InstructGPT aligns a pretrained language model with human feedback, extending the use of human preferences beyond absolute image-rating datasets.The passage introduces this alignment approach as related work on human feedback.

3 Human Preference Dataset v2

HPD v2 is a large-scale human-preference dataset built from cleanly annotated comparisons of images generated from shared text prompts. Its collection pipeline combines diversified prompt and image sources with structured annotation and dedicated train/test splits.

  • Dataset overview: HPD v2 contains 798k binary preference choices covering 434k images, with each pair generated by different models from the same prompt.The dataset is organized around one annotator’s binary choice between two images.
  • Prompt collection: Prompts are grouped for training preference prediction, testing preference prediction, and benchmarking generative models.The groups combine COCO Captions prompts with DiffusionDB prompts cleaned by ChatGPT.
  • Prompt collection: ChatGPT rewrites DiffusionDB prompts into clearly written single sentences with fewer style words, improving annotator readability.The same cleaning pipeline is used for image-generation and generative-model-benchmarking prompts.
  • Image collection: Images are generated by different models from the same prompts, with real COCO images added for COCO-caption prompts.The sources span models with different architectures and scales, creating substantial image diversity for evaluation.
  • Preference annotation: Annotators rank images using image-text alignment and visual attractiveness, which can conflict and require evaluating their trade-off.HPD v2 uses average annotators rather than art-appreciation specialists.
  • Dataset splits: The test split has 400 image groups: 300 use DiffusionDB prompts and 100 use COCO Captions prompts, with 9 generated images per group.COCO-based test groups additionally include one real COCO image and are annotated by 10 distinct annotators.

4 Human Preference Score v2

HPS v2 fine-tunes CLIP on HPD v2 human preference pairs using a preference-learning objective. It achieves better accuracy on both ImageReward and HPD v2 test sets, while reflecting average human preferences learned from many annotators.

  • Training: HPS v2 fine-tunes CLIP on HPD v2 image pairs and prompts labeled according to which image humans prefer.The model scores image–prompt similarity and predicts pairwise preferences.
  • Training: The model is optimized for 4,000 steps with AdamW, a 3.3 × 10−6 learning rate, 0.35 weight decay, batch size 128, and 500 warm-up steps.Training uses a cosine learning-rate schedule and updates the last 20 image-encoder layers and last text-encoder layers.
  • Evaluation: HPS v2 exhibits better preference-prediction accuracy on both ImageReward and HPD v2 benchmarks, demonstrating stronger generalization.HPD v2 covers a wider range of image distributions than the ImageReward test set, which contains only Stable Diffusion images.
  • Human agreement: HPS v2 learns average human preferences from many annotators and can exceed the accuracy of a single person compared with another single person.A single person compared with an averaged ranking still performs better than HPS v2.

5 Benchmarking Text-to-Image Generative Models

The benchmark evaluates text-to-image models with HPS v2 across four styles using 3,200 prompts, then compares recent academic, community, and industry models with existing metrics. Community models consistently outperform academic models, while CogView2 illustrates a difference between aesthetic quality and controllability.

  • Evaluation setup: HPS v2 evaluates models separately across Animation, Concept-art, Painting, and Photo styles using 3,200 prompts.Each style contributes 800 ChatGPT-generated evaluation prompts.
  • Evaluation setup: 800 prompts per style provide statistical stability across evaluated models while avoiding excessive computational overhead.The benchmark uses 800 prompts for each of the four styles.
  • Benchmark results: The benchmark covers recent text-to-image generative models from academic, community, and industry sources.HPS v2 is plotted against CLIP Score and Aesthetic Score for comparison.
  • Benchmark results: Community models consistently outperform academic models in the benchmark.This trend is reported for popular community models across the evaluated set.
  • Benchmark results: CogView2 produces high-aesthetic-quality images but has worse controllability.Its results are highlighted as an interesting data point when comparing HPS v2 with existing evaluation metrics.

6 Limitations

The work has limitations in prompt coverage, prompt-cleaning and annotation bias, and potential negative social impacts from generative models. The benchmark may overlook specialized design topics and may not fully represent preferences beyond its data sources and annotator pool.

  • 6 Limitations: The prompts come from DiffusionDB and COCO Captions, while benchmark styles beyond “Photo” reflect general interest observed in DiffusionDB.DiffusionDB may overlook topics such as logo and graphic design, whose industries have distinct criteria and preferences.
  • 6 Limitations: ChatGPT-based prompt cleaning may introduce bias, and annotations from 57 annotators may also suffer from bias.The authors report that they did not currently find patterns from prompt-cleaning bias.
  • 6 Limitations: More plausible generated content may be maliciously used to spread convincing misinformation and fake content.Generative models may also amplify biases and stereotypes present in their training data.

7 Conclusion

The work introduces a less biased pipeline for collecting prompts and images for human preference annotations, yielding Human Preference Dataset v2 with 798k comparison pairs. These data enable Human Preference Score v2, a state-of-the-art preference prediction model intended to better align text-to-image evaluation with human values and judgments.

  • The authors design a less biased pipeline for collecting prompts and images for human preference annotations.
  • 798k carefully annotated comparison pairs comprise Human Preference Dataset v2.
  • Human Preference Dataset v2 enables training Human Preference Score v2, described as a state-of-the-art preference prediction model.
  • Human Preference Score v2 is intended to better align text-to-image generation model evaluation with human values and judgments.

Checklist … C CLIP Details

The supplementary material documents checklist compliance, prompt cleaning and categorization, annotation guidance, and the CLIP foundation used for human-preference prediction. It also records reproducibility, licensing, safety, and compensation details.

  • Checklist: The dataset and pre-trained model are slated for release, with participant consent discussed in Sec. 3.3 and annotator wages reported as 20CNY/hour and 25CNY/hour.The reported rates correspond to annotators and checkers, respectively.
  • A Prompt Cleaning: ChatGPT cleans prompts by removing modifiers unrelated to image content, including resolution, sharpness, lighting, image quality, authors, and online platforms.The cleaned description is requested as one succinct sentence.
  • A Prompt Cleaning: Prompts are categorized into paintings, anime and cartoon, real photo, concept-art, or others using one-word ChatGPT responses.The Photo category is retained despite originating from COCO Captions to mitigate possible ChatGPT classification mistakes.
  • A Prompt Cleaning: DiffusionDB prompts are filtered at a 0.4 NSFW threshold and then checked by DALL·E 2’s stricter safety checker during image generation.This two-stage process further removes inappropriate vocabularies.
  • B Annotation: Annotators rank image groups using universal and personal aesthetic appeal, balancing text-image alignment and image quality.Aesthetics and fidelity can outweigh alignment when the alternative is severely distorted or blurry; when aesthetics are similar, alignment receives greater weight.
  • B Annotation: Annotators are instructed to scrutinize capitalized names and verify unfamiliar or mistranslated proprietary terms using online sample images and explanations.This guidance addresses misunderstandings introduced by machine translation.
  • C CLIP Details: CLIP is a vision-language model trained on massive noisy image-text pairs, learning joint image-text representations through image-caption matching.The passage motivates using this representation-learning framework for predicting human preference.

D Generation Details · E Pairwise Accuracy

The HPS v2 benchmark releases generated images with model-specific generation settings spanning multiple text-to-image systems. HPS v2 predictions align closely with human pairwise choices between most generative models in the test split.

  • D Generation Details: The benchmark provides generation details for each model and releases the corresponding images with HPD v2.
  • D Generation Details: Stable Diffusion v1.4 and v2.0 use 512 × 512 images, guidance scale 7.5, 50 steps, and a DDIM sampler.
  • D Generation Details: GLIDE, LAFITE, and VQ-Diffusion generate 256 × 256 images using their stated model configurations, with VQ-Diffusion using guidance scale 5.0 and 100 sampling steps.GLIDE uses guidance scale 3.0 for 27 DDIM steps, while LAFITE follows its default configuration using model*.
  • D Generation Details: Latent Diffusion, Versatile Diffusion, and community Stable Diffusion models use DPM Solver sampling with 25 steps and model-specific guidance scales.Latent Diffusion uses η = 0.3 and guidance scale 6.0; Versatile Diffusion and the listed community models use guidance scale 7.5.
  • D Generation Details: Other systems use distinct procedures, including FuseDream optimization, CogView2 category-specific styles, official DALL·E 2 API requests, VQGAN+CLIP optimization, and default DALL·E mini settings except grid_size = 1.DeepFloyd-XL uses staged generation and upsampling, producing 768 × 768 images after resizing and Stable Diffusion upscaling.
  • E Pairwise Accuracy: HPS v2 is evaluated on human-choice agreement between all pairs of generative models in the test split.The predictions align closely with human choices between most models, and model chrominance and luminance distributions are shown as a reference.

F Ablation Study · G Example Usages of HPS v2

The paper ablates HPS v2 training hyperparameters and presents example usages illustrating the metric’s sensitivity and accuracy, including evaluations on community models.

  • F Ablation Study: Table 10 reports ablation studies on learning rates for training HPS v2.
  • F Ablation Study: Table 12 reports ablation studies on the number of free layers in the text encoder.
  • F Ablation Study: Table 11 reports ablation studies on the number of free layers in the image encoder.
  • G Example Usages of HPS v2: Figure 11 shows examples of HPS v2 evaluated on community models.
  • G Example Usages of HPS v2: The example-usages section presents two applications of HPS v2 to demonstrate its sensitivity and accuracy.

G.1 Retrieval initialization · G.2 Evaluating adapted model in HPS v1

The method addresses Stable Diffusion’s initialization mismatch by retrieving a prior image and using its encoded noisy latent, improving generated layouts and HPS v2 scores. HPS v2 also enables quantitative evaluation of an adapted model previously validated only through user studies because HPS v1 had unresolved limitations.

  • G.1 Retrieval initialization: Stable Diffusion training never uses a fully random input latent, whereas inference starts from random noise, creating a training–inference mismatch.The authors relate initialization’s influence on image quality to this mismatch between training and inference noise schedules.
  • G.1 Retrieval initialization: Given a prompt, Dreamlike Photoreal 2.0 generates a prior image that is encoded into latent space at a training-like noise level.This encoded noisy latent is used to initialize Stable Diffusion inference.
  • G.1 Retrieval initialization: The retrieved image’s noisy latent replaces the default initialization previously sampled from N(0, I) without a prior.The method therefore incorporates image-derived information before denoising begins.
  • G.1 Retrieval initialization: Initializing from retrieved images improves Stable Diffusion outputs relative to default initialization according to HPS v2.Table 13 is cited as verifying the improvement brought by retrieved-image initialization.
  • G.1 Retrieval initialization: Random initial noise strongly influences the generated image’s global layout, so random initialization can produce undesired layouts.Figure 12 contrasts standard denoising with blending random initial noise and a prior image.
  • G.2 Evaluating adapted model in HPS v1: Wu et al.’s adapted Stable Diffusion model was previously shown to align better with human preference only through user studies, while HPS v1 was unsuitable for evaluation.The stated obstacles were under-studied bias and evaluation prompts, plus HPS v1’s incorporation of adapted-model training.

H Datasheet … If others want to extend/augment/build on this dataset, is there a mechanism for them to do so? If so, is there a process for tracking/assessing the quality of those contributions. What is the process for communicating/distributing these contributions to users?

The HPD v2 datasheet describes a self-contained dataset of prompt-image comparisons annotated by human preferences, with documented collection, preprocessing, distribution, maintenance, and limitations. It is publicly distributed through GitHub under Apache 2.0, while dataset updates are maintained by the authors and contributions from others have no official process.

  • H.1 Motivation / H.5 Uses: The dataset was created to support future academic Computer Vision research on human aesthetic preference and has been used to train HPS v2.It can also support other tasks related to human preference on generated images.
  • H.2 Composition: HPD v2 contains prompts and generated images, with each instance comprising two same-prompt images and one human choice annotation.The dataset includes 789,090 comparisons, 433,760 images, and 107,515 prompts; its benchmark also contains 3,200 unique prompts without paired images.
  • H.2 Composition: The dataset omits image-generation parameters, including diffusion steps and guidance scale, because the authors prioritize users’ preferences over how images are created.Generation hyperparameters are illustrated in Appendix D.
  • H.2 Composition / H.5 Uses: Human preference diversity may create noise or disagreement about aesthetic criteria, and the limited number of annotators may induce bias in future uses.The authors state that annotator identities were not tracked, although they report age, gender, and occupancy subpopulation statistics.
  • H.2 Composition: The prompts may contain sensitive information, including public figures and religious beliefs, and a very small amount of inappropriate content may remain despite filtering.Prompts were sourced from DiffusionDB; filtering retained prompts below an NSFW threshold and accepted by DALL·E2.
  • H.6 Data Distribution / H.2 Composition: The dataset is accessible to anyone online through a GitHub download link and is distributed under an Apache 2.0 license with a terms-of-use agreement.Distribution is planned for when the paper is accepted; the dataset is self-contained.
  • H.7 Dataset Maintenance: The authors maintain the dataset, will create errata as errors are identified, and record updates in GitHub when inappropriate content, model inconsistency, or other improvements warrant changes.The authors reserve the right to maintain the dataset, and no official mechanism is described for others to extend, assess, or distribute contributions.
Loading 2306.09341v2…