Source-linked AI summary
Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation
Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, Omer Levy
TL;DR
Text-to-image research lacked a large, open dataset of authentic user preferences. The paper collects these judgments through a web app, trains PickScore, and finds that it predicts preferences and aligns model evaluation with human rankings better than existing automatic metrics, while also improving model outputs through ranking.
Problem
Text-to-image generation lacked a large, open dataset of human preferences over state-of-the-art image-generation models.
Method
The authors collect consented preferences from real users through a web app and fine-tune CLIP-H to train PickScore.
Results
PickScore achieves 70.5% preference-prediction accuracy versus 68.0% for humans and correlates 0.917 with human preferences, compared with -0.900 for FID.
Takeaways & Limitations
Pick-a-Pic prompts and PickScore provide a basis for human-aligned text-to-image model evaluation and ranking-based improvement.
Takeaways & Limitations
Some NSFW content, careless judgments, and user biases may affect the dataset's quality and reliability.
Abstract
from arXiv · showhide
The ability to collect a large dataset of human preferences from text-to-image users is usually limited to companies, making such datasets inaccessible to the public. To address this issue, we create a web app that enables text-to-image users to generate images and specify their preferences. Using this web app we build Pick-a-Pic, a large, open dataset of text-to-image prompts and real users' preferences over generated images. We leverage this dataset to train a CLIP-based scoring function, PickScore, which exhibits superhuman performance on the task of predicting human preferences. Then, we test PickScore's ability to perform model evaluation and observe that it correlates better with human rankings than other automatic evaluation metrics. Therefore, we recommend using PickScore for evaluating future text-to-image generation models, and using Pick-a-Pic prompts as a more relevant dataset than MS-COCO. Finally, we demonstrate how PickScore can enhance existing text-to-image models via ranking.
1 Introduction
Pick-a-Pic addresses the lack of large, open human-preference data for text-to-image generation by collecting authentic user judgments. The dataset supports PickScore, model evaluation, and ranking-based improvement of text-to-image models.
- The field lacked a large, open dataset of human preferences over state-of-the-art text-to-image models.
- Pick-a-Pic collects prompts, generated-image pairs, and preference labels from real users through a consent-based web application.Examples include a preferred image or a tie when neither image is significantly preferred.
- PickScore fine-tunes CLIP-H on human preferences to estimate which generated image users prefer for a prompt.Its objective maximizes the probability of selecting a preferred image, including tie cases.
- 70.5% accuracy makes PickScore outperform humans at 68.0% for predicting user preferences.Zero-shot CLIP-H achieves 60.8%, while the aesthetics predictor achieves 56.8%.
- PickScore correlates more strongly with human model rankings than other automatic metrics and supports evaluation with Pick-a-Pic prompts rather than MS-COCO captions.Against MS-COCO captions, PickScore correlates 0.917 with human preferences, whereas FID correlates -0.900.
- Ranking candidate outputs with PickScore improves selections preferred by human raters over selections from CLIP-H, an aesthetics predictor, and the vanilla model.The reported win rates are 71.3%, 85.1%, and 71.4%, respectively.
2 Pick-a-Pic Dataset
Pick-a-Pic logs interactive choices from text-to-image users to create a large preference dataset. Its design captures prompt-specific judgments, supports model comparisons, and favors two-image annotation with ties for engagement and agreement.
- Pick-a-Pic contains over 500,000 examples and 35,000 distinct prompts, each pairing two generated images with a preference or tie label.Images come from Stable Diffusion 2.1, Dreamlike Photoreal 2.0, and Stable Diffusion XL variants across guidance scales.
- The web app repeatedly presents two images, records the user's preferred option or tie, and replaces the rejected image with a newly generated one.Users can clear or edit prompts, allowing the interaction to continue with revised requests.
- The dataset targets intrinsically motivated users recruited through social media rather than paid crowd workers, with quality-control measures to limit misuse.
- Annotation Methodology: Two images with ties produced higher user engagement and inter-rater agreement than the tested alternatives.
- Preprocessing: The dataset is filtered for NSFW phrases and banned users, while the authors acknowledge that some NSFW content remains.
- Model Selection and Evaluation: User preferences can compare generation settings and backbone models, with users usually preferring Dreamlike Photoreal 2.0 over Stable Diffusion 2.1.
3 PickScore
PickScore is a CLIP-based scoring function trained on human preferences to estimate how well a generated image matches a prompt. It uses preference-aware optimization and is intended for predicting user preferences and supporting downstream text-to-image evaluation and improvement.
- Scoring function: PickScore combines a CLIP-style prompt-image scorer with a variant of InstructGPT’s reward-model objective, trained on Pick-a-Pic preferences.The objective is designed to favor preferred images and accommodate ties.
- Results and uses: PickScore predicts held-out user preferences better than other publicly available scoring functions, surpassing expert human annotators.The paper identifies model evaluation, image ranking, dataset construction, and weak supervision as potential uses.
- Scoring function: The scoring function represents prompts and images as d-dimensional transformer-encoder vectors and returns their temperature-scaled inner product.The learned scalar temperature T is inherited from CLIP.
- Training objective: The training objective minimizes KL-divergence between the user’s preference distribution and softmax-normalized scores for two candidate images.Preference vectors are [1, 0], [0, 1], or [0.5, 0.5] for the first choice, second choice, or a tie.
- Training objective: Inverse prompt-frequency weighting reduces the risk of overfitting when multiple examples come from the same prompt.The weighting is applied when reducing the loss across examples in a batch.
4 Preference Prediction
PickScore predicts real-user image preferences more accurately than the evaluated baselines, including expert annotators, while revealing disagreements with CLIP-H.
- PickScore outperforms all evaluated baselines, including expert human annotators, on predicting human preferences.
- Figure 4 compares CLIP-H and PickScore disagreements on validation images, marking human-preferred images with green borders.
- 70.5 ± 0.142 PickScore accuracy exceeds human experts at 68.0, while CLIP-H reaches 60.8 and the aesthetics score 56.8.The evaluation metric accounts for ties, and the random chance baseline is also 56.8.
- 66.7 HPS and 61.1 ImageReward scores are both below PickScore on the Pick-a-Pic test set.
- PickScore often selects more aesthetically pleasing images than CLIP-H, sometimes at the cost of prompt faithfulness.
5 Model Evaluation
The paper argues that Pick-a-Pic prompts better represent text-to-image users’ intents than MS-COCO captions and that PickScore aligns more closely with human model rankings than FID.
- Model Evaluation Prompts: Pick-a-Pic prompts are sampled from real users and are presented as a better representation of natural text-to-image intents than MS-COCO captions.MS-COCO captions mainly depict everyday objects and people in photographed settings, whereas users also seek fictional images.
- FID: PickScore scores each image conditioned on its prompt, whereas FID measures set-level resemblance to real images and ignores prompts.
- FID: 0.917 PickScore correlation with human model win rates exceeds FID’s -0.900 correlation on MS-COCO captions.The comparison uses 100 captions, nine models, and human expert rankings with ties.
- FID: Higher classifier-free guidance scales can produce more vivid images that humans prefer while receiving worse FID scores because they differ from MS-COCO’s image distribution.
- Other Evaluation Metrics: The broader evaluation uses 14,000 real-user preferences from Pick-a-Pic test prompts spanning 45 models.
6 Text-to-Image Ranking
The paper evaluates ranking as a way to improve text-to-image outputs by generating many candidates and selecting the highest-scoring image, finding PickScore’s choices consistently preferable to baseline selections.
- The ranking procedure generates 100 images per prompt and selects one using PickScore, CLIP-H, an aesthetics score, or random choice.Images are generated with Dreamlike Photoreal 2.0 at CFG scale 7.5, including a null-template control.
- Human annotators compare PickScore’s selected image against each alternative scoring function’s selection.
- PickScore consistently selects images that humans prefer over baseline selections.
- Figure 8 contrasts a vanilla model image with the image selected by PickScore from 100 generations.
- 68.5% of PickScore’s choices have higher aesthetic scores than CLIP-H’s choices.
- 90.5% of PickScore’s choices have higher CLIP-H text-alignment scores than images selected by the aesthetics scorer.
7 Related Work
Prior work collected human judgments for text-to-image outputs using synthetic prompts, crowd ratings, or scraped community ratings. Pick-a-Pic differs by collecting general preferences from real users, while performance differences against related scorers remain attributable to multiple factors.
- Text-to-image preference research has received less attention than analogous work in NLP, partly because large open preference datasets were unavailable.
- SAC contains almost 200,000 aesthetic ratings, whereas Pick-a-Pic compares generated images and captures general user preferences, including ties.
- Lee et al. train a scoring function on 30,000 crowd-labeled examples generated from synthetic prompts covering count, color, and background challenges.
- ImageReward uses crowd-worker judgments to produce 136,892 examples from 8,878 prompts, while HPS uses about 25,000 ratings covering about 100,000 images from StabilityAI Discord.
- PickScore outperforms HPS and ImageReward on Pick-a-Pic, but differences in implementation, dataset scale, and data distributions prevent isolating the specific causes.Pick-a-Pic is more than five times larger than the data used to train HPS and ImageReward.
8 Limitations and Broader Impact
The dataset may contain NSFW content, careless judgments, and user biases that can affect its quality, reliability, and broader impact. These issues should be considered when using the collected preferences.
- NSFW images and prompts may bias the dataset, while some users may have made judgments without due care.
- User preferences may contain biases that are reflected in the collected data and may affect its overall quality and reliability.
9 Conclusions
The paper builds an open dataset from willingly collected text-to-image user preferences and trains PickScore to predict those preferences. It reports stronger alignment with human judgments than public automatic metrics, more relevant evaluation using Pick-a-Pic prompts, and improved model selection through ranking.
- The web application collects over half a million examples of prompts, generated images, and user-labeled preferences in the open Pick-a-Pic dataset.
- PickScore is a state-of-the-art text-image scoring function that achieves superhuman performance when predicting user preferences.
- PickScore aligns better with human judgments than other publicly available automatic metrics, while Pick-a-Pic prompts support more relevant model evaluation than FID over MS-COCO.
- The paper demonstrates that using PickScore to select images can improve the quality of text-to-image models.
Appendix
The appendix contrasts the natural-language prompts in Pick-a-Pic with descriptive MS-COCO captions and reports an alternative scoring objective that underperforms PickScore. The comparison highlights differences in prompt style and the effect of the training objective.
- Comparing Pick-a-Pic Prompts with MS-COCO Captions: Pick-a-Pic prompts include imaginative concepts, stylistic modifiers, and detailed generation instructions, unlike the predominantly literal MS-COCO captions shown.
- Training a Scoring Function: An alternative CLIP-like loss incorporates remaining batch examples as in-batch negatives.
- Training a Scoring Function: 65.2 accuracy was achieved on the Pick-a-Pic test set by the alternative objective, which PickScore significantly outperformed.