Learning Multi-dimensional Human Preference for Text-to-Image Generation

1. Introduction

• Statistical metrics can diverge from human judgments, while single-score preference models overlook differences across evaluation perspectives.The paper illustrates that preferences vary when people assess aesthetics, semantic alignment, and detail quality.Source: §1 · pp. 1–2
• MHP contains 607,541 generated images and 918,315 pairwise preference comparisons annotated for aesthetics, detail quality, semantic alignment, and overall score.The dataset uses prompts balanced across categories and images from diffusion, GAN, and autoregressive models.Source: §1 · p. 2
• MPS uses a unified model with preference conditions, using condition-related prompt words to guide image-text cross-attention before predicting preference scores.The condition mask is formed from similarities between preference attributes and prompt words.Source: §1 · p. 2
• MPS outperforms existing methods across three datasets in predicting overall and multidimensional human preferences.The reported comparison covers both overall and dimension-specific preferences.Source: §1 · p. 3

2.1. Text-to-image Generation and Evaluation

• Statistical image-quality metrics, including IS and FID, do not necessarily align with human perceptual preferences for generated images.The section also describes diffusion, GAN, and autoregressive approaches to text-to-image generation.Source: §2.1 · p. 3

2.2. Learning human preferences

• Prior preference datasets vary in scale and coverage, but commonly emphasize overall judgments or combine alignment and fidelity rather than separately annotating dimensions.PickScore lacks detailed multidimensional annotations, ImageReward combines alignment and fidelity, and AGIQA datasets are smaller than existing work.Source: §2.2 · p. 3
• MHP advances prompt collection, image generation, and preference annotation, with balanced prompts, images from nine text-to-image models, and multiple preference dimensions.The prompt collection contains 66,389 prompts; the image sources include diffusion, GAN, and autoregressive models.Source: §2.2 · pp. 3–4, Figure 2 · p. 3, Table 2 · p. 4

3.1. Prompt collection and annotation

• After annotators filtered 59,396 collected prompts, GPT-4-generated additions supplemented low-count categories, yielding 66,389 prompts with balanced category distribution.The initial prompts were collected from PromptHero, DiffusionDB, and an internal dataset; annotators also removed incoherent or incomprehensible additions.Source: §3.1 · p. 4

3.2. Image collection and annotation

• Images come from diffusion, GAN, and autoregressive models, with varied resolutions and aspect ratios; each model generates 2–4 images per prompt.Pairs include images from different models and from the same model using different random seeds.Source: §3.2 · p. 4
• Annotators compare images generated from the same prompt across aesthetics, detail quality, semantic alignment, and holistic overall assessment.Aesthetics covers composition, light, color, clarity, tone, style, depth of field, atmosphere, and artistry; detail quality includes texture and distortions, while alignment checks prompt consistency and missing or redundant content.Source: §3.2 · pp. 4–5

3.3. Statistics

• 66,389 prompts, 9 recent text-to-image models, 607,541 images, and 918,315 image pairs form the dataset, with 20% same-model and 80% cross-model pairs.Each pair is annotated by three people across four dimensions.Source: §3.3 · p. 5
• The dataset uses 898,315 training pairs, 10,000 validation pairs, and 10,000 test pairs; the test set includes diffusion, GAN, and autoregressive outputs.The test-set model families are included to represent the data distribution.Source: §3.3 · p. 5

4.1. Model Structure of MPS

• MPS extracts full image and prompt feature sequences with CLIP and fuses them through cross-attention, rather than relying only on selected image and text features.The preference condition is also represented as features; condition-specific prompt words such as color, light, and clarity can receive greater attention.Source: §4.1 · p. 5
• A condition mask gives prompt tokens relevant to the preference condition greater attention when computing the score.The condition is represented by attribute words, and token relevance determines which positions are masked or retained in cross-attention.Source: §4.1 · p. 6
• MPS supplements the fused feature’s CLS token with the prompt’s last-token feature to address short prompts that may lack condition-related words.The score combines these features with a learned scalar.Source: §4.1 · p. 6

4.2. Training

• Training minimizes KL-divergence between human pairwise preference annotations and softmax-normalized MPS predictions.The annotations encode either image preference or a tie; CLIP-H initializes the text and vision encoders, and training runs for 30,000 steps.Source: §4.2 · p. 6

5.1. Experimental Setup

• The evaluation compares CLIP Score, Aesthetic Score, Image Reward, HPS, and PickScore, using publicly available pretrained models without finetuning.The preference conditions are represented by word sets for aesthetics, detail quality, semantic alignment, and overall preference.Source: §5.1 · p. 6

5.2. Evaluation Results

• MPS achieves better overall-preference accuracy than relevant baselines on the ImageReward, HPD v2, and MHP test sets.The MHP comparison uses only data annotated with overall scores.Source: §5.2 · p. 6
• MPS outperforms related works by a large margin on multidimensional preference prediction in the MHP dataset.Table 4 evaluates scoring functions across multidimensional human preferences; the authors also report that MPS maintains high consistency across dimensions.Source: Table 4 · p. 7, §5.2 · p. 7
• MPS attends to different prompt words and image regions under different preference conditions, with the mask retaining condition-relevant prompt words.The visualizations associate aesthetics with color and lighting, semantic alignment with attributes and position, and detail quality with instances.Source: §5.2 · p. 7, Figure 6 · p. 8
• Cross-attention improves overall, aesthetic, and semantic-alignment accuracy, while the condition mask especially improves detail-quantity prediction.The ablation compares against a PickScore baseline trained on MHP; separately trained preference models perform worse than simultaneous multi-preference training.Source: §5.2 · pp. 7–8, Table 5 · p. 8

5.3. MPS Benchmark

• The MPS benchmark covers four scoring dimensions across 4,000 prompts in seven categories, with 500 prompts per category.Its scores are intended to help users select models based on personal preferences and select preferred generated images.Source: §5.3 · p. 8

• MHP comprises 918,315 human preference choices across aesthetics, semantic alignment, detail quantity, and overall score.The dataset improves on prior methods in prompt collection, image generation, and preference annotation.Source: §6 · p. 8
• MPS uses a unified network and condition mask to retain preference-related prompt words before combining the retained prompt with the image to compute a score.Source: §6 · p. 8
• MPS outperforms related works in predicting multidimensional human preferences across three datasets.Source: §6 · p. 8