PaperlayerView exact revision on arXiv

Source-linked AI summary

HPSv3: Towards Wide-Spectrum Human Preference Score

Yuhang Ma, Yunhao Shui, Xiaoshi Wu, Keqiang Sun, Hongsheng Li

arXiv:2508.03789v2cs.CV

TL;DR

Existing human-centric metrics have limited data coverage, feature extraction, and training schemes for evaluating text-to-image models. HPSv3 introduces a wide-spectrum preference dataset, a VLM-based uncertainty-aware scorer, and CoHP for iterative refinement, with experiments supporting robust evaluation and human-aligned quality improvement.

  • Problem

    Existing human-centric metrics rely on narrow data distributions, less effective multimodal feature extractors, and training schemes that overlook annotation uncertainty.

  • Method

    HPSv3 combines the HPDv3 dataset, VLM-based visual encoders, uncertainty-aware ranking loss, and CoHP for iterative image refinement.

  • Results

    HPSv3 provides robust wide-spectrum evaluation, while CoHP improves image generation quality through iterative HPSv3-guided selection without extra training data.

  • Takeaways & Limitations

    The framework supports human-aligned evaluation and image refinement across diverse generated and real-world image qualities.

  • Takeaways & Limitations

    HPDv3’s prompt distribution may underrepresent specialized domains such as medicine, biology, and physics.

Abstract

from arXiv · show

Evaluating text-to-image generation models requires alignment with human perception, yet existing human-centric metrics are constrained by limited data coverage, suboptimal feature extraction, and inefficient loss functions. To address these challenges, we introduce Human Preference Score v3 (HPSv3). (1) We release HPDv3, the first wide-spectrum human preference dataset integrating 1.08M text-image pairs and 1.17M annotated pairwise comparisons from state-of-the-art generative models and low to high-quality real-world images. (2) We introduce a VLM-based preference model trained using an uncertainty-aware ranking loss for fine-grained ranking. Besides, we propose Chain-of-Human-Preference (CoHP), an iterative image refinement method that enhances quality without extra data, using HPSv3 to select the best image at each step. Extensive experiments demonstrate that HPSv3 serves as a robust metric for wide-spectrum image evaluation, and CoHP offers an efficient and human-aligned approach to improve image generation quality. The code and dataset are available at the HPSv3 Homepage.

1. Introduction

Existing human-centric metrics are limited by narrow data coverage, weaker multimodal feature extraction, and training objectives that overlook annotation uncertainty. HPSv3 addresses these gaps with HPDv3, VLM-based scoring, uncertainty-aware ranking, and CoHP for iterative refinement.

  • Existing human-centric evaluation models are constrained by data distributions, feature extractors, and simplified training schemes.
  • Prior metrics mainly use diffusion-generated images, limiting evaluation of newer diffusion-transformer and autoregressive models.The same data coverage can also introduce bias through nonstandardized annotation practices.
  • CLIP- and BLIP-based extractors may provide less comprehensive multimodal representations, while KL-divergence optimization overlooks inconsistent annotation ground truth.
  • HPDv3: HPDv3 contains 1.08M text-image pairs and 1.17M annotated pairwise comparisons spanning state-of-the-art generated images and high-quality real photographs.Its construction combines newer model outputs, Internet photographs, and Midjourney data with 9–19 annotators per sample.
  • HPSv3: HPSv3 uses VLM visual encoders and an uncertainty-aware ranking loss to capture richer features and annotation uncertainty.
  • CoHP: CoHP iteratively refines generated images without extra training data, using HPSv3 to select the highest-quality candidate at each step.The process evaluates semantic alignment, realism, and aesthetic appeal during iterative selection.

3. Human Preference Dataset v3

HPDv3 extends earlier preference datasets across image quality, model types, prompt sources, and annotation reliability. It combines current generated outputs with real photographs and human comparisons to support broader evaluation and training.

  • Existing benchmarks lack high-quality real photographs and current generative-model coverage, weakening their evaluation framework.HPDv2 excluded high-quality real-world photographs and covered models only through Stable Diffusion 2.0.
  • HPDv3 contains 1.08M text-image pairs and 1.17M binary preference choices, with each decision validated by 9–19 specialist annotators meeting a 90% agreement threshold.
  • Data sources: HPDv3 adds high-fidelity photographs and outputs from current leading generative models to address HPDv2’s missing coverage.The collection retains HPDv2 prompts while adding outputs from models including FLUX.1-dev, Infinity, Hunyuan, Kolors, and SD3.
  • Data sources: The dataset uses categorized, distribution-aligned prompts, aesthetic filtering, Midjourney data, and pairwise user preferences to broaden its sources and image types.Midjourney contributes 331,955 user-generated images converted into pairwise data.
  • Dataset scope: HPDv3 covers 16 GAN-based, diffusion-based, and autoregressive models, exceeding the model diversity of other datasets.Its prompt sources include JourneyDB, Midjourney, and captions from high-quality photography.
  • Dataset scope: HPDv3 uniquely compares high- and low-quality real images with generated images, enabling comparisons relative to actual photographs.
  • Annotation reliability: 76.5% average annotation convergence for HPDv3 exceeds HPDv2’s 59.9%, indicating higher agreement in the newer dataset.

4. Human Preference Score v3

HPSv3 uses a VLM backbone and uncertainty-aware ranking to learn human preferences from paired comparisons. CoHP then applies HPSv3 iteratively to select models and images for progressive refinement.

  • Motivation and Outcome: HPSv3 is presented as a scalable evaluator that learns human preferences from annotations, while CoHP uses it to improve image quality and semantic alignment.The approach targets the high cost and limited scalability of direct human evaluation.
  • Architecture: HPSv3 uses QWen2-VL to extract features from images and text, then an MLP maps them to the final output.The architecture processes paired images with their text prompt and preference annotations.
  • Uncertainty-Aware Ranking: Uncertainty-aware ranking models each image reward as a Gaussian distribution rather than a single deterministic score.The model predicts µ and σ, representing the reward as r ∼ N(µ, σ) to account for annotator uncertainty.
  • Uncertainty-Aware Ranking: Uncertainty-aware ranking captures inconsistencies in pairwise annotations and improves ranking accuracy by modeling the underlying preference distribution.This approach avoids relying solely on one scalar score for hard or uncertain cases.
  • CoHP: CoHP has model-wise and sample-wise stages, using HPSv3 to evaluate candidates and guide iterative selection.Model-wise selection identifies a suitable model, while sample-wise selection evaluates generated image batches across rounds.
  • CoHP: In sample-wise refinement, the highest-scoring image becomes the reference for the next round, and the highest-scoring image across rounds is selected as final.The selected reference is blended with noise and combined with the original prompt before subsequent generation rounds.

5. Experiment

Experiments evaluate HPSv3 as a human-aligned preference metric and CoHP as an iterative image-refinement method. HPSv3 achieves the strongest reported alignment and CoHP improves image quality through repeated model- and sample-wise selection.

  • Preference Comparison: HPSv3 achieves the highest reported correlation with human preferences, with Spearman r = 0.94 and Kendall τ = 0.8222.It distinguishes models across the performance spectrum more effectively than HPSv2, PickScore, ImageReward, and CLIP.
  • HPDv3 Benchmark and Evaluation: 72.8%, 85.4%, and 76.9% accuracy are reported on the PickScore, HPDv2, and HPDv3 testsets, respectively.The results are presented as state-of-the-art accuracy scores across the evaluated testsets.
  • Ablation Study: Qwen2VL-7B improves HPDv3 performance by 10.6 pp over Qwen2VL-2B and 13.4 pp over CLIP under identical training conditions.The uncertainty-aware ranking loss improves accuracy across all datasets and raises PickScore test-set accuracy by 2.88%.
  • CoHP Evaluation: CoHP uses HPSv3 in model-wise and sample-wise stages to select candidates iteratively, progressively improving image quality and text-image alignment.The model-wise stage selects a suitable generator, while the sample-wise stage further enhances details and structural coherence.
  • CoHP Evaluation: CoHP-HPSv3 produces more coherent and faithful results than alternatives, which show oversaturation, semantic inconsistencies, or instability.Human evaluation reports an 87% win rate over CoHP-ImageReward and a clear advantage over CoHP-PickScore and CoHP-HPSv2.
  • Ablation Study: Four rounds balance computational cost and performance: the model-wise HPSv3 score rises from 11.34 in Round 1 to 11.69 in Round 4, then decreases slightly to 11.65 in Round 5.Sample-wise scores also improve through Round 4 before a marginal Round 5 drop.

6. Conclusion

The paper presents HPDv3, HPSv3, and CoHP as a wide-spectrum preference dataset, human-preference model, and iterative refinement approach. Experiments validate HPSv3 for evaluation and CoHP for improving image generation quality.

  • 6. Conclusion: HPDv3 contains 1.08 million text-image pairs and 1.17 million annotated comparisons from generative models and high-quality real images.It is designed as a wide-spectrum human preference dataset.
  • 6. Conclusion: HPSv3 uses VLM-based feature extraction and uncertainty-aware ranking loss for accurate image scoring.The model is trained on HPDv3 to better capture human preferences.
  • 6. Conclusion: CoHP performs iterative image refinement without requiring extra training data.It uses HPSv3 to select preferred images during the refinement process.
  • 6. Conclusion: Experiments validate HPSv3 as a robust benchmark and CoHP as an efficient, human-aligned method for enhancing image generation quality.The conclusion states these outcomes without reporting a single aggregate score.

Supplementary Material

The supplementary material details HPDv3’s heterogeneous image sources, category balancing, curation, generation settings, and pairwise comparison construction. It also documents residual dataset noise and the procedures used to improve representativeness and reliability.

  • Category Distribution: Prompt categories are organized into 12 classes, and HPDv3 most closely matches JourneyDB’s category proportions among the compared datasets.This alignment is reported as evidence that HPDv3 captures user-preference distributions more representatively.
  • Real Image Collection: Approximately 5M Internet images are collected across categories including architecture, people, objects, animals, plants, landscapes, products, and posters.The collection is intended to provide broad category coverage.
  • Limitations: The collected real-image set remains noisy and contains irrelevant samples requiring further refinement, while the aesthetic filtering model may favor oil-painting styles.These issues constrain how universally the curation scores represent image quality.
  • Real Image Collection: The curation pipeline filters images below aesthetic score 4.0 and applies category-specific proportional selection, producing 58k high-quality real images.The selected-image distribution is reported to remain aligned with the target category proportions.
  • Image Sources and Dataset Composition: HPDv3 combines high-quality real photographs with autoregressive, DiT-based, diffusion, and Midjourney-generated images, totaling 1.08M text-image pairs.The source composition spans major image-generation model types and real-world photographs.
  • Image Generation: Real-image descriptions preserve original aspect ratios, while images from HPDv2 and JourneyDB prompts use square dimensions.The protocol aims to preserve structure for real-image-derived prompts while maintaining consistency elsewhere.
  • Pairwise Data Construction: Images sharing a prompt are paired across models so relative generative performance can be evaluated and used for annotation and model training.Pairwise comparisons isolate model differences under identical textual prompts.

4. Annotation Details

The annotation process uses standardized pairwise comparisons, multidimensional criteria, repeated expert judgments, and convergence analysis. Supplementary materials describe the interface, annotator demographics, quality controls, and dataset composition.

  • Annotator Profile: The annotator pool is 56.13% female and 43.87% male, with 88.95% aged 18–40 and the 21–30 group largest at 38.65%.These demographics are reported in the supplementary annotator profile.
  • Annotation Interface: Annotators compare two images generated from the same prompt through an interface that displays both images and the prompt.They select the preferred image according to the evaluation guidelines.
  • Quality Control: Each image is scored by 9−19 experts, with an inter-annotator agreement threshold of 0.9 for reliability.The same rigorous methodology is maintained across dataset-creation stages.
  • Evaluation Criteria: The guidelines assess prompt alignment, aesthetic quality, and overall coherence to capture multidimensional image preferences.These criteria are intended to identify meaningful quality differences while minimizing subjective bias.
  • Convergence Analysis: Annotation convergence is measured by agreement among annotators judging image pairs associated with the same text prompt.Figure S6 reports convergence by category.

5. HPSv3 Training Details

HPSv3 is trained on a diverse mixture of curated, generated, and real-world preference data. Its loss analysis shows that KL-divergence and Bradley-Terry formulations reduce to the same pairwise logistic ranking objective.

  • Training Dataset: 1.5 million pairwise samples combine HPDv3, Pick-A-Pic, ImageReward, and real user preferences from Midjourney.The mixture includes 350,000 Pick-A-Pic samples, 120,000 ImageReward samples, and 150,000 Midjourney pairs.
  • Training Dataset: HPDv3 includes confidence-filtered comparisons and a manually curated golden set that expands model and pairwise diversity.Comparisons with confidence scores of at least 0.95 are selected, while 250,000 filtered HPDv2 pairs are randomly sampled and some golden samples are duplicated.
  • Dataset Ablation: HPDv3 training achieves 66.8% on ImageReward, 72.8% on PickScore, 85.4% on HPDv2, and 76.9% on HPDv3, leading all tested datasets.The ablation evaluates models trained on HPDv2, ImageReward, PickScore, and HPDv3 across four test sets.
  • Loss Function: The ranking setup compares a preferred sample’s reward rh against a dispreferred sample’s reward rl through a negative log-likelihood objective.The Bradley-Terry form maximizes the probability of the higher-ranked sample over the lower-ranked sample.
  • Loss Function: KL-divergence and Bradley-Terry formulations ultimately share the same pairwise logistic ranking loss.The clarification derives this equivalence despite the formulations’ different theoretical origins.

6. HPDv3 Dataset Visualization

HPDv3 visualization illustrates pairwise preference data and a standardized benchmark spanning generated images and real photographs. The benchmark combines reference-based and text-only evaluation across 12 categories.

  • Dataset Visualization: HPDv3 pairs images generated from the same prompt across models and real photographs, with annotators judging every image combination.This produces detailed pairwise preference data capturing different visual representations of the same concept.
  • Benchmark: The HPDv3 Benchmark contains 1,000 prompts for each of 12 categories, covering varied styles and prompt lengths.Prompts are drawn from HPDv3, HPDv2, and JourneyDB.
  • Benchmark: HPDv3 prompts include real-world reference images, whereas HPDv2 and JourneyDB prompts are text-only.This distinguishes reference-based comparison from evaluation based solely on textual input.
  • Benchmark: The benchmark assesses image synthesis from multiple perspectives by combining reference-image alignment with text-conditioned generation.Its design covers different content categories and prompt styles.

7. More Result of CoHP

CoHP results show iterative selection and refinement across models and samples, while comparisons with other preference models examine effects on quality, diversity, and human alignment.

  • CoHP Results: CoHP’s model-wise stage selects strong models and its sample-wise stage refines details across multiple iterations.Examples use Flux, Kolors, and Playground v2.5; the model-wise stage emphasizes semantic understanding and composition.
  • Preference Models: Comparative visualizations evaluate how different human preference models influence generated-image quality, diversity, and human alignment.Figures S13 and S14 provide expanded comparisons within the CoHP framework.
  • HPSv3 as Reward Model: HPSv3 is used as a reward model in DanceGRPO to guide realism, aesthetic quality, text-image alignment, and human preference.DanceGRPO scores sampled diffusion trajectories and applies policy-gradient optimization using reward-based advantages.
  • Reward Hacking: MHPSv2 produces more meaningless accessories and decorative effects than MHPSv3 in the reported DanceGRPO examples.The passage characterizes these additions as reward hacking and reports significantly less of this phenomenon with MHPSv3.

9. Term of Use of HPDv3

HPDv3 includes Internet-sourced images and is restricted to noncommercial use, with stated commitments to licensing, attribution, and responsible use. The dataset also faces unresolved annotator disagreement in some cases.

  • Terms of Use: HPDv3 contains Internet-sourced images that are not MizzenAI’s property, and its funding, dataset, and models are limited to noncommercial use.The stated usage position is linked to fair-use principles for educational and research purposes.
  • Ethical Use: The authors state that materials were vetted for licensing and attribution and encourage ethical use to avoid intellectual-property misuse.The commitment is presented as part of the dataset’s responsible-use policy.
  • Limitation: Some annotator groups struggled to converge on a consistent score, which can affect dataset accuracy and its use as a definitive benchmark.The multi-annotator mechanism is nevertheless described as valuable for improving reliability overall.

10. Limitation

HPDv3’s broad coverage may not generalize equally to specialized domains, and its unified scoring metric limits fine-grained evaluation.

  • Prompt Distribution Bias: Prompt categories drawn primarily from JourneyDB may underrepresent specialized domains such as medicine, biology, and physics.Models designed for medical imaging or scientific diagram generation might therefore not perform accurately when benchmarked with HPDv3.
  • Prompt Distribution Bias: HPDv3’s prompt distribution captures typical generative use cases but may limit evaluation of models tailored to specialized applications.The stated boundary concerns generalizability and applicability in contexts requiring unique data.
  • Unified Scoring Metric: A unified scoring metric evaluates text-image pairs holistically across all dimensions rather than exposing fine-grained dimensions.This simplifies evaluation but does not provide detailed insight into individual quality dimensions.
Loading 2508.03789v2…