PaperlayerView exact revision on arXiv

Source-linked AI summary

DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Models

Jaemin Cho, Abhay Zala, Mohit Bansal

arXiv:2202.04053v3cs.CVcs.AIcs.CL

TL;DR

Text-to-image models have lacked detailed evaluation beyond realistic generation and conventional alignment and quality metrics. The paper introduces PAINTSKILLS and social-bias analyses, finding gaps in counting and spatial reasoning and specific gender and skin-tone biases learned from web image-text pairs. These evaluations support measuring progress toward stronger reasoning and less biased representations.

  • Problem

    Detailed quantitative evidence about text-to-image models’ visual reasoning abilities and social biases is limited beyond image-text alignment and image quality evaluation.

  • Method

    The paper evaluates visual reasoning with PAINTSKILLS and object detectors, and evaluates gender and skin-tone bias distributions across generated professions and attributes.

  • Results

    Models perform better at object recognition than counting or spatial relations, while learning specific gender and skin-tone biases from web image-text pairs.

  • Takeaways & Limitations

    The evaluation provides a way to measure progress on challenging visual reasoning skills and understanding social biases in text-to-image models.

  • Takeaways & Limitations

    The conclusions reflect four popular publicly available models and English-heavy training datasets, not necessarily all models or languages.

Abstract

from arXiv · show

Recently, DALL-E, a multimodal transformer language model, and its variants, including diffusion models, have shown high-quality text-to-image generation capabilities. However, despite the realistic image generation results, there has not been a detailed analysis of how to evaluate such models. In this work, we investigate the visual reasoning capabilities and social biases of different text-to-image models, covering both multimodal transformer language models and diffusion models. First, we measure three visual reasoning skills: object recognition, object counting, and spatial relation understanding. For this, we propose PaintSkills, a compositional diagnostic evaluation dataset that measures these skills. Despite the high-fidelity image generation capability, a large gap exists between the performance of recent models and the upper bound accuracy in object counting and spatial relation understanding skills. Second, we assess the gender and skin tone biases by measuring the gender/skin tone distribution of generated images across various professions and attributes. We demonstrate that recent text-to-image generation models learn specific biases about gender and skin tone from web image-text pairs. We hope our work will help guide future progress in improving text-to-image generation models on visual reasoning skills and learning socially unbiased representations. Code and data: https://github.com/j-min/DallEval

1. Introduction

The paper argues that existing text-to-image evaluations overlook visual reasoning and social biases. It introduces evaluations showing weaknesses in counting and spatial understanding, alongside learned gender and skin-tone biases.

  • Existing evaluations mainly measure image-text alignment and image quality, leaving broader abilities and limitations insufficiently analyzed.
  • PAINTSKILLS measures object recognition, object counting, and spatial relation understanding using controlled compositional images and object detection.The dataset uses a 3D simulator with uniform distributions over objects and relations, and its detector-based scores correlate with human judgment.
  • Recent models perform relatively well at object recognition but show a large gap from upper-bound accuracy in object counting and spatial relation understanding.
  • The social-bias evaluation measures gender and skin-tone distributions across professions and attributes in generated images.
  • Recent text-to-image models learn specific gender and skin-tone biases from web image-text pairs, including profession- and attribute-linked associations.
  • The authors conclude that current models offer initial progress while leaving avenues for improving challenging visual reasoning and understanding social biases.

2. Related Works

Prior work commonly evaluates text-to-image systems through image quality and image-text alignment, while related research studies multimodal social biases. The paper positions PAINTSKILLS as a more interpretable evaluation of compositional visual reasoning.

  • Text-to-Image Generation Models: Text-to-image generation research includes multimodal transformer language models and diffusion models that learn image representations conditioned on text.
  • Metrics for Text-to-Image Generation: Common automated metrics assess image quality through diversity and visual reality, or image-text alignment through similarity between generated images and text descriptions.
  • PAINTSKILLS: PAINTSKILLS evaluates object recognition, object counting, and spatial relation understanding by generating prompts that require each skill.
  • PAINTSKILLS: Unlike existing alignment metrics, PAINTSKILLS uses object detection to provide more interpretable evidence for reasoning about generated content.
  • Measuring Bias in Multimodal Models: Related multimodal research examines social biases in visually grounded embeddings, image-text datasets, and images of people across occupations and gender.

3. PAINTSKILLS: A Diagnostic Evaluation Dataset for Compositional Visual Reasoning Skills

PAINTSKILLS is a controlled diagnostic dataset for measuring object recognition, object counting, and spatial relation understanding in text-to-image models. It uses uniformly distributed scene configurations and 3D-simulated rendering to evaluate compositional visual reasoning while reducing statistical shortcuts.

  • Skills: PAINTSKILLS measures object recognition, object counting, and spatial relation understanding as compositional visual reasoning skills.Accuracy is calculated from detection results on generated images.
  • Skills: Object recognition tests whether a generated image contains the intended object class from a textual description.The example task describes a specific class, such as an airplane.
  • Skills: Object counting tests whether a generated image contains the specified number of objects of a class.The task varies the described count, such as three dogs.
  • Skills: Spatial relation understanding tests whether two generated objects appear with the specified relation, such as one object being right of another.The relation is encoded in scene configurations and corresponding text prompts.
  • Scope: The three selected skills do not cover other image-generation abilities, including text rendering.The paper focuses on object control skills viewed as fundamental to more complex skills.
  • Dataset Collection: PAINTSKILLS controls objects, attributes, and relations with uniform distributions, then generates prompts and 3D-rendered images from those configurations.The simulator uses Unity5, supports object attributes such as class, location, and scale, and can be extended with custom objects and attributes.
  • Dataset Collection: The dataset includes separate train and test scene sets for object recognition, object counting, and spatial relation understanding.The reported train/test scene counts are 23,250/2,325, 21,600/2,160, and 13,500/2,700, respectively.

4. Evaluations

The evaluation measures visual reasoning and social biases in text-to-image models through controlled skill tests and diagnostic prompts. It uses automated detectors, human validation, and distribution-based bias metrics.

  • The evaluation covers object recognition, object counting, spatial relation understanding, and social bias in generated images.
  • Visual Reasoning Skill Evaluation: PAINTSKILLS evaluates visual reasoning by applying DETR detectors to generated images, with detector accuracy on test images treated as an upper bound.The three skills are object recognition, counting, and spatial relations.
  • Visual Reasoning Skill Evaluation: Object recognition measures whether the detector identifies the target class, while counting measures whether it identifies all target-class objects.Recognition uses detector class, confidence, and ground-truth class; counting compares detected objects with the target count.
  • Visual Reasoning Skill Evaluation: Spatial evaluation requires correct identification of both target object classes and their pairwise relation among above, below, left, and right.Relations are determined from the directions between object positions in 2D coordinates.
  • Social Bias Evaluation: Social-bias evaluation generates images from 252 prompts combining 3 gender formulations with 83 professions, plus three prompts without professions.Nine images are sampled for each diagnostic prompt to assess gender, skin tone, and attributes.
  • Social Bias Evaluation: Gender, skin tone, and attributes are detected automatically and checked with human evaluation before comparing category distributions with an unbiased uniform distribution.Gender uses two categories, skin tone uses the 10-tone Monk Skin Tone Scale, and attributes are analyzed by detection frequency.
  • Social Bias Evaluation: MAD measures distributional bias relative to uniformity, reaching 0 for uniform distributions and its maximum when all examples fall in one category.The mean reference is 0.5 for gender and 0.1 for skin tone.

5. Experiments and Results

Experiments evaluate visual reasoning and social biases across public text-to-image models. Models recognize single objects relatively well, but counting, spatial relations, and demographic representation remain problematic.

  • Visual reasoning: 98.0% average oracle accuracy and 2.8% accuracy on randomly shuffled ground-truth images support PAINTSKILLS as a placement-sensitive automated evaluation.High oracle performance contrasts with the shuffled-image baseline, which cannot score highly without correct object placement.
  • Visual reasoning: Stable Diffusion reaches 96.2% accuracy on object skill, while minDALL-E performs better on counting and spatial skills.The comparison covers three models evaluated after PAINTSKILLS finetuning.
  • Visual reasoning: All models show a large gap from upper-bound accuracy in counting and spatial understanding, often generating incorrect object quantities or missing spatially described objects.Stable Diffusion may generate five instead of three dogs, one instead of two bicycles, or omit a person or umbrella.
  • Fine-grained analysis: Counting accuracy decreases as prompts contain more objects, while spatial accuracy is similar across four relations; Stable Diffusion’s bottleneck is distinguishing relations.Spatial tasks require both correct objects and satisfaction of the specified relation.
  • Evaluation validation: DETR-based and expert human evaluations achieve similar accuracy and identical relative model performance across all three skills.The expert assessed 150 images per skill across three models.
  • Social bias: All three models tend to generate male images overall, while profession-specific gender tendencies differ between models.For singer prompts, minDALL-E skews male, whereas Karlo and Stable Diffusion skew female.
  • Social bias: Gendered attribute patterns associate skirts with woman prompts and suits, jackets, and ties more frequently with man prompts.These differences are measured using gender-specific prompts.
  • Social bias: All models concentrate skin-tone outputs around Monk Skin Tone categories 5 and 6, marginalizing very light and dark tones.Their MAD values of 0.16–0.18 resemble concentrated 1-hot and 2-hot distributions rather than a uniform distribution.

6. Conclusion

The paper adds visual reasoning and social-bias evaluation to text-to-image assessment. Its results show stronger object recognition than counting or spatial understanding, alongside learned gender and skin-tone biases.

  • Contributions: The paper introduces PAINTSKILLS to measure object recognition, object counting, and spatial relation understanding.The dataset is designed as a compositional diagnostic evaluation resource.
  • Visual reasoning: Recent models perform better at object recognition than counting or spatial relations, with a large gap from upper-bound accuracy on the latter two skills.The conclusion identifies counting and spatial understanding as the more difficult evaluated capabilities.
  • Social bias: The models learn specific gender and skin-tone biases from web image-text pairs.The conclusion frames these findings as evidence from the paper’s bias evaluations.
  • Implications: The evaluation is intended to provide insights for future work on challenging visual reasoning skills and social biases.The authors position the work as a way to measure progress in these areas.

7. Limitations

The evaluation has methodological and scope boundaries. It relies partly on pretrained evaluators, covers limited traits and reasoning skills, represents selected public models, and focuses on English-heavy training data.

  • Evaluation scope: Pretrained evaluation models may not provide robust evaluation for text-to-image models trained on unseen data distributions.This limitation applies to some of the paper’s analyses.
  • Bias scope: The study covers gender and skin tone as parts of physical appearance, leaving more diverse phenotypes for future investigation.The authors explicitly limit the appearance traits examined.
  • Reasoning scope: PAINTSKILLS measures three visual reasoning skills, while future work could add 3D spatial relations and text rendered in images.The current benchmark does not cover these more complex reasoning capabilities.
  • Model scope: The takeaways reflect four popular publicly available models and may not generalize to all text-to-image models, including the unavailable original DALL-E.The authors identify evaluated-model availability as a scope boundary.
  • Language scope: The evaluation focuses on models trained on English-heavy datasets, although the methods are described as extensible to other languages.Multilingual model evaluation is left for future work.

B.5. Additional Image Generation Samples

Additional materials provide further PAINTSKILLS-generated examples and the gender- and skin-tone-neutral prompts used for social-bias evaluation.

  • Image samples: Additional samples show images generated by models finetuned on PAINTSKILLS.These examples supplement the main visual reasoning results.
  • Evaluation prompts: The appendix lists gender- and skin-tone-neutral prompts used in the social-bias evaluation.These prompts include occupational descriptions such as a person who works as a nurse.

C.2. Evaluation of Automated Detection Models

The evaluation compares automated detectors for gender, attributes, and skin tone, validating them against human judgments and testing alternative skin-tone estimation pipelines.

  • BLIP-2 is selected for gender and attribute detection, while colorspace-based skin segmentation is selected for skin-tone detection.The evaluation compares detector accuracy and reliability before choosing the final models.
  • 82.38% accuracy for BLIP-2 versus 65.83% for CLIP on the Adience gender dataset supports BLIP-2 for gender detection.BLIP-2 also shows lower bias and higher recall than CLIP in the reported comparison.
  • 99.2% accuracy against human evaluation indicates that BLIP-2 reliably detects gender when human annotators can reasonably determine it.Images for which gender could not be reasonably determined were skipped.
  • Face landmark crop plus average albedo ITA provides the most accurate skin-tone estimation among the tested configurations.The result highlights the importance of accurate skin segmentation and accounting for lighting conditions.
  • 91.71% BLIP-2 accuracy versus 78.63% for CLIP in expert annotation supports BLIP-2 for attribute detection.The attribute task asks whether a person is wearing specified items such as a suit or jeans.

C.3. Additional Gender/Skin tone/Attribute Detection Results.

Additional results reveal systematic gender and attribute associations, while generated skin tones cluster near the center of the Monk Skin Tone scale across professions and attributes.

  • Gender Detection Results: All three models show an overall preference toward male outputs, although profession-specific gender bias varies substantially across models.The profession “Manager” is cited as an example with a broad range of bias between models.
  • Skin Tone Detection Results: Across attributes and professions, generated skin tones generally remain close to the center of the Monk Skin Tone scale.The reported profession-level averages are fairly similar across models.
  • Attribute Detection Results: All three models generate dresses and skirts more frequently for woman prompts and suits, jackets, and ties more frequently for man prompts.Per-prompt attribute distributions are reported separately for minDALL-E, Karlo, and Stable Diffusion.
  • The evaluation combines gender, skin-tone, and attribute detection with image-text alignment and image-quality assessments.Image-text alignment uses captioning, retrieval, and human evaluations, while image quality uses FID.

D.3. Image-Text Alignment Results

Image-text alignment and image quality are evaluated with automated and human measures, with Stable Diffusion generally leading the reported comparisons.

  • Stable Diffusion > X-LXMERT ≈ minDALL-E > DALL-ESmall on the reported image-text alignment evaluations.The evaluations use captioning, retrieval, and human annotator scores, with upper bounds supplied by real-image or maximum-score references.
  • Stable Diffusion achieves the lowest FID, followed by minDALL-E, while X-LXMERT has lower FID than DALL-ESmall.Lower FID indicates greater similarity between generated and real-image feature statistics.

E. Human Evaluation Setup

The paper’s human-evaluation setup combines expert judgments for visual reasoning and detection with crowd ratings for image-text alignment, across controlled prompts and multiple generation models.

  • Visual Reasoning Evaluation: Expert annotators select required components in generated images to evaluate object recognition, object counting, and spatial relation understanding.Counting additionally requires selecting the number of occurrences.
  • Image-Text Alignment Evaluation: Image-text alignment receives five-worker agreement scores on a 1–5 Likert scale for each image-caption pair.The evaluation uses Amazon Mechanical Turk and takes agreement among the five ratings as the final answer.
  • Human Evaluation Setup: Crowd workers must be from the United States, Great Britain, Australia, or Canada, with at least 95% approval and 1,000 approved tasks.These qualifications define the reported MTurk worker pool.
  • Bias Evaluation: Figures 8 and 9 show skin-tone distributions across attributes and professions, and Figure 10 compares human and automated skin-tone estimates.The displayed distributions are described as concentrated in the center few tones.
  • Bias Evaluation: Skin-tone bias is measured on the Monk Skin Tone scale, while gender bias uses profession-neutral prompts and attribute analyses compare gendered prompts.Tables report profession-level skin-tone and gender patterns alongside attribute occurrence across models.
  • Model Details: The study evaluates DALL-ESmall, minDALL-E, X-LXMERT, Stable Diffusion, and Karlo using their default sampling strategies.The model set includes both transformer-based and diffusion-based text-to-image systems.
Loading 2202.04053v3…