PaperlayerView exact revision on arXiv

Source-linked AI summary

Stable Bias: Analyzing Societal Representations in Diffusion Models

Alexandra Sasha Luccioni, Christopher Akiki, Margaret Mitchell, Yacine Jernite

arXiv:2303.11408v2cs.CY

TL;DR

Social-bias auditing in Text-to-Image systems is difficult because generated humans lack inherent gender or ethnicity while their depictions still invite social interpretation. The paper varies identity markers in prompts, compares outputs across professions, and combines text, image-space, and interactive analyses; across systems, it reports under-representation of marginalized identities alongside correlations with U.S. labor demographics.

  • Problem

    TTI systems create synthetic human depictions without inherent gender or ethnicity, making conventional diversity evaluation difficult while bias auditing is increasingly necessary as deployment grows.

  • Method

    The method varies gender and ethnicity markers in prompts, compares generated images across 146 professions, and combines text-based, clustering-based, and interactive analyses.

  • Results

    All three evaluated systems correlate with U.S. labor demographics while under-representing marginalized identities to different extents.

  • Takeaways & Limitations

    The released datasets, code, and interactive platforms support targeted auditing and exploration of additional TTI systems and topics.

  • Takeaways & Limitations

    The analysis is limited by biases in captioning and VQA models, incomplete control over Dall·E 2’s API behavior, and a Western-oriented focus on identity axes.

Abstract

from arXiv · show

As machine learning-enabled Text-to-Image (TTI) systems are becoming increasingly prevalent and seeing growing adoption as commercial services, characterizing the social biases they exhibit is a necessary first step to lowering their risk of discriminatory outcomes. This evaluation, however, is made more difficult by the synthetic nature of these systems' outputs: common definitions of diversity are grounded in social categories of people living in the world, whereas the artificial depictions of fictive humans created by these systems have no inherent gender or ethnicity. To address this need, we propose a new method for exploring the social biases in TTI systems. Our approach relies on characterizing the variation in generated images triggered by enumerating gender and ethnicity markers in the prompts, and comparing it to the variation engendered by spanning different professions. This allows us to (1) identify specific bias trends, (2) provide targeted scores to directly compare models in terms of diversity and representation, and (3) jointly model interdependent social variables to support a multidimensional analysis. We leverage this method to analyze images generated by 3 popular TTI systems (Dall-E 2, Stable Diffusion v 1.4 and 2) and find that while all of their outputs show correlations with US labor demographics, they also consistently under-represent marginalized identities to different extents. We also release the datasets and low-code interactive bias exploration platforms developed for this work, as well as the necessary tools to similarly evaluate additional TTI systems.

1 Introduction

Diffusion-based Text-to-Image systems have rapidly gained popularity, but their assembled components make downstream behavior and bias difficult to disentangle. This paper introduces tools for auditing social biases by comparing targeted prompt outputs across professions, gender, and ethnicity.

  • 1 Introduction: Diffusion-based approaches have quickly become popular techniques for prompted image generation.Examples include Stable Diffusion, Make-a-Scene, Imagen, and Dall·E 2.
  • 1 Introduction: TTI systems combine text and image processing components, including safety filters and embeddings, that all contribute to final image generation.Their modular structure makes it difficult to isolate which component drives downstream behavior.
  • 1 Introduction: Widespread deployment may amplify and perpetuate existing societal inequities, while TTI biases remain sparsely documented and difficult to audit.Biases are often described only broadly in model cards and papers introducing new systems.
  • 1 Introduction: The paper audits Stable Diffusion v.1.4, Stable Diffusion v.2, and Dall·E 2 using professions alongside explicit gender and ethnicity markers.It also releases datasets and tools for exploring additional systems and topics.

2 Background

Prior work shows that machine-learning bias includes performance disparities, stereotypes, and representational distortions, while multimodal systems introduce underdeveloped questions about how biases across modalities interact. Existing studies document societal bias in downstream multimodal outputs and identify multiple sources of bias in TTI systems.

  • 2 Background: Bias in machine learning encompasses disparate performance, propagated social stereotypes, and misrepresentations encoded in datasets and models.These effects can also be misunderstood by users.
  • 2 Background: Bias-evaluation methods for multimodal models have lagged behind their rapid growth, and it remains unclear whether biases across modalities amplify or compound one another.This unresolved interaction is the gap the paper seeks to address.
  • 2 Background: Prior multimodal studies find race, gender, and appearance biases in downstream tasks such as image captioning and image search.Reported examples include gender stereotypes in captions and lower-quality captions for people with darker skin.
  • 2 Background: Research on TTI systems has examined safety filters, semantic spaces, stereotypical demographic representations, memorization, and biases in CLIP-guided diffusion.These strands address both system behavior and components used during generation.
  • 2 Background: Potential TTI bias sources include harmful, pornographic, mislabeled, and corrupted web-scraped training data, followed by filtering based on other classifiers.The filtering criteria themselves may introduce additional effects.

3 Methodology: Auditing Social Biases in TTI Systems

The methodology evaluates social bias without assigning inherent gender or ethnicity labels to synthetic humans. It varies identity markers in prompts, compares outputs across 146 professions, and combines text-based, image-space, and interactive analyses.

  • 3 Methodology: Auditing Social Biases in TTI Systems: The method treats perceived gender and ethnicity markers as multidimensional and socially interpreted rather than inherent, discrete properties of generated humans.This motivates avoiding a priori definitions based solely on appearance.
  • 3 Methodology: Auditing Social Biases in TTI Systems: The evaluation varies gender and ethnicity references in prompts, then generates profession prompts to audit social diversity and representativity.The profession set contains 146 occupations.
  • 3 Methodology: Auditing Social Biases in TTI Systems: Two analyses use image captions and VQA text representations alongside clustering-based comparisons in image embedding space.The clustering analysis uses the Identities dataset to identify embedding-space regions associated with perceived social features.
  • 3 Methodology: Auditing Social Biases in TTI Systems: Prompts use the pattern “Photo portrait of a [X] [Y ]”, with identity markers and profession names drawn from U.S. Bureau of Labor Statistics occupations.The BLS also supplies demographic and salary information for professions.
  • 3 Methodology: Auditing Social Biases in TTI Systems: The study evaluates Stable Diffusion v.1.4, Stable Diffusion v.2, and Dall·E 2 using Identities and Professions datasets.The Identities dataset contains 68 prompts and 2040 images; the Professions dataset covers 146 professions for each system.
  • 3 Methodology: Auditing Social Biases in TTI Systems: Interactive tools support ad-hoc exploration of visual characteristics that automated analyses may not capture.Their primary goal is qualitative story-based examination rather than quantitative insight.

4 Results

The results use text markers and image-space clusters to examine gender and ethnicity representation across professions and TTI systems. They find systematic differences from labor demographics, including under-representation of woman-associated regions and model-specific variation.

  • 4.1 Gender Bias Analysis through Text Markers: 97.66% of captions contained gender-marked terms, compared with 45.56% of VQA appearance predictions.The difference is consistent with captions being full sentences and VQA outputs mostly being single-word predictions.
  • 4.1 Gender Bias Analysis through Text Markers: Dall·E 2 showed the largest discrepancy from BLS gender proportions, while Stable Diffusion v.1.4 showed the smallest, approximately 9% for captions and VQA.Dall·E 2 captions mentioned women 27% less and VQA 25% less than the BLS figures.
  • 4.2.1 Characterizing Identity Regions in the Image Space: The cluster method groups images from 68 gender–ethnicity phrase combinations into 24 regions, avoiding direct identity-label assignment while quantifying visual-feature distributions.The regions summarize variation correlated with identity characteristics used in prompts.
  • 4.2.1 Characterizing Identity Regions in the Image Space: Region 4 accounted for over 40% of Profession images and tended to contain images prompted with White men, while four woman-associated regions together accounted for 25.5%.The regions do not themselves correspond to specific genders or ethnicities; their interpretations come from the prominent phrases in assigned prompts.
  • 4.2.1 Characterizing Identity Regions in the Image Space: For laboratory technician images, region 4 was twice as prominent in Dall·E 2 outputs as in Stable Diffusion outputs.The example analysis used 630 images, 210 per system, and visualized generations assigned to selected regions.
  • 4.2.2 Gender and Ethnicity Representation across Systems: The quintile analysis groups 146 professions by BLS gender and ethnicity distributions and compares grouped-region image proportions with BLS averages.This comparison assesses whether systems under- or over-represent groups and whether they attenuate, reproduce, or exacerbate social biases.
  • 4.2.2 Gender and Ethnicity Representation across Systems: All systems under-represented regions associated with woman and Black phrases; the effect was least pronounced for Stable Diffusion v1.4 and most pronounced for Dall·E 2.Stable Diffusion v1.4 matched the US distribution for the least diverse professions but under-represented woman-associated regions in more balanced professions.

5 Limitations and Future Work

The authors identify limitations involving biased auxiliary models, limited control over Dall·E 2, constrained social attributes, and Western-oriented analysis. They call for broader, more nuanced studies and interactive tools supporting wider stakeholder participation.

  • Limitations: Captioning and VQA models introduce biases that the authors cannot fully control, so they compensate by comparing multiple models and using BoVW.The authors use multiple models and less symbolic representations to mitigate these effects.
  • Limitations: Dall·E 2’s API limits control over prompt injection, filtering, and model consistency across days, requiring an assumption that outputs correspond to input prompts.This constrains direct comparative certainty for Dall·E 2.
  • Limitations: The analysis covers gender and ethnicity, although these attributes are fluid, multidimensional, and non-discretizable.The authors also note that their identities and terminology do not capture every relevant social dimension.
  • Limitations: The authors acknowledge a Western bias in the discrimination axes studied and stress the need to extend analysis across cultures and contexts.They also note limited first-hand experience with many identity characteristics discussed.
  • Future Work: Future work should examine additional bias dimensions, account for interconnected attributes, and develop interactive tools for less technically experienced stakeholders.The authors specifically mention age, visual markers of religion, and community-oriented exploration.

Supplementary Materials

The supplementary materials define the prompt configurations used to evaluate gender, ethnicity, and profession attributes.

  • Prompt Attributes: Gender prompts include three specified gender values and one unspecified option, while unspecified ethnicity and adjective terms are omitted.All profession prompts specify a profession value.

B.0.1 Characterizing the “Identities” clusters.

The analysis embeds and clusters identity-prompt images to characterize visual variation across gender and ethnicity terms. BLIP VQA embeddings best separate these attributes, while clusters often represent intersections rather than single attributes.

  • Clustering Method: Average cluster entropy is used to compare embedding methods; lower entropy means prompted social attributes are more tightly clustered.The analysis embeds and clusters images for each gender–ethnicity prompt before calculating entropy.
  • Embedding Comparison: BLIP VQA question embeddings have the lowest entropy and are therefore the most useful representation of visual depictions of social attributes in this setting.The comparison uses a 99% confidence-interval bootstrap estimator with values reported as ±0.02.
  • Cluster Structure: Most clusters correspond to specific gender–ethnicity intersections, while some emphasize only gender or ethnicity.The full disaggregated distributions are provided in appendix tables.
  • Visual Variation: Visual embeddings allow ethnicity terms to be represented through varied visual features rather than single labels.The Latinx-focused clusters contain different skin tones and hair types, illustrating variation within the ethnicity group.
  • Visual Variation: The Latinx-focused clusters both most frequently contain “Latinx” ethnicity and “woman” gender prompt terms, despite differing in hair types and skin tones.These clusters demonstrate that visual depictions associated with one identity grouping are not uniform.

B.0.2 Measuring the aggregated social diversity of TTI system outputs.

The paper measures output diversity as entropy across visual-feature clusters and compares identity-prompt images with profession-prompt images. Identity outputs are similarly diverse across systems, whereas unspecified-attribute profession outputs show stronger differences and under-representation of some identities.

  • Diversity Measure: Entropy measures diversity across embedding regions, increasing from all examples in one region to equal probability across regions.The measure quantifies how evenly generated examples occupy the identified visual-feature clusters.
  • Identity Outputs: Identity-dataset entropy is similar across systems, with values mostly within corresponding confidence intervals.This indicates comparable ranges of visual features when gender and ethnicity attributes are explicitly prompted.
  • Profession Outputs: Dall·E 2 ranks last in profession-dataset diversity, while Stable Diffusion v2 and v1.4 occupy higher positions, with v2 between the two.The systems differ more when social attributes are unspecified and prompts focus on professions.
  • Profession Outputs: Dall·E 2, Stable Diffusion v2, and Stable Diffusion v1.4 cover progressively larger portions of the common embedding space for target-attribute images.The same ordering appears when all image embeddings are projected into a shared two-dimensional space.
  • Identity Representation: Dall·E 2 under-represents clusters associated with “Black” and “African American” prompts while slightly over-representing a cluster associated with “non-binary” prompts.The characterization links cluster frequencies in profession images to identity terms in the corresponding identity-image prompts.

B.0.3 Disaggregated analysis by profession

Profession-level diversity varies across TTI systems and can align with workforce gender balance along different identity axes. Stable Diffusion v1.4 and Dall·E 2 also exhibit distinct stereotype patterns, while ethnicity markers can separate generated images visually.

  • Profession-level patterns: Profession diversity aligns with US workforce gender balance, but the dominant variation axis differs by profession.The variation is primarily gender for “singer,” ethnicity for “taxi driver” and “maid,” and their intersection for “social worker.”
  • Embedding-space patterns: Prompts containing Native American identity words form a distinct region in 2D projections of BLIP VQA and CLIP embeddings.The separation appears in projections for both “ethnicity” and “appearance” questions.
  • Embedding-space patterns: Profession images show distribution differences across the shared 2D embedding space, unlike the broadly even distribution of identity images across clusters.The projection uses one common space fitted to aggregated identity embeddings.
  • Embedding-space patterns: Clusters sharing “Latin-o-x” or “Hispanic” as the most represented prompt word still show substantial appearance variation.This variation occurs even among clusters whose prompts contain the same gender words.

C Cluster Composition

The 12-cluster analysis captures social variation through gender, ethnicity, and their intersections. Some clusters emphasize one dimension, while many combine both attributes.

  • Cluster composition: Some clusters primarily encode gender, while others emphasize ethnicity or combine gender and ethnicity.Examples include a non-binary cluster, a Native American cluster, and clusters combining racial categories with mostly women or men.
  • Intersectional clusters: Clusters 2 and 4 combine “White/Hispanic/Unspecified” identities with mostly women and mostly men, respectively.Clusters 6 and 8 similarly combine “Black/Multiracial” identities with mostly women and mostly men.

D Cluster Variation by Profession and Adjective

The paper organizes profession and adjective outputs by cluster assignments and diversity, comparing model results with BLS gender statistics. The supplied table captions identify separate cross-model, Stable Diffusion v1.4, and Dall·E 2 analyses.

  • Adjective variation: Table 11 reports partial cluster assignments and diversity for adjective-conditioned images across all models.Adjectives are ordered by the proportion assigned to clusters whose top gender word is “Man.”
  • Adjective variation: Table 12 compares adjective-conditioned cluster assignments and diversity between Stable Diffusion v1.4 and Dall·E 2.The table also reports the gender coding associated with the assigned clusters.
  • Profession variation: Tables 14–16 rank professions by diversity using entropy in the 12-cluster setting and provide BLS gender ratios for comparison.The rankings include all models together, Stable Diffusion v1.4 separately, and Dall·E 2 separately.

F Top Caption and VQA Predictions per Model and Profession

Table 18 presents BLIP VQA and ViT-GPT2 image-captioning results for professions across models. The supplied profession entries identify example occupations included in the broader evaluation.

  • Model predictions: Table 18 reports VQA and image-captioning results for all professions and models.The methods are BLIP VQA and image captioning with ViT-GPT2.
  • Profession coverage: The profession set includes computer support specialist, customer service representative, medical records specialist, and public relations specialist.These occupations are listed as individual entries associated with the evaluation outputs.
Loading 2303.11408v2…