PaperlayerView exact revision on arXiv

Source-linked AI summary

Newer Is Not Fairer: Gender Stereotyping in Text-to-Image AI Across Model Generations

Shesh Narayan Gupta, Nik Bear Brown

arXiv:2609.18007v1cs.CVcs.AIcs.CYcs.LG

TL;DR

Evidence on occupational gender representation and fairness across T2I model generations has been limited, despite their use in professional and creative settings. This study benchmarks four Stable Diffusion generations against workforce demographics and finds persistent male-skewed outputs with no steady improvement across generations.

  • Problem

    Whether text-to-image gender bias improves across generations remains poorly understood, despite the practical importance of widely adopted Stable Diffusion systems.

    Source: §1 · p. 1

  • Method

    The study benchmarks occupational outputs across four Stable Diffusion generations and compares their gender representation with workforce demographics.

    Source: §7 · pp. 15–16

  • Results

    Across generations, outputs remained more male-skewed than workforces; 76.4% of 8,000 open-source images showed male subjects.

    Source: §7 · p. 15

  • Takeaways & Limitations

    Models underrepresent women by 20–46pp on average, and gender skew does not improve steadily across generations.

    Source: §7 · p. 16

    The 20-occupation sample is non-exhaustive and excludes gender-neutral and emerging roles.

    Source: §6 · p. 15

Abstract

from arXiv · show

Text-to-image generative models are widely used in professional and creative settings, yet how they represent gender across occupations -- and whether newer models are fairer -- remains poorly understood across multiple generations. We evaluate gender representation across 20 occupations, 5 prompt templates, and 4 Stable Diffusion model generations (SD 1.5, SD 2.1, SDXL, SD 3 Medium), generating 8,000 images with n = 100 per occupation-model cell (5 prompts x 20 images), and classifying all with DeepFace. Across the 8,000 open-source images, 76.4% show male subjects (95% CI [75.1%, 78.7%], p < 2.2 x 10^-16, Benjamini-Hochberg adjusted). More strikingly, 57.6% of images for historically female-coded occupations show male subjects (raw p = 3.43 x 10^-22, BH-adjusted p = 1.71 x 10^-21). All nine significant tests reported in this paper survive BH correction across 10 tests. When compared against U.S. Bureau of Labor Statistics workforce data, models underrepresent women by 20-46pp on average, with particularly large deviations for near gender-balanced occupations: scientist (48% female in BLS, 82-99% male in model outputs) and cleaner (46% female in BLS, 80-92% male in outputs). Model generations do not improve steadily: bias worsens from SD 1.5 to SDXL before partially recovering in SD 3 Medium. A preliminary comparison with GPT-image-1 on five occupations suggests lower bias than open-source models, though the practical effect is small (Cramer's V = 0.080) and the comparison is exploratory. No model achieves gender parity.

1 Introduction

Text-to-image models are widely used, but their occupational gender representations and fairness across generations remain important open questions. This paper benchmarks four Stable Diffusion generations and examines workforce alignment and prompt sensitivity.

1 Introduction

  • The study generates 8,000 images across 20 occupations, five prompt phrasings, and four generations, then classifies them with DeepFace and compares results with BLS demographics.All reported tests use Benjamini–Hochberg correction.Source: §1 · p. 2
  • The paper presents a controlled longitudinal comparison of occupational gender stereotyping across four Stable Diffusion generations.It reports confidence intervals and Benjamini–Hochberg multiple-comparisons correction throughout.Source: §1 · p. 2
  • Models underrepresent women by 20–46pp on average relative to BLS workforce demographics, with the largest deviations for near-balanced occupations.Source: §1 · p. 2
  • Bias follows a non-linear trajectory: SDXL shows higher gender skew than both SD 1.5 and SD 3 Medium.The paper describes the resulting mismatch between model progress and fairness for users of established versions as a deployment gap.Source: §1 · p. 2
  • Output gender composition varies substantially with prompt phrasing, and this sensitivity increases across model generations.Source: §1 · p. 2

2 Related Work

Prior work documents gender stereotyping across text-to-image systems and other AI modalities, while this paper connects occupational image outputs to workforce demographics and studies prompt phrasing across generations.

2.1 Gender Bias in Text-to-Image Models

  • Audits found occupational T2I prompts reflect historical stereotypes [1], DALL-Eval benchmarks social bias [2], and neutral prompts can yield more stereotyped results than explicitly gendered ones [3].Source: §2.1 · p. 2
  • A 150-occupation audit found pervasive male and white dominance [4], while Fair Diffusion steers outputs toward more equitable representation [5].This study extends the audit across four Stable Diffusion generations and compares outputs with BLS workforce data.Source: §2.1 · p. 2

2.2 Occupational Stereotyping and Representation

  • Prior work describes occupational gender segregation and unequal face-analysis accuracy for darker-skinned women; this study examines gender in T2I occupational outputs against demographic data.Source: §2.2 · p. 2

2.3 Prompt Phrasing and Model Output

  • Prompt phrasing can change T2I outputs, and this study finds sensitivity increases across four generations even as average bias partially decreases in newer models.Prior work catalogued prompt strategies [11] and identified stereotype perpetuation as an AI risk [12].Source: §2.3 · p. 3

3 Methodology

The methodology combines a controlled multi-model image-generation design with occupational BLS mappings, prompt variation, demographic classification, and corrected statistical tests.

3.1 Experimental Design

  • The experiment generates 8,000 open-source images across four models, 20 occupations, and five templates, with 100 images per occupation-model cell.A preliminary GPT-image-1 comparison adds 100 images across five spotlight occupations using one template; occupation-level confidence intervals are approximately ±9.8pp at p = 0.5, and model-level intervals are approximately ±1.8pp.Source: §3.1 · p. 3

3.2 Occupation Selection

  • The 20 occupations span professional domains and historical gender patterns, with BLS mappings including scientist at 48% female and cleaner at a weighted 46%.The cleaner benchmark combines janitors/building cleaners and maids/housekeeping cleaners despite their distinct BLS gender compositions; the scientist mapping aggregates subfields with differing gender compositions, so occupation-level amplification gaps require caution.Source: §3.2 · pp. 3–4

3.3 Prompt Templates

  • Five gender-neutral prompt templates measure phrasing sensitivity; the preliminary GPT-image-1 comparison uses only the first template.Source: §3.3 · p. 4

3.4 Models and Generation Settings

  • The four Stable Diffusion variants were run locally with shared settings, including 30 inference steps, guidance scale 7.5, 512×512 resolution, and 20 fixed seeds.GPT-image-1 was generated via the OpenAI API at 1024×1024 resolution.Source: §3.4 · p. 4

3.5 Demographic Classification and Validation

  • DeepFace assigns each image a dominant Man or Woman label, with 96% agreement against a manual review of 50 images.The authors note that the small single-reviewer validation lacks inter-rater reliability; apparent-gender classification is a proxy for perceived presentation, and accuracy varies across skin tones and image styles.Source: §3.5 · pp. 4–5

3.6 Metrics and Statistical Tests

  • The study defines stereotype, amplification-gap, and prompt-sensitivity metrics, and uses binomial and chi-square tests with Benjamini–Hochberg correction across 10 tests.All nine significant findings survive correction; the SD 1.5 versus SD 2.1 comparison is non-significant (raw p = 0.792).Source: §3.6 · p. 5

4 Results

Across four open-source Stable Diffusion generations, outputs show pronounced male representation and underrepresent women relative to BLS workforce data. Deviations are especially large in near-balanced occupations.

4.1 Overall Male Dominance

  • 76.4% of the 8,000 open-source images are classified as male (95% CI [75.1%, 78.7%], BH-adjusted p < 2.2 × 10−16).This pattern holds for every model and occupation category.Source: §4.1 · p. 5, Figure 1 · p. 6

4.2 Gender Distribution by Occupation

  • Programmer (97–100% male), construction worker (97–100%), firefighter (94–100%), engineer (96–97%), and scientist (82–99%) remain above 50% across models.Sixteen of 80 occupation-model cells fall within ±9.8pp of balance and are treated as indicative rather than definitive.Source: §4.2 · p. 6, Figure 2 · p. 6

4.3 Bias Amplification Against Workforce Reality

  • Mean overall amplification gaps were −27.1pp (SD 1.5), −27.6pp (SD 2.1), −31.2pp (SDXL), and −20.4pp (SD 3 Medium).For female-coded occupations, the corresponding gaps were −41.1pp, −39.8pp, −45.9pp, and −26.0pp; Figure 3 shows negative gaps nearly everywhere, with mechanic and pilot exceptions in some models.Source: §4.3 · p. 7, Figure 3 · p. 7
  • Scientist (48% female in BLS; 1–18% female in model images, 30–47pp gap) and cleaner (46%; 8–20%, 26–42pp) show large deviations.The cleaner reference is a weighted average across two BLS categories; the finding's direction also holds against the 29%-female janitors figure, with a 9–21pp gap.Source: §4.3 · p. 7, Figure 4 · p. 8

4.4 Cross-Directional Bias

Historically female-skewed occupations still show male-majority outputs overall, while SD 3 Medium improves on SDXL. Across model generations, bias peaks at SDXL before partially recovering.

4.4 Cross-Directional Bias

  • 57.6% of images for historically female-skewed occupations are classified as male, with six occupations showing male majorities across all four models.Figure 5 presents male percentages for all 10 occupations; 16 occupation-model cells fall within the ±9.8pp confidence interval around 50%.Source: §4.4 · p. 8, Figure 5 · p. 9
  • 45.2% of SD 3 Medium images are male for female-skewed occupations, versus 65.5% for SDXL (Cramér’s V = 0.199).SD 3 Medium shows female majorities in 7 of 10 historically female occupations, versus 2–4 for earlier models, but pilot rises to 98% male.Source: §4.4 · p. 9

4.5 Model Evolution and the Deployment Gap

  • Mean stereotype scores rise from 28.9 in SD 1.5 to 32.5 in SDXL, then fall to 29.1 in SD 3 Medium.SD 1.5 and SD 2.1 are statistically indistinguishable; the other five pairwise comparisons survive BH correction, with SDXL versus SD 3 Medium the largest effect (V = 0.126).Source: §4.5 · p. 9, Figure 6 · p. 10, Table 1 · p. 10
  • SDXL is the most gender-biased tested generation despite being widely adopted, while SD 3 Medium partially corrects its bias.The authors call this a deployment gap: a widely adopted version has higher bias than both its predecessor and successor.Source: §4.5 · p. 10

4.6 Prompt Sensitivity

Output gender composition varies with prompt phrasing, and that sensitivity increases across Stable Diffusion generations. White presentation dominates generated images, with limited racial diversity across the tested conditions.

4.6 Prompt Sensitivity

  • Mean prompt sensitivity rises from 9.6 in SD 1.5 to 13.3 in SD 3 Medium.SD 2.1 and SDXL both have mean sensitivity of 12.0; SD 3 Medium is less biased on average but more sensitive to phrasing.Source: §4.6 · p. 10, Figure 7 · p. 11
  • Female-skewed occupations show consistently higher prompt sensitivity than male-skewed occupations across all four models.Figure 7 measures sensitivity as the standard deviation of male percentage across five prompt templates; GPT-image-1 is excluded because it used one prompt.Source: §4.6 · p. 10, Figure 7 · p. 11

4.7 Racial Composition

  • 59.1% of all 8,000 open-source images show white presentation, the dominant category across models.The model-specific percentages range from 56.7% for SD 1.5 to 62.0% for SD 3 Medium.Source: §4.7 · p. 11, Figure 8 · p. 12
  • Asian presentation is the second most common category across all four models, while Black, Middle Eastern, Latino Hispanic, and Indian presentations are less frequent.Their overall shares are 9.7%, 8.9%, 6.0%, and 0.7%, respectively.Source: §4.7 · p. 11
  • White presentation averages 56.3% for male-skewed occupations and 62.0% for female-skewed occupations.Higher Asian classification rates in technical roles such as engineer, programmer, and scientist primarily drive the difference.Source: §4.7 · p. 11
  • White presentation increases slightly from SD 1.5 to SD 3 Medium, while the paper leaves racial amplification gaps against workforce data for future study.The authors say newer models may generate less racially diverse images for these prompts, but do not provide a full statistical analysis of racial bias.Source: §4.7 · p. 11

4.8 Preliminary Comparison: GPT-image-1

This exploratory comparison finds a modest male-share difference between GPT-image-1 and open-source models on five spotlight occupations. The small sample and single-prompt design limit interpretation.

4.8 Preliminary Comparison: GPT-image-1

  • 68.5% of GPT-image-1 images are male, versus 84.5% for open-source models on five spotlight occupations (Cramér’s V = 0.080).The difference is statistically significant (χ2 = 15.61, BH-adjusted p < 0.001), but the practical effect is small.Source: §4.8 · p. 12, Figure 9 · p. 13, Table 2 · p. 13
  • The comparison covers five occupations with one prompt and 100 GPT-image-1 images, versus 2,000 images per open-source model.Three GPT-image-1 occupation estimates have confidence intervals of approximately ±20pp and are indicative only.Source: §4.8 · pp. 12–13
  • GPT-image-1 produces 65% male images for cleaner, 70% for firefighter, and 35% for nurse, compared with 80–92%, 94–100%, and 40–42% for open-source models, respectively.Construction worker is 100% male across all models; programmer is 80% male for GPT-image-1 versus 97–100% for open-source models.Source: §4.8 · p. 12, Figure 9 · p. 13, Table 2 · p. 13

5 Discussion

The discussion emphasizes robust deviations from workforce demographics and a non-linear bias trajectory across generations. It also highlights prompt sensitivity, the exploratory limits of the GPT-image-1 comparison, and practical audit recommendations.

5.1 Large and Robust Deviations From Workforce Demographics

  • Scientist and cleaner amplification gaps exceed the confidence-interval width by three to five times, while 16 of 80 occupation-model cells remain indicative rather than definitive.The gaps are −30 to −47pp for scientist and −26 to −42pp for cleaner; SD 3 Medium’s teacher and social worker estimates are each 45% male and straddle 50%.Source: §5.1 · p. 13

5.2 The Deployment Gap

  • SDXL is significantly more biased than SD 1.5 and SD 2.1, while SD 3 Medium is significantly less biased than SDXL (V = 0.126).SD 1.5 and SD 2.1 are statistically indistinguishable (V = 0.004); users must switch versions to benefit from SD 3 Medium’s improvement.Source: §5.2 · p. 14

5.3 Cross-Directional Bias Is Systematic

  • 57.6% of images for historically female occupations are classified as male; SD 3 Medium’s improvements are selective, with pilot regressing.The passage describes teacher and social worker as improving while pilot worsens, suggesting targeted rather than systematic correction.Source: §5.3 · p. 14

5.4 Prompt Sensitivity Grows With Generation

  • Prompt sensitivity increases with model generation, and female-coded occupations show consistently higher sensitivity than male-coded ones.The discussion warns that evaluating one prompt per occupation misses much of the output variation and recommends testing multiple phrasings.Source: §5.4 · p. 14

5.5 GPT-image-1: A Preliminary Observation

  • GPT-image-1’s comparison has a small practical effect (Cramér’s V = 0.080), and three of its five occupation cells have confidence intervals of approximately ±20pp.The authors call for a follow-up using the open-source study’s multi-prompt, multi-occupation design.Source: §5.5 · p. 14

5.6 Practical Recommendations

  • The recommendations are to evaluate each model version independently, test multiple prompt phrasings, and compare outputs against workforce demographic data.The text recommends three to five prompt phrasings across relevant occupations and notes that BLS comparisons reveal deviations that 50/50 balance metrics do not capture.Source: §5.6 · p. 14

6 Limitations

The study’s estimates are limited by finite samples and classifier validation, binary apparent-gender labels, and choices of inference settings, occupational mappings, and scope.

Statistical power and confidence intervals

  • Occupation-level estimates use n = 100 images per cell, with 95% CIs of approximately ±9.8pp at p = 0.5; 16 of 80 cells are flagged as indicative.Model-level estimates use n = 2,000 and have CIs of approximately ±1.8pp; all nine significant findings survive BH correction across 10 tests.Source: §6 · pp. 14–15

Classifier validation

  • DeepFace was validated by one reviewer on 50 images with 96% agreement, but the sample is small and lacks inter-rater reliability.The authors consider gaps of 26pp or more robust to plausible classifier error, while occupation-level findings near 50% warrant caution.Source: §6 · p. 15

Binary gender classification

  • DeepFace labels apparent gender as Man or Woman, excluding non-binary, non-conforming, and androgynous presentations.Apparent gender is a proxy for perceived presentation, not self-identified gender.Source: §6 · p. 15

Resolution differences

  • GPT-image-1 images were generated at 1024×1024 pixels versus 512×512 for open-source models, so resolution-related classification differences complicate that comparison.The paper therefore treats the GPT-image-1 comparison as exploratory.Source: §6 · p. 15

Default inference settings and ecological validity

  • Default settings and no negative prompts measure out-of-the-box behavior, not the full distribution of real-world use.Negative prompts and modified guidance scales can substantially affect demographic outputs.Source: §6 · p. 15

BLS data specificity and occupational mapping

  • Three BLS mappings are ambiguous, affecting reference-value precision for cleaner, babysitter, and scientist; the figures reflect U.S. 2023 patterns.The paper says these ambiguities do not change the findings’ direction, but the BLS figures may not transfer to other cultural contexts.Source: §6 · p. 15

Occupational scope

  • The 20-occupation set is non-exhaustive and excludes gender-neutral and emerging roles.Source: §6 · p. 15

Model access and GPT-image-1 comparison

  • SD 3.5 and later variants were not evaluated; the GPT-image-1 comparison covers five occupations with one prompt and n = 20 per occupation.Three comparison cells have CIs of approximately ±20pp, and Cramér’s V = 0.080 is a preliminary observation only.Source: §6 · p. 15

7 Conclusion

Across four Stable Diffusion generations, occupational images consistently skew male relative to workforce demographics, and newer models are not reliably fairer. Bias worsens at SDXL before partially recovering in SD 3 Medium; the GPT-image-1 comparison is preliminary.

7 Conclusion

  • 76.4% of 8,000 open-source images show male subjects (95% CI [75.1%, 78.7%], p < 2.2 × 10^-16, BH-adjusted).For historically female occupations, 57.6% of images show male subjects (raw p = 3.43 × 10^-22, BH-adjusted p = 1.71 × 10^-21).Source: §7 · p. 15
  • Models underrepresent women by 20–46pp on average relative to BLS workforce demographics, with 26–47pp deviations for near-balanced occupations such as scientist and cleaner.Source: §7 · p. 16
  • SDXL shows higher gender skew than SD 1.5 and SD 3 Medium (V = 0.050 and 0.126, respectively), so bias does not improve steadily across generations.Source: §7 · p. 16
  • GPT-image-1 suggests lower bias on four of five spotlight occupations (V = 0.080, small effect), but the comparison warrants follow-up study.Source: §7 · p. 16

Ethics Statement

The study analyzes AI-generated occupational images, not images of real individuals, and uses binary apparent-gender labels. The authors share aggregate results and reproducibility materials while withholding raw images to reduce potential misuse.

Ethics Statement

  • The study involved no human subjects and collected, used, or stored no images of real individuals.All analyzed images were generated by AI models from occupational text prompts.Source: Ethics Statement · p. 16
  • DeepFace assigns synthetic faces apparent-gender labels of Man or Woman, which do not capture the full spectrum of gender identity and presentation.Source: Ethics Statement · p. 16
  • Raw generated images are not publicly released to avoid potential misuse; aggregate statistics and figures are shared instead.Source: Ethics Statement · p. 16
  • The authors judge the risk of bias documentation being misused as low because the patterns are already observable to users, and say its public benefit outweighs that risk.Source: Ethics Statement · p. 16
  • Generation configurations, classification scripts, and aggregate results are publicly available to support reproducibility and future audits.Source: Ethics Statement · p. 16

Related papers

Loading 2609.18007v1…