PaperlayerView exact revision on arXiv

Source-linked AI summary

Gender bias across LLMs is common and highly heterogeneous

Edoardo Bolzoni, Valerio Capraro

arXiv:2609.38036v2cs.CLcs.AIcs.CYcs.HC

TL;DR

Prior research left unclear how widespread gender bias is across LLMs, despite their growing role in everyday decisions. This study tests ten models across two paradigms and finds bias common but highly heterogeneous, underscoring that one model cannot represent the category.

  • Problem

    Prior studies used different paradigms, leaving the prevalence and model-to-model heterogeneity of gender bias unclear, even as LLMs enter consequential everyday decisions.

    Source: §1 · p. 1

  • Method

    The study tested ten models from nine vendors using stereotyped-phrase gender attribution and moral judgments in sacrificial dilemmas.

    Source: §1 · p. 2

  • Results

    Eight of ten models showed a significant gender-related asymmetry in at least one study, and the patterns varied across models.

    Source: §1 · p. 2, §4 · p. 8

  • Takeaways & Limitations

    A bias or its absence in one model cannot be assumed to characterize LLMs as a category, supporting ongoing multi-vendor monitoring.

    Source: §1 · p. 2, §4 · p. 9

    The audit used English-language prompts and nuclear-apocalypse framing, and an interruption means change in Claude Fable 5 between collection windows cannot be ruled out.

    Source: §3.1.5 · p. 5, §4 · p. 9

Abstract

from arXiv · show

Understanding gender biases in large language models (LLMs) is increasingly important as these systems become embedded in decision-support tools with real consequences. Prior research has focused only on a small set of models, leaving open the extent to which gender biases are common and heterogeneous across LLMs. We address this gap across ten models released between April 2025 and June 2026, spanning nine vendors, using two paradigms: gender attribution to stereotyped phrases (Study 1) and moral judgment of abuse or torture against a woman or a man to prevent a catastrophic outcome (Study 2). In Study 1, two of ten models attributed masculine-stereotyped phrases to female writers more often than the reverse, while three models showed the opposite pattern. In Study 2, several models converged on a male-disadvantaging asymmetry that was directionally consistent with a documented human tendency to protect female targets from harm, though the specific conditions under which this asymmetry emerged varied by model; three other models, by contrast, showed no variation across conditions. These results indicate that gender-related biases are common in LLMs. Their direction and magnitude, however, are highly heterogeneous, to the point that some models behave in diametrically opposite ways to others. Bias auditing should therefore be treated as an ongoing, multi-vendor process, rather than a one-time assessment.

1. Introduction

Prior work left unclear how widespread gender bias is across models, despite the stakes of LLMs entering everyday decisions. Across ten models and two paradigms, this study finds gender bias common but highly heterogeneous.

1. Introduction

  • Because LLMs increasingly touch everyday decisions at scale, their biases can translate into real, systemic disadvantage (Mökander et al., 2024).Source: §1 · p. 1
  • Prior studies reported female-favoring judgments across models (Pikuliak, 2025), but differing paradigms left prevalence and model variation unresolved.Earlier work also examined phrase attribution and moral judgments, including violence against men versus women (Fulgu and Capraro, 2024).Source: §1 · p. 1
  • The study examined ten models from nine vendors using stereotyped-phrase gender attribution and moral judgment in sacrificial dilemmas.The models were released between April 2025 and June 2026.Source: §1 · p. 2
  • Gender bias was common and heterogeneous across the ten models, varying in favored gender, strength, and whether an asymmetry appeared at all.The broad convergence suggested by prior work did not hold consistently across this set of models.Source: §1 · p. 2

2.1. Methods

The methods define gender as a socially attributed category, test ten models on phrase-attribution stimuli, and compare responses using an inclusivity index and statistical tests.

2.1.1. A note on sex and gender terminology

  • Following SAGER guidelines, the paper uses gender for socially attributed categories inferred from cues, not biological sex.Study 1 uses girl/boy terminology to match its child-aged writer stimuli.Source: §2.1.1 · p. 2

2.1.2. Models tested

  • Ten models were listed in release-date order and tested with default settings, usually through consumer interfaces; Mistral Small 4 was accessed via API.Claude Fable 5 data were collected in two sessions around a temporary suspension; testing ran 15–22 May 2026 except for that model, tested on 12 June and 1 July.Source: §2.1.2 · p. 2, Table 1 · p. 3

2.1.3. Stimuli

  • Study 1 reused 20 phrase pairs: 17 stereotyped contrasts and three explicit-gender controls, written in informal, childlike style.Source: §2.1.3 · p. 2

2.1.4. Procedure

  • Each model received the same writer-inference prompt for each phrase ten times in memory-isolated temporary chats.The prompt asked for a potential writer’s name, age, and gender.Source: §2.1.4 · p. 3

2.1.5. Coding of responses

  • Responses were coded girl=1, boy=0, and non-binary=0.5; co-equal attributions were retained separately, and transgender writers were coded by stated identity.Source: §2.1.5 · p. 3

2.1.6. Inclusivity index and statistical analysis

  • Phrase inclusivity is the mean absolute distance from the stereotypical response; feminine and masculine phrase means were compared using t-tests and rank-sum tests.The analysis also repeated t-tests after excluding the control phrases.Source: §2.1.6 · p. 3

2.2. Results

Study 1 found significant gender-related asymmetries in opposite directions across models, while five models showed no significant difference. The Mistral Small 4 effect survived exclusion of control phrases, unlike Claude Sonnet 4.6’s.

2.2. Results

  • Two models showed I_M > I_F; Mistral Small 4 remained significant after control phrases were excluded, but Claude Sonnet 4.6 did not.Claude Sonnet 4.6: t(32) = −1.87, p = .070; Mistral Small 4: t(32) = −2.88, p = .007.Source: §2.2 · pp. 3–4
  • Three models showed the opposite significant asymmetry, I_F > I_M: Gemini 3.1 Pro, DeepSeek V4-Flash, and Qwen3.6.For Qwen3.6, I_M = 0 with zero variance across masculine-stereotyped phrases.Source: §2.2 · p. 4
  • The remaining five models showed no significant difference between I_F and I_M.They were Llama 4 Scout, Grok 4.1 Fast, GPT-5.5, Microsoft Copilot, and Claude Fable 5.Source: §2.2 · p. 4
  • The companion Wilcoxon-Mann-Whitney rank-sum test yielded qualitatively equivalent conclusions for every model.Figure 1 connects each model’s feminine-phrase index (green) to its masculine-phrase index (orange); line length and position indicate asymmetry, and asterisks mark significant differences.Source: §2.2 · p. 4

3. Study 2: Moral dilemmas

Study 2 tested moral judgments across abuse and torture scenarios involving a woman or a man, using the same ten models as Study 1. It repeated each condition, accounted for refusals and an interruption to Claude Fable 5 data collection, and used paired-condition statistical comparisons.

3.1.1. Design

  • The 2 × 2 design crossed victim gender (woman or man) with abuse or torture in a nuclear-apocalypse dilemma, distinguishing general protection from an asymmetry specific to gender-politically salient abuse.Abuse is closely tied to real-world discourse on gender-based violence; torture carries no comparable gendered connotation.Source: §3.1.1 · p. 4

3.1.2. Models tested

  • The same ten models as Study 1 were tested under the same access conditions, mostly between 22–29 May 2026.Claude Fable 5 was tested on 12 June and 1 July 2026; Mistral Small 4 was accessed via API, while other models used default settings and standard web/app interfaces.Source: §3.1.2 · p. 4

3.1.3. Procedure

  • Each model received four versions of the question, with each dilemma presented 50 times using incognito or temporary-chat sessions.The response scale ran from 1 (strongly disagree) to 7 (strongly agree).Source: §3.1.3 · p. 5

3.1.4. Missing data and refusals

  • Refusals were treated as missing cells; only Gemini 3.1 Pro and Claude Fable 5 had condition samples below 50.Gemini had n = 45 for abuse-woman and n = 47 for torture-woman; Claude Fable 5 had n = 34 for abuse-woman.Source: §3.1.4 · p. 5

3.1.5. Claude Fable 5: export-control interruption

  • An export-control suspension interrupted Claude Fable 5’s torture-condition collection, and the authors could not rule out model changes between sessions.After comparing pre-suspension and post-restoration responses, they resumed collection; torture-woman responses were identical across the 50 iterations, and torture-man responses did not differ significantly between halves.Source: §3.1.5 · pp. 5–6

3.1.6. Statistical analysis

  • For each model, four pairwise comparisons used independent-samples t-tests and companion Wilcoxon-Mann-Whitney tests; zero-variance cases were described rather than assigned undefined test statistics.Source: §3.1.6 · p. 6

3.2. Results

Study 2 found sharply different moral judgments across models, including models with no response variation and others with gender-related gaps. Among models showing gaps, their size and the conditions in which they appeared varied.

3.2. Results

  • Table 3 reports mean agreement by model and condition, and Figure 2 presents the same means as a heatmap across all ten models.The response scale ranges from 1 (strongly disagree) to 7 (strongly agree).Source: §3.2 · p. 6, Table 3 · p. 7, Figure 2 · p. 7
  • Llama 4 Scout and Microsoft Copilot rated every condition completely unacceptable (M=1.00), while DeepSeek V4-Flash rated every condition completely acceptable (M=7.00).All three models had no within-model variance, so none of the four comparisons had a defined test statistic.Source: §3.2 · p. 6
  • Claude Sonnet 4.6 and Gemini 3.1 Pro both showed gender gaps, but with different patterns across abuse and torture conditions.Sonnet rated abuse-woman at M=1.48 while its other three conditions were M=7.00; Gemini showed gaps for both violence types, with a larger gap for abuse.Source: §3.2 · p. 6
  • Grok 4.1 Fast, Mistral Small 4, GPT-5.5, Qwen3.6, and Claude Fable 5 showed mixed gender-gap patterns and widely varying condition means.GPT-5.5 had gender gaps for both abuse and torture; Qwen3.6 had a significant torture gap and an abuse difference confirmed by the rank-sum test despite zero variance in both abuse conditions.Source: §3.2 · p. 6
  • Within woman-victim conditions, torture was rated significantly more acceptable than abuse (t(82)=3.56, p<.001), with no such difference within man-victim conditions (t(98)=0.73, p=.466).Source: §3.2 · p. 7

4. Discussion

The discussion emphasizes that gender-related biases are common but vary across models, tasks, and conditions. It connects some moral-judgment patterns to prior research while noting limits on what this audit can establish.

4. Discussion

  • Eight of ten models showed a significant gender-related asymmetry in at least one study, while Study 1’s direction varied across models.Two models showed one direction of attribution asymmetry, three showed the opposite, and five showed no significant effect.Source: §4 · p. 8
  • Six models judged abuse of a man more acceptable than abuse of a woman, aligning directionally with reported human moral chivalry toward female targets.Across the ten models, the mean gender gap was larger for abuse (2.13 points) than for torture (0.78 points). The authors note that their data cannot confirm whether models amplify a human intuition in training data.Source: §4 · pp. 8–9
  • Models differed in baseline judgments as well as gender-gap patterns, with some giving opposite judgments on the dilemma and others showing either response instability or consistent uncertainty.Llama 4 Scout and Microsoft Copilot rated all conditions M=1.00, while DeepSeek V4-Flash rated all M=7.00; Mistral Small 4’s midpoint average reflected widely spread responses, whereas Claude Fable 5’s reflected tightly clustered responses.Source: §4 · p. 8
  • Refusals by Gemini 3.1 Pro and Claude Fable 5 occurred only in woman-victim conditions, and were more frequent for abuse-woman than torture-woman prompts.Gemini refused 5 of 50 abuse-woman prompts and 3 of 50 torture-woman prompts; Claude Fable 5 refused 16 abuse-woman prompts and none for torture-woman.Source: §4 · p. 8
  • Gender is one of several factors associated with LLM moral judgments, and reported gender-related patterns also occur in attribution, policy generation, and stereotype-related outputs.The discussion also reviews research reporting political-orientation asymmetries in conversational LLMs.Source: §4 · p. 9
  • Whether users adopt models’ gender-related asymmetries remains an open research question, so the discussion calls for experiments comparing judgments with an unaided baseline.The cited evidence on AI advice includes both cases of user influence and findings that antisocial advice did not reduce generosity.Source: §4 · p. 9
  • The audit tested English prompts with nuclear-apocalypse framing and cannot establish whether the models behave similarly under other languages or framings.It also cannot explain why particular models diverge because their training data, fine-tuning procedures, and safety interventions are proprietary and inaccessible to external researchers.Source: §4 · p. 9

A. Full statistics for Study 1

The appendix reports Study 1 results using a companion rank-sum test alongside the inclusivity indices.

A. Full statistics for Study 1

  • Table A.1 reports the Study 1 inclusivity indices with a Wilcoxon-Mann-Whitney rank-sum test, and its conclusions are qualitatively identical to those from the t-test for every model.Source: A. Full statistics for Study 1 · p. 12

B. Full statistics for Study 2

The appendix provides both t-test and rank-sum statistics for Study 2’s model comparisons.

B. Full statistics for Study 2

  • Tables B.1 and B.2 report all four pairwise comparisons per model using t-tests and Wilcoxon-Mann-Whitney rank-sum tests, with qualitatively identical conclusions in every case.Source: B. Full statistics for Study 2 · p. 12

C. Distribution of Study 2 responses, by model

The appendix pairs a model-level classification of response-distribution shapes with full condition distributions and per-model pairwise-comparison tables.

C. Distribution of Study 2 responses, by model

CRediT authorship contribution statement

The statement assigns distinct contribution portfolios to Edoardo Bolzoni and Valerio Capraro.

CRediT authorship contribution statement

Loading 2609.38036v2…