Causally Testing Gender Bias in LLMs: A Case Study on Occupational Bias

1 Introduction

• The paper proposes a causal formulation and five benchmark desiderata, addressing reported shortcomings in existing generative-model bias benchmarks.The criteria include independent prompt and stereotype formation, objective stereotype labeling, a small prediction space, coverage of explicit and implicit bias, and inclusion of non-binary genders.Source: §1 · pp. 1–2
• OCCUGENDER measures occupational gender bias by predicting gender from occupations, using gender-dominated U.S. jobs and assessing explicit and implicit bias across male, female, and non-binary predictions.The benchmark forms jobs independently of prompt templates and uses an objective labeling pipeline.Source: §1 · p. 2
• Ten tested open-source LLMs show strong stereotypical associations between gender and stereotypically gendered jobs.The models include Llama-2-7B, Llama-3-8B, Mistral-7B, Gemma-7B, Gemma-2-9B, and their instruction-tuned versions.Source: §1 · p. 2
• OCCUGENDER does not define ideal model behavior, such as equal predictions across genders, allowing the framework to adapt to different contexts.The paper also extends its formulation and desiderata to gender bias in education as an example of generalizability.Source: §1 · p. 2

2 A Causal Formulation of Bias Measurement

• The paper introduces a causal-inference formulation for measuring bias, targeting the effect of stereotypes on gender predictions.The setup represents prompts with a stereotype and a template, with a potential common factor affecting both.Source: §2 · p. 2, §2.1 · pp. 2–3

2.2 Introducing do-Interventions

• A prompt designer's intention can confound stereotype and template choices, so gender predictions may reflect template cues rather than the stereotype.For example, a template containing “she” can itself lead the model toward a female prediction.Source: §2.2 · p. 3
• A do-intervention sets the stereotype while leaving the other variables to follow their original distributions, removing the spurious path through the confounder.The paper illustrates this with an interventional probability for “herself” after setting stereotypes to “confident” and “afraid.”Source: §2.2 · p. 3

2.3 Causal Effect Estimation

• The conditional probability of a gender given a stereotype captures both causal and spurious associations, and estimates the interventional probability correctly only when stereotype and gender are unconfounded.With confounding, that conditional probability provides little insight into whether the model associates the stereotype with females.Source: §2.3 · p. 3

OCCUGENDER

• OCCUGENDER forms stereotypes and templates independently, breaking the spurious path and making the conditional probability a valid estimate of the interventional probability.This independence is the benchmark's condition for interpreting measured associations as causal effects.Source: OCCUGENDER · p. 3

2.5 Mathematical Comparison

• When templates and stereotypes are independent, the template probability conditional on a stereotype reduces to the template probability; dependence prevents this simplification.The paper uses this independence to simplify its expression for the interventional probability.Source: §2.5 · p. 3
• Because iterating over every possible template is infeasible, OCCUGENDER approximates the causal effect using a wide variety of templates generated by ChatGPT-4o.For an occupation such as “firefighter,” the procedure replaces [Job] in the templates and averages the probability of predicting a gender over the resulting prompts.Source: §2.5 · p. 3

3 OCCUGENDER Benchmark

• OCCUGENDER pairs every stereotype with every independently selected template to address template confounding in measuring occupational gender associations.The benchmark is organized around five desiderata for bias measurement.Source: §3 · pp. 3–4, Figure 1 · p. 4

3.1 Objective Stereotype Labeling

• Using 2021 U.S. Bureau of Labor Statistics data, the benchmark selects 20 jobs each from occupations with the highest female and male worker rates.Source: §3.1 · p. 4

3.2 Predicting Genders Given Occupations

• Prompts ask a model to generate a pronoun for a person in a given job, and OCCUGENDER measures male, female, and diverse prediction probabilities.“Diverse” includes non-binary predictions and cases where gender is unknown.Source: §3.2 · pp. 4–5

3.3 Comprehensive Template Set to Evaluate Both Explicit and Implicit Biases

• OCCUGENDER assesses both explicit and implicit bias using 50 prompts distributed evenly across the two types.Table 2 shows task-prompt examples and corresponding gender expressions.Source: §3.3 · p. 5, A benchmark · p. 5, Table 2 · p. 5

A benchmark

• The benchmark includes non-binary gender as a measurement target, alongside male and female predictions.Source: A benchmark · p. 5

Models

• Experiments cover Llama, Mistral, and Gemma base models and their instruction-tuned versions, enabling comparisons within model families.The models were selected as open-source and resource-friendly.Source: §4 · p. 5

Query Setup

• The researchers query gender-category probabilities and average predictions across female- and male-dominated jobs.The collected jobs average 10.8% male and 89.2% female workers for female-dominated jobs, and 94.4% male and 5.6% female workers for male-dominated jobs.Source: §4 · pp. 5–6

• On StereoSet, paraphrasing original expressions can change association predictions, indicating noise from verbalization.Source: Figure 2 · p. 6

5.2 Instruction-Tuning Amplifies Biases

• Instruction-tuned models tend to predict female more often for female-dominated jobs than their base versions, but this trend is less apparent for male-dominated jobs.Gemma-2-9B-Instruct is an exception for male-dominated jobs, with lower male predictions and substantially higher diverse predictions than Gemma-2-9B.Source: §5.2 · p. 6

5.3 Base Models Only Recognize Binary Genders

• Base models’ diverse predictions are close to zero for explicit prompts and below 10% for implicit prompts; instruction-tuned models tend to predict diverse genders more often.The authors describe this as a tendency toward binary gender associations in base models and limited increased awareness of non-binary identities after instruction tuning.Source: §5.3 · p. 6

5.1 Asymmetry in Bias Across Occupations

• 73.5%–100% explicit and 72.7%–94.1% implicit male predictions for male-dominated jobs, versus 54.1%–86.9% explicit and 55.3%–87.2% implicit female predictions for female-dominated jobs.The reported averages cover more than 20 female-dominated and 20 male-dominated jobs.Source: §5.1 · p. 6, §5 · p. 6, Table 3 · p. 7

5.4 Importance of Template Diversity

• Active-voice templates were associated with male predictions and could override or amplify stereotype effects, motivating diverse templates to average individual-template effects.Source: §5.4 · p. 7

5.5 Level of Bias within Model Families

• Newer models did not necessarily reduce bias: Llama-3-8B predicted males on 84.4% of explicit male-dominated-job prompts versus 73.5% for Llama-2-7B.Gemma-2-9B predicted males on 98.9% of such prompts versus 91.2% for Gemma-7B.Source: §5.5 · p. 7

6.1 Bias Mitigation

• The preliminary experiments inserted six similarly intended debiasing instructions into ten templates, while prompting is presented as a zero-shot alternative to computationally expensive fine-tuning.The templates covered explicit and implicit bias prompts.Source: §6.1 · pp. 7–8
• Debiasing instructions had varying effects; instructions 3 and 6 made Gemma-2-9B more gender-neutral on implicit-bias prompts, while the male/female prediction gap remained large for male-dominated jobs.The authors describe the experiments as preliminary and identify whether some stereotypes are harder to mitigate as a research direction.Source: §6.1 · p. 8

6.2 Generalization to Other Biases or Cultural Context

• The framework can extend to other social biases when stereotypes and demographics are objective and observable, including fields of study and cultural contexts with corresponding statistics.For fields of study, the proposed estimation target is Field →Gender, with prompts varied across template and gender combinations.Source: §6.2 · p. 8

6.3 Ideal Behavior of An Unbiased Model

• OCCUGENDER does not prescribe one ideal unbiased behavior: equal male and female predictions fit a binary-gender objective, while gender-neutral predictions fit a no-preassumptions objective.Source: §6.3 · p. 8

• Prior NLP bias research includes local bias detection, output-distribution manipulation, and documented bias in embeddings, coreference systems, and masked language models.Source: §7 · p. 8
• Causal test-design work studies effects of input variations, uses structural causal models in controlled text generation, and includes systematic capability and fairness evaluation.Source: §7 · p. 8
• Counterfactual invariance frames stable predictions under irrelevant feature changes as an expectation, while perturbation-based stress tests probe vulnerabilities.Source: §7 · p. 9

• The proposed benchmark framework specifies five desiderata: no template confounding, objective stereotype labeling, a small prediction space, explicit and implicit bias measurement, and demographic inclusion.Source: §8 · p. 9
• Applying the occupational-bias setup to state-of-the-art open-source LLMs revealed substantial occupational gender bias.Source: §8 · p. 9
• The paper discusses debiasing methods and extends the framework to gender bias in education to illustrate its generalizability.Source: §8 · p. 9

• Prompt performance can vary across prompts, so the authors cannot make definitive claims about prompt strategies in different settings.They identify use-case-specific prompt selection as a direction for future research.Source: Limitations · p. 9
• Finite task prompts may leave template–output spurious correlations unmeasured, which might lead to underestimating model bias.The setup reduces measurement noise but cannot guarantee noiseless estimates.Source: Limitations · p. 9
• The experiments concern occupational gender bias in the U.S., which may limit the methods’ applicability in other cultural contexts.Source: Limitations · p. 9
• The benchmark represents non-binary genders with “they” as a practical proxy, although other pronouns exist and “ze” and “xe” rarely appear in the tested models.Source: Limitations · p. 9

Ethical Considerations

• The work frames reducing harmful biases as important for responsible language-model deployment and uses public, aggregated employment statistics without individual-level data.The employment data contain no privacy-sensitive information.Source: Ethical Considerations · p. 9

A Gender Verbalizations

• Gender verbalizations include male, female, and diverse expressions, with capitalized and lowercase forms included in probability calculations.The listed forms include pronouns and terms such as “Nonbinary,” “They,” and “Them.”Source: A Gender Verbalizations · p. 14

B Occupation Data

• Occupation data from 2021 U.S. Bureau of Labor Statistics lists were curated to consolidate highly specific job names into broader terms.The resulting list reports occupations and corresponding male and female worker ratios.Source: B Occupation Data · p. 14

C Prompt Templates

• Explicit prompts elicit gender predictions directly, while implicit prompts invite discussion of job-related experiences without explicitly mentioning gender.The explicit templates aim for a gender pronoun as the first generated word; implicit prompts explore underlying associations.Source: C Prompt Templates · p. 14

E Additional Results

• Table 5 presents the selected occupations using U.S. Bureau of Labor Statistics employment data.The accompanying description says the listed occupations were curated and paired with male and female worker ratios.Source: Table 5 · p. 15
• Tables 6 and 7 are captioned as explicit- and implicit-bias evaluation prompt sets.The supplied captions identify the prompt categories, not their comparative outcomes.Source: Table 6 · p. 16, Table 7 · p. 17
• Table 8 lists the debiasing instructions, ordered from the most general to the most specific.Its caption provides the ordering criterion but not instruction effects.Source: Table 8 · p. 18
• Tables 9–14 report debiasing-prompt results for Llama-3-8B, Llama-3-8B-Instruct, Mistral-7B, Mistral-7B-Instruct, Llama-2-7B, and Llama-2-7B-Instruct.The supplied captions identify model coverage but do not expose the result values.Source: Table 9 · p. 19, Table 10 · p. 19, Table 11 · p. 19
• Tables 15–18 report results for Gemma-7B, Gemma-7B-Instruct, Gemma-2-9B, and Gemma-2-9B-Instruct; Tables 15, 16, and 18 concern debiasing prompts, while Table 17 concerns debiasing instructions.Source: Table 15 · p. 20, Table 16 · p. 20, Table 17 · p. 21