PaperlayerView exact revision on arXiv

Source-linked AI summary

Linguistic Triggers of Gender and Racial Bias in Open-Weight LLMs Applied to Recruitment

Kosuke Kitahara, Nobuhiro Yamaguchi

arXiv:2609.18106v1cs.CLcs.AIcs.CY

TL;DR

The paper addresses limited evidence on how job-posting language affects LLM hiring behavior, a concern relevant to high-risk recruitment systems and adverse-impact analysis. It audits open-weight models through controlled language manipulations and recruiter- and job-seeker simulations, finding that posting-language triggers are associated with hiring bias and operate through explicit demographic labels.

  • Problem

    The study addresses limited evidence on whether job-posting language itself shapes LLM hiring outcomes, beyond candidate attributes, in a setting relevant to recruitment-related regulatory concerns.

    Source: §1 · pp. 1–2

  • Method

    The authors manipulate gender- and race-related posting language and evaluate six open-weight models in recruiter and job-seeker simulations, with follow-up analyses.

    Source: §1 · p. 2, §3.4 · p. 5

  • Results

    The audit finds that posting-language triggers shape LLM recruitment bias across gender and race, with findings indicating that explicit demographic labels drive the responses.

    Source: §6 · p. 10

  • Takeaways & Limitations

    The authors identify linguistic auditing of postings as a tractable mitigation strategy and argue that coexisting racial-bias patterns make one-size-fits-all mitigation inadequate.

    Source: §6 · p. 10

    Job-seeker simulations measure stated interest from LLM personas, not real applicants’ behavior under material constraints, so they should not be treated as predictions of human behavior.

    Source: §5.5 · p. 10

Abstract

from arXiv · show

Open-weight large language models are rapidly entering hiring pipelines, yet their discriminatory failure modes -- and the regulatory exposure these create under the EU AI Act high-risk classification (Annex III) and U.S. EEOC adverse-impact analysis -- remain poorly understood. We present the first systematic, multi-model audit of open-weight LLMs that treats job-posting language as the primary experimental variable, evaluating six models (Llama 3.2, Mistral, Gemma 3, Qwen 3, Phi 3, DeepSeek-R1) across four controlled experiments that jointly probe recruiter-simulation and job-seeker-simulation tasks. We find that (1) agentic posting language depresses recruiter recommendation scores for female candidates (r_rb = 0.309, p_Bonf = 7x10^-5; model-fixed-effects r_rb = 0.448), while communal language partially reverses the penalty; and (2) coded-exclusion language suppresses non-White recruiter scores at large effect sizes (r_rb = 0.646-0.758) and, on the job-seeker side, selectively deters non-White personas from expressing interest -- operationalizing a chilling-effect mechanism at scale. A label-ablation experiment isolates the explicit demographic persona label as the primary causal driver, and Word Embedding Association Tests corroborate these findings at the representational level (d = 1.01-1.45 under Caliskan et al.'s multi-word gender attribute lists). We translate these results into a concrete pre-deployment audit protocol -- posting-vocabulary scoring, persona-conditioned LLM probing, and adverse-impact flagging against the four-fifths threshold -- that operationalizes the documentation and risk-management obligations Annex III imposes on high-risk AI in recruitment.

1 Introduction

The study examines whether job-posting language shapes LLM hiring outcomes across gender and race, from both recruiter and job-seeker perspectives. It frames this audit as relevant to regulatory concerns about discrimination in recruitment.

1 Introduction

  • Recruitment and employment AI is classified as high-risk under the EU AI Act, while the EEOC warns that AI selection may create adverse-impact risks.The paper presents linguistic auditing as relevant across these regulatory frameworks, despite differences in protected categories and liability allocation.Source: §1 · p. 1
  • The proposed pre-deployment approach combines posting-vocabulary screening with controlled persona probes to estimate group-conditional outputs and document potential adverse impact.The paper connects this audit trail to Annex III obligations and the four-fifths rule.Source: §1 · pp. 1–2
  • Prior work links agentic job-ad language to reduced belonging and application intent among women, while race-associated names and cultural matching can disadvantage minority candidates.These patterns motivate examining whether language-sensitive LLM hiring could reproduce disparities in evaluation and access.Source: §1 · p. 2
  • The audit varies posting vocabulary and candidate attributes in four controlled experiments, with follow-up analyses, to investigate when LLM hiring disparities emerge.Synthetic profiles make gender, race, and ethnicity explicit while eliciting both recruiter and applicant responses.Source: §1 · p. 2
  • The study asks whether agentic versus communal posting language changes evaluations of equally qualified candidates by gender, and whether inclusive versus coded language changes evaluations across racial or ethnic backgrounds.Both questions are examined from recruiter and job-seeker perspectives.Source: §1 · p. 2

2 Related Work

Prior research documents gendered and racialized patterns in hiring, including in algorithmic systems, but existing LLM audits have largely held job postings fixed. This study instead varies posting language and examines both evaluator and applicant responses.

2.1 Gender and Racial Biases in AI-Assisted Hiring

  • Human participants followed simulated LLM recommendations up to 90% of the time in a resume-screening experiment, even when the AI showed moderate race-based preferences.This finding underscores that model preferences can matter in human-in-the-loop settings.Source: §2.1 · p. 3
  • Existing LLM hiring audits primarily vary candidate attributes while holding postings fixed, leaving the posting’s linguistic environment unexamined as a potential bias driver.Earlier work documents gender stereotyping, racialized evaluation, and the persistence of hiring bias in AI-assisted settings.Source: §2.1 · pp. 2–3, §2.2 · p. 3

2.2 Novelty of the Present Study

  • The study varies agentic–communal and inclusive–coded-exclusion posting language, treating the posting as an experimental variable rather than a fixed stimulus.The authors describe this as the first unified multi-model audit to manipulate posting language across gender and race bias.Source: §2.2 · p. 3
  • By simulating recruiter and job-seeker perspectives together, the study examines whether the same linguistic trigger affects evaluator and applicant behavior asymmetrically.Source: §2.2 · p. 3

3 Methods

The methods use six open-weight models, controlled gender- and race-related posting stimuli, and recruiter and job-seeker simulations. Group comparisons and embedding analyses quantify differences across conditions.

3.1 Models

  • Six open-weight models, ranging from 3B to 8B parameters, were used across all experiments; temperature-zero queries and structured JSON outputs were used to limit response variation.The models were selected for deployment via Ollama on consumer hardware.Source: §3.1 · p. 3, Table 1 · p. 3

3.2 Stimuli

  • Gender stimuli comprised 40 synthetic postings with agentic or communal vocabulary, while race stimuli comprised 20 matched pairs varying inclusive versus coded-exclusion language.Gender postings co-varied titles and description bodies; race pairs held titles constant and varied the description body.Source: §3.2 · p. 4

3.3 Experimental Procedure

  • Recruiter simulations rated how well a posting suited a candidate, while job-seeker simulations rated interest; candidate profiles varied demographic attributes while other profile fields were held constant.The recruiter task uses a 1–10 recommendation-strength score, and the job-seeker task uses a 1–10 interest score.Source: §3.3 · p. 4, Figure 1 · p. 5

3.4 Analysis

  • Group differences were tested with two-sided Mann–Whitney U tests and within-experiment Bonferroni correction; effect sizes used rank-biserial correlations.WEAT compared agentic and communal posting targets using gender attribute lists, cosine similarity, and two-sided permutation tests with 2,000 permutations.Source: §3.4 · pp. 4–5

4 Results

Across recruiter and job-seeker simulations, posting language was associated with gender- and race-linked differences in evaluations and expressed interest. The findings also varied with persona framing, posting components, and explicit demographic labels.

4.1 Gender Bias and the “Agentic Language” Penalty

  • Female candidates’ recruiter scores were lower under agentic than communal postings (6.72 vs. 7.53; rrb = 0.309, pBonf = 7.2 × 10−5), while the male comparison was non-significant.The model-fixed-effects replication confirmed the female comparison (rrb = 0.448, pBonf = 3.9 × 10−9), while the male comparison remained non-significant.Source: §4.1 · pp. 5–6, Table 2 · p. 6
  • In job-seeker simulations, female personas rated agentic and communal postings similarly (8.35 vs. 8.47; pBonf = 0.623), unlike female recruiter evaluations.The male job-seeker comparison was also non-significant in the pooled analysis; model-fixed-effects analysis found only a small reverse shift.Source: Table 2 · p. 6, §4.1 · p. 6
  • Across both embedding models and title and description corpora, agentic postings had stronger male-attribute associations than communal postings (d = 1.01–1.45).Figure 2 shows the agentic distribution above the communal distribution in every panel; these embedding models differ from the behavioral chat models.Source: §4.1 · p. 6, Figure 2 · p. 7

4.2 Racial Bias and the “Cultural Fit” Penalty

  • Under inclusive postings, White candidates had lower recruiter scores than Black, Hispanic, and Asian candidates, a pattern also present but attenuated among job seekers.The recruiter gap persisted after mean-centering model scores.Source: §4.2 · p. 7
  • Black, Hispanic, and Asian candidates’ recruiter scores fell under coded-exclusion language (rrb = 0.742, 0.758, and 0.646, respectively), with larger declines than White candidates (rrb = 0.324).Model-fixed-effects replications yielded rrb values of 0.884, 0.899, 0.774, and 0.468, respectively.Source: §4.2 · p. 7, Table 3 · p. 8
  • Coded-exclusion language reduced job-seeker interest more for non-White personas (1.08–1.60 points) than for White personas (0.68 points).The pooled effects were significant for all groups; model-fixed-effects replication preserved the rank ordering.Source: §4.2 · p. 7, Table 3 · p. 8
  • With race unspecified, coded-exclusion postings reduced interest by 0.73 points uniformly; labeled-group declines exceeded that baseline for Asian, Hispanic, and Black personas.The reported label increments were −0.35, −0.52, and −0.87, respectively; White personas’ decline was within the language-only baseline.Source: §4.2 · pp. 7–8, Table 4 · p. 8
  • In the race-by-gender job-seeker experiment, Black personas had the lowest coded-exclusion scores, while within-race gender differences were negligible and no Race × Gender interaction was detected.Gender differences were at most 0.16 points; interaction tests were non-significant in both language conditions.Source: §4.2 · p. 8

5 Discussion

The discussion reports gender and racial disparities across recruiter and job-seeker simulations, including label-linked penalties and a race-dominant pattern. It also bounds generalizability and proposes a four-stage pre-deployment audit protocol.

5.1 Linguistic Style as a Bias Trigger for Gender

  • r_rb = 0.309 (pBonf = 7.2×10^-5) for female-candidate recruiter scores, with model-fixed-effects r_rb = 0.448, under agentic posting language.Three models individually reached significance, and a description-body ablation supports posting text—not job title—as the primary linguistic mechanism.Source: §5.1 · p. 8

5.2 Prosocial Overcorrection versus Cultural Fit Penalty in Racial Bias

  • Up to +2.03 points for non-White candidates under inclusive language in Gemma 3 exemplifies the descriptively termed prosocial overcorrection.The authors note that whether this preferential treatment is bias or correction is contested, while it may still raise adverse-impact concerns regardless of direction.Source: §5.2 · p. 9
  • −1.27 points for non-White candidates relative to White candidates under coded-exclusion language in Mistral was the strongest cultural-fit-language penalty.The H1 decomposition attributes the between-group differential to the explicit race label, including a −0.87 incremental effect for Black personas.Source: §5.2 · p. 9

5.3 Self-Selection and the Chilling Effect of Exclusionary Language

  • Mean ∆ = −1.31 points for non-White personas’ interest under coded-exclusion language, versus −0.68 for White personas (all pBonf < 10^-5).Without a race specification, the inclusive–coded gap is uniformly −0.73 points; race-label increments are −0.35 for Asian, −0.52 for Hispanic, and −0.87 for Black personas.Source: §5.3 · p. 9

5.4 Intersectionality and the Dominant Role of Race

  • Gender differences within racial groups were negligible (≤ 0.16 points), while race-label increments were 0.35–0.87 points; tests found no Race × Gender interaction (both p ≥ 0.21).The authors suggest the race-coded register may leave little additional signal for gendered priors, while gender-coded registers could elicit an interaction.Source: §5.4 · p. 9

5.5 Limitations

  • Generalizability is constrained by six 3B–8B models, minimal profiles with explicit demographic fields, and stated-interest simulations rather than observed job-seeker behavior.The authors caution that minimal profiles may amplify effects relative to rich-resume or implicit-inference pipelines; persona interest should not be read as a prediction of human behavior.Source: §5.5 · pp. 9–10

5.6 Governance Implications: A Pre-Deployment Audit Protocol

  • The proposed four-stage audit scores posting vocabulary, probes balanced demographic personas, flags selection-rate ratios below the four-fifths threshold, and documents inputs, outputs, computations, and revisions.The protocol is intended to support Annex III documentation, risk-management, data-governance, and transparency obligations.Source: §5.6 · p. 10
  • The four-fifths rule is U.S.-specific; other jurisdictions need local legal tests, and the protocol audits posting-language sensitivity rather than posting legality.The authors also note that S2 requires a representative grid of personas and stimulus pairs.Source: §5.6 · p. 10

6 Conclusion

The study audits how job-posting language triggers bias across open-weight LLM recruitment and job-seeker simulations. It connects these findings to a four-stage pre-deployment audit protocol.

6 Conclusion

  • The audit finds that gendered and culturally coded posting language reliably triggers recruitment biases, with effects operating through explicit demographic labels.The study combines recruiter and job-seeker simulations and reports label-ablation evidence consistent with a causal interpretation.Source: §6 · p. 10
  • Two racial failure modes coexist: inclusive language can advantage non-White candidates, while coded-exclusion language penalizes them and can chill minority job-seeker interest.The combined recruiter and job-seeker framework surfaces potential self-selection costs that callback audits cannot observe.Source: §6 · p. 10
  • The four-stage pre-deployment protocol combines vocabulary scoring, persona-conditioned probing, four-fifths adverse-impact flagging, and Annex III documentation.The protocol connects the empirical findings to operational obligations for high-risk recruitment AI in the U.S. and EU.Source: §6 · p. 10

Ethical Statement

The ethical statement describes the study's synthetic stimuli, dual-use consideration, and research framing. It also identifies limits in group representation and the authors' positionality.

Ethical Statement

  • All stimuli are synthetic, no personally identifiable information was collected, and the study required no IRB review under standard exemption criteria.The research concerns publicly deployable software.Source: Ethical Statement · p. 11
  • The research aims to expose and measure recruitment bias, with findings reported at the model-family level for practitioners, auditors, and regulators.The authors state that the work is not intended to amplify bias.Source: Ethical Statement · p. 11
  • The persona-assignment method could be repurposed to craft biased postings, a dual-use risk the authors judge outweighed by audit value.The authors note that the identified linguistic patterns are already documented in human-hiring literature (Rivera 2012; Gaucher, Friesen, and Kay 2011).Source: Ethical Statement · p. 11
  • The authors’ Japanese HR-technology industry positions and nonmembership in the studied non-White U.S. groups shape the study’s U.S.-framework-based interpretation.They ask readers to weigh their interpretations accordingly.Source: Ethical Statement · p. 11
  • Collapsing race and ethnicity into four groups simplifies representation; finer-grained analyses and non-U.S. contexts remain future directions.The demographic labels follow widely adopted social-science categorizations.Source: Ethical Statement · p. 11

A Prompt Templates

The appendix reproduces the prompt templates for recruiter and job-seeker experiments. They assign demographic information to the candidate or simulated job-seeker and supply the posting in the relevant message.

A Prompt Templates

  • Recruiter prompts place the job title and description in the recruiter’s system message and candidate details in the user message.The candidate prompt asks for a recommendation score from 1 to 10.Source: A Prompt Templates · p. 13
  • Job-seeker prompts assign the persona in the system message, then present the job title and description in the user message.The persona prompt asks for an interest score from 1 to 10.Source: A Prompt Templates · p. 13
  • The templates share the same system-message template across experiments, with the demographic field varying between conditions.In recruiter experiments the demographic label appears in the user message; in job-seeker experiments it appears in the system message.Source: A Prompt Templates · p. 13

B.1 Gender Experiment Stimuli (Gender/Recruiter, Gender/Job-Seeker)

The gender experiments use 40 job postings constructed following Gaucher et al. (2011), split evenly between agentic and communal themes. Both job titles and descriptions vary with the theme.

B.1 Gender Experiment Stimuli (Gender/Recruiter, Gender/Job-Seeker)

  • The stimulus set contains 40 postings—20 agentic and 20 communal—with job titles and descriptions co-varied by theme.These postings support the pooled and model-fixed-effects analyses in the gender experiments.Source: §B.1 · p. 13
  • Agentic examples pair titles such as Strategy Consultant and Software Architect with competitive, assertive, or independent-role language.Other examples include Senior Sales Strategist, Field Engineer, and Account Executive.Source: §B.1 · pp. 13–14
  • Communal examples pair roles such as Patient Care Coordinator and HR Partner with supportive, cooperative, or empathetic language.Other examples include Mental Health Intake Coordinator, Early Childhood Assistant, and Team Assistant.Source: §B.1 · pp. 14–15

B.2 Race/Ethnicity Experiment Stimuli (Race/Recruiter, Race/Job-Seeker)

The race and ethnicity stimuli comprise 20 job-posting pairs adapted from Rivera (2012), with each pair holding the job title constant and varying the description between inclusive and coded-exclusion registers. The examples span roles from software engineering and data analysis to healthcare, education, and retail.

B.2 Race/Ethnicity Experiment Stimuli (Race/Recruiter, Race/Job-Seeker)

  • 20 job-posting pairs adapted Rivera (2012) class- and race-coded cues to job ads, holding each pair’s job title constant while varying only the description body.The 40 entries use inclusive and coded-exclusion registers.Source: §B.2 · p. 15
  • Inclusive descriptions emphasize equal opportunity, structured or job-relevant evaluation, and welcome to applicants across racial and ethnic backgrounds.Examples include software engineering, data analysis, and UX research postings.Source: §B.2 · p. 16
  • Coded-exclusion descriptions invoke conventional professional image, established norms, or fit with leadership, clients, and organizational culture.Examples include software engineering, data analysis, product management, and UX research postings.Source: §B.2 · p. 16
  • The paired examples cover varied occupations, including marketing, HR, finance, nursing, education, research, sales, legal work, and retail.Across the pairs, inclusive versions mention practices such as structured screening, transparent criteria, and welcoming diverse applicants, while coded versions use cues such as familiar presentation or traditional norms.Source: §B.2 · p. 16
Loading 2609.18106v1…