Source-linked AI summary
Looking for a Handsome Carpenter! Debiasing GPT-3 Job Advertisements
Conrad Borchers, Dalia Sara Gala, Benjamin Gilburt, Eduard Oravkin, Wilfried Bounsi, Yuki M. Asano, Hannah Rose Kirk
TL;DR
The paper addresses limited evidence on how language-model gender bias appears in applied tasks such as job-ad generation, where biased wording may discourage applicants. It compares GPT-3 job ads generated with zero-shot prompts, prompt-engineering, and fine-tuning. Prompt-engineering does not improve bias or realism, whereas fine-tuning on low-bias real ads yields the most unbiased and realistic ads.
Problem
Research on language-model gender bias has rarely examined applied generation tasks such as job ads, where biased wording may discourage candidates from applying.
Source: §1 · pp. 1–2Method
The authors compare zero-shot GPT-3 job ads with prompt-engineered and fine-tuned ads, evaluating text-level bias and realism against real-world ads.
Source: §1 · p. 2, §3.2 · p. 3, §5 · p. 6Results
Takeaways & Limitations
Carefully selecting a small sample of low-bias real job ads may support resource-friendly reduction of gender bias while retaining realism.
Source: §1 · p. 2, §5 · p. 6The study cannot experimentally determine whether less-biased ads lead to a more diverse applicant pool.
Source: §5.1 · p. 7
Abstract
from arXiv · showhide
The growing capability and availability of generative language models has enabled a wide range of new downstream tasks. Academic research has identified, quantified and mitigated biases present in language models but is rarely tailored to downstream tasks where wider impact on individuals and society can be felt. In this work, we leverage one popular generative language model, GPT-3, with the goal of writing unbiased and realistic job advertisements. We first assess the bias and realism of zero-shot generated advertisements and compare them to real-world advertisements. We then evaluate prompt-engineering and fine-tuning as debiasing methods. We find that prompt-engineering with diversity-encouraging prompts gives no significant improvement to bias, nor realism. Conversely, fine-tuning, especially on unbiased real advertisements, can improve realism and reduce bias.
1 Introduction
The paper studies whether GPT-3 can generate realistic, less-biased job ads and compares prompt-engineering with fine-tuning. Prompt-engineering does not improve bias or realism, while fine-tuning on low-bias real ads achieves the best outcomes.
1 Introduction
- Biased job-ad language may discourage candidates from applying, while prior research links gendered wording with traditional gender divisions in the labour force.The paper frames this as a risk of allocational harm in an applied setting.Source: §1 · pp. 1–2
- Prior work documents gender associations in language models, but their translation into applied language-generation tasks and job ads remains unclear.The authors also note a lack of guidance on debiasing outputs while retaining quality, accuracy, and realism.Source: §1 · p. 2
- Fine-tuning GPT-3 on low-bias real ads produces the best outcomes, while prompt-engineering improves neither bias nor realism.The figure contrasts both approaches with a highly biased, readily identifiable zero-shot baseline.Source: Figure 1 · p. 1, §1 · p. 2
- The study applies GPT-3 to job-ad generation, quantifies text-level gender bias, and compares zero-shot, prompt-engineered, and fine-tuned ads for bias and realism.The authors describe this applied scenario as previously unresearched to their knowledge.Source: §1 · p. 2
2 Bias Statement
The paper frames gender-biased job-ad language as a possible source of representational and allocational harms. Its text-level measure captures overt word patterns, but may miss covert forms of bias.
2 Bias Statement
- Gender-biased language may reinforce gendered power hierarchies and unequal divisions of job applications if it discourages gender-diverse candidates.The paper distinguishes representational harm from allocational harm involving access to labour-market opportunities.Source: §2 · p. 2
- The study uses gender-coded word lists to measure overt text-level bias, while noting that the lists may reflect cultural or data biases.Using multiple measures is intended to cover blind spots or specific biases in any one list.Source: §2 · p. 2
- The proposed metric may be less effective at detecting covert forms of gender bias in job ads or job search more broadly.The paper identifies these forms as potentially equally damaging.Source: §2 · pp. 2–3
3 Methods
The methods define job-ad generation and evaluate practical prompting and fine-tuning approaches on selected UK jobs. Bias is measured from gender-laden word frequencies, while realism is assessed using a human-versus-GPT-3 discriminator validated with annotators.
3 Methods
- Job ads are defined as texts of typically 100–500 characters advertising a specific opening, evaluated against human-written ads using generation protocols and bias and realism measures.The methods focus on outputs that are unbiased and realistic.Source: §3 · p. 3
3.1 Data Collection and Generation
- The study selects 15 UK jobs by labour-force prevalence, job-ad availability, and GPT-3’s estimated gender association, then collects real ads and generates ads with default GPT-3 settings.The selected jobs span male-biased, female-biased, and neutral categories; real ads were collected from a UK job site.Source: §3.1 · p. 3
3.2 Experimental Conditions
- Experimental conditions compare real and neutral-prompt zero-shot baselines with diversity- or bias-explicit prompts and fine-tuning on real-ad datasets.Fine-tuning datasets include all real ads, the 10% least-biased ads, or the 10% most-biased ads; combining prompting and fine-tuning worsened bias and realism.Source: §3.2 · pp. 3–4
3.3 Measuring Bias
- Text-level bias is measured as the frequency of gender-favouring words using a composite score based on six existing lists and principal-component analysis.The authors caution that word-list frequencies do not directly establish biased outcomes and may miss other forms of bias.Source: §3.3 · p. 4
3.4 Measuring Realism
- Realism is estimated from a classifier’s probability that an ad is real, with human annotators labeling a sample as human- or machine-generated.The classifier uses truncated texts and a train-validation-test split of real and generated ads.Source: §3.4 · pp. 4–5
- Figure 2 presents reciprocal projections of word-count bias measures onto the first two principal components for real and neutral-prompt zero-shot ads.The PCA is performed on six bias measures.Source: §3.3 · p. 4, Figure 2 · p. 5
4 Results
Fine-tuning on low-bias real ads reduced bias across job types more than the real-ad average while retaining realism. Prompt-engineering showed little overall benefit, and the PCA structure found in real ads did not replicate for synthetic ads.
4.1 Dimensionality Reduction
- PCA of real-ad bias measures yielded components explaining 28% and 18% of variance, but that model did not replicate for synthetic ads.The study consequently defined its bias measure as the average frequency of selected male-bias words.Source: §4.1 · p. 5
4.2 Debiasing Experiments
- Prompt-engineering did not effectively improve bias or realism overall, aside from a small significant bias reduction for male-dominated jobs.Average bias was marginally worse than the zero-shot baseline, with substantial variation between prompts.Source: §4.2 · p. 5
- Fine-tuning on real ads produced longer, more realistic ads, averaging 260 words versus 82 words for the zero-shot baseline.Annotators labeled fine-tuned ads as real in 90% of high-bias cases and all low-bias cases.Source: §4.2 · pp. 5–6
- Fine-tuning on low-bias ads significantly reduced bias across all job types, below the real-ad average, while retaining realism.Figure 3 compares bias by job category; the reported realism was retained.Source: §4.2 · p. 6, Figure 3 · p. 6
5 Discussion
The discussion finds low-bias fine-tuning promising, while emphasizing measurement, generalizability, and real-world impact limits. The experiments address binary gender bias in job-ad text, not whether ads change applicant diversity or perform effectively in practice.
5 Discussion
- Fine-tuning on 127 carefully selected low-bias real ads produced the most unbiased and realistic ads, while prompt-engineering did not improve both outcomes.The authors conclude that careful sample selection, rather than sample size, matters for this result.Source: §5 · p. 6
5.1 Limitations and Future Work
- The text-level bias measures are simplistic, rely on gender-word lists, and are also used to filter fine-tuning ads.The authors note this may reinforce linguistic gender stereotypes.Source: §5.1 · p. 6
- Realism was proxied by a classifier and checked in a small human-annotation study, but reactions to machine-written ads in real settings remain unassessed.Some generated ads were still nonsensical or repeated prompt text.Source: §5.1 · p. 6
- The authors describe generative job ads as unlikely to be widely used soon, since human review and company-specific adaptation may erode computational cost savings.They suggest rewriting human-authored ads as a possible near-term application.Source: §5.1 · p. 7
- The study cannot establish that less-biased text leads to more diverse applicants; applicant effects require further social-science and experimental research.Gender-neutral wording may still dissuade a particular group from applying.Source: §5.1 · p. 7
- The study measures binary gender bias, not non-binary genders or other protected characteristics, for which suitable word lists may be harder to find.The authors call for more diverse measures and cross-validation against independent bias measures.Source: §5.1 · pp. 6–7
6 Conclusion
The conclusion presents low-bias fine-tuning on pre-selected real job ads as a potentially effective, resource-friendly way to reduce gender bias while retaining realism. It says achieving both goals matters for safe adoption in hiring and other settings with allocational-harm risks.
6 Conclusion
- Fine-tuning on pre-selected low-bias ads from real job sites may reduce gender bias in generated ads while keeping them realistic to human-written text.The authors characterize this as potentially resource-friendly.Source: §6 · p. 7
- The authors say both reduced bias and realism are required for safe adoption in downstream tasks such as hiring and job search..Source: §6 · p. 7
- The conclusion frames the method as potentially effective, rather than establishing a guarantee of success across applications.The stated scope is job ads from real job-market sites.Source: §6 · p. 7
A Ethical Considerations and Risks
The ethical discussion highlights misuse risks and limits to fine-tuning's feasibility. It describes the work as an effort to reduce societal harms associated with GPT-3 use.
A Ethical Considerations and Risks
- Downstream users may still apply generative models in unsuitable scenarios without measuring or mitigating output bias.The paper highlights the risk that models may propagate or amplify societal biases.Source: A Ethical Considerations and Risks · p. 10
- Fine-tuning may not be viable where users lack basic programming ability or an adequate pre-existing dataset.The authors report that only a few hundred examples were effective in their experiments.Source: A Ethical Considerations and Risks · p. 10
- The authors frame the work as highlighting viable ways to reduce societal harms from model use, consistent with GPT-3's stated intended-use terms.The work is described as a narrow generative use case.Source: A Ethical Considerations and Risks · p. 10
B Further Detail on GPT-3 Hyperparameters
The experiments used GPT-3 with specified parameters and selected a 500-token maximum experimentally by observing how completion limits affected generated-ad length.
B Further Detail on GPT-3 Hyperparameters
- The experiments used OpenAI’s Davinci GPT-3 model with specified parameters.The supplied passage does not list those parameter values.Source: B Further Detail on GPT-3… · p. 10
- 500 max tokens was selected experimentally by progressively increasing the completion limit and observing generated word counts.The procedure used the zero-shot prompt “Write a job advertisement for a {job}.”Source: B Further Detail on GPT-3… · p. 10
- Figure 4 relates the max-token setting to the mean length of generated job advertisements.Source: Figure 4 · p. 10
C Job Selection
The study selected occupations using labour-market prevalence, online-ad relevance, and GPT-3’s gender associations, then compared neutral and diversity-oriented prompts.
C Job Selection
- Job titles were translated into accessible forms, related roles consolidated, and duplicate roles removed to support more generalisable conclusions.Source: C Job Selection · p. 10
- Prompt 5, “What gender is the {job}? The {job} is a [token],” yielded the highest proportion of gendered tokens among the tested prompts.Source: C Job Selection · p. 10
- Table 2 reports UK labour-market statistics for sampled occupations: registered workers (prevalence), online job ads (relevance), and GPT-3 bias margin, where 1 denotes all-male and -1 all-female completions.Source: Table 2 · p. 11
- Genderedness measures the share of returned tokens that are gendered, while bias margin measures female-token overrepresentation relative to a 50:50 distribution.The passage describes bias margin values from 0 to 1 as male-biased.Source: C Job Selection · p. 11
D Neutral and Engineered Prompts
- Zero-shot generation uses a prompt without predefined examples; prompt-engineering manipulates prompts to context-force desired output behaviour.The prompt comparison used 18 prompts: nine neutral and nine equality-and-diversity prompts.Source: D Neutral and Engineered Prompts · p. 11
- Table 3 presents bias and realism scores for the neutral and engineered prompts.Source: D Neutral and Engineered Prompts · p. 11
E Constructing Bias Measures
The paper combines word-count ratings into a PCA-based composite bias measure and describes several component measures based on gendered language and associated linguistic properties.
E Constructing Bias Measures
- The main-paper bias metric averages word-count ratings based on the principal component analysis.Source: E Constructing Bias Measures · p. 11
- Gender-Coded Word Prevalence operationalises Gaucher et al. (2011)'s method using masculine- and feminine-themed words in job ads and counts their prevalence in a given text, based on 42 masculine and 40 feminine words.Source: E Constructing Bias Measures · p. 11
- Superlative prevalence measures the frequency of established positive and negative superlatives used to describe job candidates.The measure uses a set of superlatives provided by Veale (2016).Source: E Constructing Bias Measures · p. 11
- Gender-laden scoring counts the unweighted prevalence of words scored for gender-ladenness and gender replication.The underlying list contains 2,311 words scored across 32 properties related to norms.Source: E Constructing Bias Measures · p. 12
- Connotation-frame scoring combines positive power and agency signifiers, using annotated transitive-verb lists with 64% overlap.Source: E Constructing Bias Measures · p. 12
- The NRC VAD measure counts words with a valence, arousal, or dominance rating above 0.75 on a 0-to-1 scale.The passage notes that weakness, passiveness, and badness may be associated with a female stereotype.Source: E Constructing Bias Measures · p. 12
F Abnormalities in Generated Ads
Generated job ads show several kinds of abnormal output: some zero-shot completions are instructions or descriptions rather than ads, while prompt-engineered and fine-tuned outputs also contain problems. These examples point to the need for human-directed assessment.
F Abnormalities in Generated Ads
- The examples indicate shortcomings in zero-shot, prompt-engineered, and fine-tuned completions and point to a need for human-directed assessment.Source: F Abnormalities in Generated Ads · p. 13
- Some zero-shot completions state what an ad should include or describe the job, rather than providing an advertisement.Examples include an instruction about listing a social worker’s responsibilities and qualifications, and a definition of an artist.Source: F Abnormalities in Generated Ads · p. 13
- Some prompt-engineered completions simply echo the prompt, while others make overt references to diversity in hiring.One example addresses applications from artists of color.Source: F Abnormalities in Generated Ads · p. 13
- Some fine-tuned ads do not match the specified job description.One completion advertises a locum staff nurse role in adult acute medicine.Source: F Abnormalities in Generated Ads · p. 13
- Other fine-tuned ads describe the correct job but include candidate requirements unrelated to the job’s nature.One example asks a team support engineer to be willing to work as a Christian without compromising personal values.Source: F Abnormalities in Generated Ads · p. 13
- Fine-tuned ads can contain repeated strings, ranging from repeated fragments such as “VID” to repeated whole words such as “bonus.”The examples include a long run of “VID” and a long run of “bonus.”Source: F Abnormalities in Generated Ads · p. 13