Source-linked AI summary
Beyond the Name: Demographic Leakage in De-Identified R\'esum\'es and Evaluation Artifacts in LLM Bias Audits
Qiangju Chen, Yang Xiao
TL;DR
Redacting explicit résumé fields may not prevent ethnocultural inference, and whether unstructured prose retains demographic signals after language is controlled remains an open question. The paper tests recovery across counterfactual résumés and examines pairwise verdict-space effects. It finds substantial prose-based recovery and strong sensitivity of pairwise outcomes to evaluation design.
Problem
Whether non-language résumé prose still supports ethnocultural inference after declared languages are controlled remains untested, while pairwise audits may be sensitive to response design.
Source: §1 · p. 1Method
The study evaluates 620 counterfactual résumés with language held constant across nine open-weight models, varying cue salience and evaluator response space.
Source: §1 · p. 2Results
With language held constant, prose supports 0.757 overall recovery, and pairwise outcomes are sensitive to verdict-space design.
Source: §1 · p. 2, §3.3 · p. 3, §4 · p. 4Takeaways & Limitations
Bias audits should distinguish demographic signals recoverable from résumé content from effects introduced by evaluation design.
Source: §4 · p. 4The constructed counterfactual résumés cover five ethnocultural conditions in an Australian professional-job context, and recoverability does not establish a causal effect on hiring decisions.
Source: Limitations · p. 5
Abstract
from arXiv · showhide
De-identified r\'esum\'e screening assumes that redacting explicit fields prevents ethnocultural inference; however, recent audits attribute residual leakage to declared languages. We investigate whether eliminating language fields resolves this leakage across nine open-weight models and 620 counterfactual r\'esum\'es. By holding language attributes strictly identical, we isolate unstructured prose across five ethnocultural conditions and three cue-salience tiers. Target-group recovery averages 0.757 overall and saturates at 1.000 under high salience, demonstrating that non-language prose sustains demographic inference. Crucially, models diverge only under faint cues (0.086-0.690), establishing salience as an essential evaluation axis. Furthermore, pairwise LLM-as-a-judge outcomes are highly sensitive to evaluation design: forbidding ties yields an apparent selection-rate ratio of 0.39 alongside strong position and content effects, whereas permitting ties produces near-universal ties for most models ($\ge94\%$). Downstream scoring shows only very small between-condition differences, highlighting the need to distinguish demographic signals recoverable from r\'esum\'e content from effects introduced by the evaluation protocol.
1 Introduction
Redacting explicit fields may not prevent ethnocultural inference, and prior audits leave non-language résumé prose unassessed. This paper tests prose-based recovery with language held constant and examines whether pairwise audits reflect demographic preferences or evaluation artifacts.
Main contributions
- 0.757 overall recovery with language held constant shows that unstructured résumé prose sustains ethnocultural inference.The benchmark holds language attributes identical across all variants.Source: §1 · p. 2
- Recovery reaches 1.000 under prominent cues, while meaningful cross-model differences appear only under faint cues.The salience-tiered audit is intended to avoid assuming model equivalence from prominent-cue results.Source: §1 · p. 2
- A 0.39 selection-rate ratio under forced choice accompanies position and content artifacts in 41,717 paired comparisons.The authors characterize the ratio as spurious rather than genuine bias.Source: §1 · p. 2
2 Data and Task Formulation
The study constructs counterfactual résumés that vary injected prose while keeping declared language fixed, then formulates tasks for recovery, pairwise preference auditing, and downstream screening.
2.1 Counterfactual Résumé Construction
- 20 base résumés across four Sydney professional occupations generate 31 variants each, yielding 620 counterfactual résumés.The variants include a marker-free control and combinations of five ethnocultural conditions, three salience tiers, and two independently authored text blocks.Source: §2.1 · p. 2, Figure 1 · p. 2
2.2 The Load-Bearing Control: Identical Declared Language
- All variants declare “Languages: English (Professional proficiency),” so they differ exclusively in unstructured prose.The prose varies in community involvement, activities, and interests; removing it requires wholesale excision rather than field-by-field redaction.Source: §2.2 · pp. 2–3
2.3 Task Formulations
- Three tasks measure sociocultural recoverability, pairwise judge preferences, and downstream screening outcomes.The task formulations distinguish background recoverability, judgespace dynamics, and screening outcomes.Source: §2.3 · p. 3
- For individual recoverability, a model receives one résumé without job descriptions or demographic prompts and reports a perceived background, confidence, and optional evidence phrases.The candidate backgrounds are five ethnocultural groups or “Cannot determine.”Source: §2.3 · p. 3
- Pairwise auditing compares each marked résumé with its omission baseline in forward and reversed orders across three verdict regimes.The regimes are Forbidden, Neutral, and Encouraged; ties are disallowed only under Forbidden.Source: §2.3 · p. 3
- Downstream screening pairs a résumé with its job description and returns a binary shortlist recommendation and an absolute suitability score.The study also computes the Top-Score Rate from peer-variant maximum scores.Source: §2.3 · p. 3
3 Results and Analysis
With declared language fields held constant, unstructured résumé prose supports substantial demographic recovery, while pairwise judgments vary with verdict-space design. Downstream endpoints show limited resolution for detecting small between-condition differences.
3.1 Models and Inference.
- The study evaluates nine open-weight models spanning 8B–36B parameters in identical local environments, using greedy decoding and grammar-constrained outputs.Structured outputs achieved a 100.0% valid parsing rate across Task 1 responses.Source: §3.1 · p. 3
3.2 Finding 1: Sociocultural Residue Is Recoverable Without Language
- 0.757 overall target-group recovery persists with language fields fixed, all models reach perfect recovery at T1, and T3 rates range from 0.086 to 0.690.At T1 models do not abstain; under faint T3 cues they still commit to specific groups rather than abstaining.Source: §3.2 · p. 3, Figure 2 · p. 4
3.3 Finding 2: Pairwise Audits Measure the Verdict Space, Not the résumé
- Permitting ties yields ≥94% ties in 7 of 9 models; forced choice produces a spurious 0.39 selection-rate ratio against Anglo comparators amid strong positional dominance.Under forced choice, group-neutral blocks are selected at 0.648 versus 0.588–0.634 for ethnocultural conditions.Source: §3.3 · pp. 3–4, Figure 3 · p. 4
3.4 Finding 3: Boundary Analysis: Downstream Screening
- < 0.13 points on a 100-point scale bounds common ethnocultural effects in continuous scoring, while shortlisting is often uniform and Top-Score Rate intervals remain too wide for stable rankings.Remaining shortlisting deviations are within legal compliance thresholds; ethnocultural rankings invert unpredictably across models.Source: §3.4 · p. 4
4 Conclusion
The conclusion emphasizes that de-identifying explicit fields does not eliminate ethnocultural recoverability from résumé prose. Pairwise results also depend on evaluation design, so audits should distinguish content signals from protocol effects.
4 Conclusion
- With declared language held constant, unstructured résumé prose still supports substantial ethnocultural inference.The conclusion notes that model differences are particularly revealed under subtle cues.Source: §4 · p. 4
- Pairwise outcomes are highly sensitive to forced-choice and positional effects.Source: §4 · p. 4
- LLM bias audits should distinguish demographic signals recoverable from résumé content from effects introduced by evaluation design.Source: §4 · p. 4
Limitations
The study's scope is limited to constructed counterfactual résumés in an Australian professional-job context and open-weight models; its controlled experiments do not measure population-level hiring discrimination.
Limitations
- The experiments use constructed counterfactual résumés across five ethnocultural conditions in an Australian professional-job context.Source: Limitations · p. 5
- The study focuses on open-weight models.Source: Limitations · p. 5
- Recoverability alone does not establish a causal effect on hiring decisions, and the experiments do not measure population-level hiring discrimination.Source: Limitations · p. 5