Counterfactual Bias Testing for Application Tracking System

Sai Yashwant; Shruti Bansal; Anurag Dubey; Samaroha Chatterjee; Satyam Kumar; Shreyash Gupta; Gantala Thulsiram

Source-linked AI summary

Counterfactual Bias Testing for Application Tracking System

Sai Yashwant, Shruti Bansal, Anurag Dubey, Samaroha Chatterjee, Satyam Kumar, Shreyash Gupta, Gantala Thulsiram

arXiv:2608.26899v1Aug 27, 2026cs.AI

TL;DR

Auditing candidate-job matching systems for demographic bias is costly because correspondence audits require matched resumes and repeated manual submissions. This paper automates resume generation and multi-metric fairness evaluation with LLM agents, finding that score parity can coexist with substantial rank churn and borderline ranking-quality findings.

Problem
Correspondence audits provide counterfactual evidence about hiring discrimination but are expensive to construct and repeat as matching models are retrained.
Method
The paper combines task-specialized LLM agents for neutral and demographic-variant resume generation with a nine-metric statistical fairness audit across counterfactual, group-fairness, and merit-aware families.
Results
Score Delta stays within the ±0.02 PASS band for every variant, while MARC averages 4.3 to 5.3 position-changes per variant and nDCG@15 flags three cells, including BASE.
Takeaways & Limitations
The illustrative run supports multi-metric, multi-family auditing because score stability did not guarantee rank stability and targeted metrics surfaced findings missed by score- or retention-only checks.
Takeaways & Limitations
Results come from one fixed-seed experimental run without repeated trials, so borderline findings may be unstable under resampling or generation randomness.

Abstract

from arXiv · show

Automated candidate-job matching systems are increasingly classified as high-risk AI under emerging regulation, yet auditing them for demographic bias is expensive: classical correspondence-audit studies require hand-crafted resumes and manual submission, which does not scale to fast pipeline retraining cycles. This paper presents a general, reusable methodology that (1) uses task-specialized LLM agents to synthesize identity-neutral base resumes and inject controlled demographic treatments across five protected-characteristic axes (sex/gender, age, residence, language, disability), producing a K x (1+N) correspondence-audit matrix; (2) qualitatively flags inferred protected characteristics per an EU AI Act-aligned prompt; (3) ranks candidates against a job description via a fine-tuned sentence-embedding model and cosine similarity; and (4) computes a nine-metric fairness suite spanning counterfactual (score delta, mean absolute rank change, flip rate), group-fairness (top-K retention, four-fifths/impact ratio), and merit-aware (Recall@K, nDCG@K, equal opportunity, equalized odds) families, each with bootstrap confidence intervals, significance tests, and Benjamini-Hochberg correction, culminating in an automated PASS/INVESTIGATE/FAIL report with a composite risk score. On an example corpus of 5 job orders, 100 base candidates, and 10 demographic treatments (90 metric x variant evaluations): score shifts, top-K retention, and merit-aware rate gaps stay within tolerance for every treatment, but a rank-stability metric (MARC) and nDCG@K each surface borderline findings - including one on the neutral baseline itself - that a score- or retention-only view would miss. The results argue for multi-metric, multi-family auditing over any single aggregate score, and for LLM-agent-generated audits as a practical, low-cost complement to human-curated audits for any candidate-job matching pipeline.

1 Introduction

Automated candidate–job matching is now central to high-volume hiring, but correspondence audits remain difficult to scale. This paper presents an end-to-end LLM-agent methodology that generates controlled demographic variants and evaluates matching fairness quantitatively.

Motivation: Correspondence audits compare equivalent resumes differing only in protected-characteristic signals to measure differential treatment and support causal discrimination claims [Bertrand and Mullainathan, 2004, Kim, 2017].Their manual construction and submission create a scalability gap for rapidly retrained matching pipelines.
Methodology: A chain of task-specialized LLM agents generates identity-neutral resumes and controlled variants that inject one demographic treatment axis while holding qualifications fixed.The methodology treats audit generation and quantitative fairness evaluation as two halves of one pipeline.
Research questions: The audit tests whether protected-characteristic text changes relevance scores or ranks, moves candidates across top-K cutoffs, or produces merit-inconsistent shifts.These questions connect counterfactual changes to operational shortlisting and fairness consequences.
Evaluation scope: The demonstration covers 5 job orders, 100 base candidates, and 10 demographic-bias treatments, illustrating the fairness-metric suite and report generator.The corpus includes one neutral baseline and nine variants, with a sample size intended to support confidence intervals and significance tests.

2 Literature Survey

The survey connects correspondence audits, machine-learning fairness, fairness-in-ranking, and LLM-based synthetic-data generation, identifying a gap in scalable, integrated auditing for automated candidate–job matching. Prior work establishes causal audit designs and relevant fairness criteria but largely treats audit data as given and audits as manually constructed or non-continuous.

Correspondence-Audit Methodology: Correspondence audits identify discrimination by holding job-relevant attributes fixed while varying a signaled protected characteristic, enabling causal comparison of employer callbacks.The approach originated in labor-market studies using fictitious matched applications, including distinctively White- versus Black-sounding names.
Correspondence-Audit Methodology: Third-party audits can change vendor behavior, but deployed-system audits are typically hand-built, costly, and run once rather than continuously [Buolamwini and Gebru, 2018; Raji and Buolamwini, 2019].Related evidence documents automated-screening failures, including penalties for resumes containing “women’s” and gaps between vendors’ bias-mitigation claims and practice [Dastin, 2018; Bogen and Rieke, 2018; Raghavan et al., 2020].
Fairness Metrics and Ranking: Classifier fairness criteria such as equalized odds and equal opportunity condition on true labels, while ranking fairness emphasizes exposure because ordered shortlists concentrate recruiter attention at the top [Hardt et al., 2016; Singh and Joachims, 2018].These literatures motivate multiple fairness perspectives because candidate–job matching produces rankings rather than independent binary decisions.
LLM-Based Synthetic Data Generation: LLMs can exhibit measurable social bias yet generate controlled, fluent, domain-specific documents from structured constraints, making them potential generators of correspondence-audit resumes [Gallegos et al., 2024; Bommasani et al., 2021; Ouyang et al., 2022].Benchmark suites such as StereoSet [Nadeem et al., 2021] and BBQ [Parrish et al., 2022] demonstrate controlled prompting for bias probing.
Research Gap: No prior framework identified here combines agent-driven correspondence-audit generation, production-representative semantic ranking, and statistically tested counterfactual, group, and merit-aware fairness metrics.The stated gap includes per-metric significance testing and false-discovery-rate control within one auditing framework.

3 Methodology

The methodology defines a five-stage pipeline that uses task-specialized LLM agents to generate, translate, optionally flag, and quantitatively audit synthetic resumes. Its agent roles are deployment choices that may be implemented with different underlying model families rather than one fixed model.

Pipeline: The five-stage pipeline combines optional bias-descriptor elicitation, two-phase synthetic resume generation, translation, optional qualitative bias flagging, and quantitative fairness auditing.The quantitative stage ranks accumulated candidates and computes fairness metrics; Algorithm 1 and Figure 1 present the pipeline.
Pipeline: Stages 0–3 are performed by task-specialized LLM agents accessed through a standardized interface, supporting modular execution across the methodology.
Deployment: The model assigned to each agent role is a deployment choice, and the example corpus uses different model families across the agents and Stage-4 calls.No single underlying model serves all roles in the reported instantiation.

3.1 Multi-Agent Synthetic Correspondence-Audit Generation

The methodology generates identity-neutral base resumes first, then creates matched demographic variants while preserving qualification content, yielding a counterfactual audit matrix. Descriptor, generation, translation, and optional flagging agents automate treatment creation and auditing, with checks for parsing failures and qualification drift.

Agent orchestration: A configurable Descriptor Agent supplies candidate treatments, while scripted clarification and fallback parsing handle occasional refusal-like responses or malformed structured output.The pipeline confirms legitimate correspondence-audit intent before retrying, then falls back to header-based text splitting if JSON parsing fails.
Two-phase generation: The two-phase design generates K identity-neutral base resumes, then N variants per base that inject only specified demographic signals while preserving skills, experience, and job relevance.The resulting K × (1 + N) matrix compares each BASE resume with V01…V0N treatments under an unchanged match flag, making score or rank differences attributable to the injected treatment.
Controls: Including both matching and non-matching base candidates tests match persistence and prevents demotion-only audits from missing discriminatory score inflation.This design supports retaining both Equal Opportunity and Equalized Odds rather than evaluating only the former.
Validation: A qualification-preservation check applies the ranking pipeline to BASE and variant pairs, using cosine similarity to detect systematic relevance changes caused by demographic injection.A large systematic ∆qual would indicate that Phase B altered perceived qualification instead of only the intended demographic signal, undermining the two-phase design.
Downstream processing: Each generated resume can be translated after bias injection and optionally analyzed by a Flagging Agent for inferred protected characteristics and qualitative bias indicators.The flagging output is retained in the audit trail as a dedicated Protected Characteristics / Bias Analysis field.

3.2 Protected-Characteristic Axis Taxonomy and Automated Classification

The audit classifies candidates across five protected-characteristic axes using a deterministic, structured LLM call with neutral defaults for invalid outputs. The taxonomy is EU AI Act-derived and instantiated through nine illustrative variants against a fixed neutral BASE treatment, with one tractability-driven axis simplification.

Automated classification: Five protected-characteristic axes are assigned per candidate by a temperature-0 LLM JSON classification call, with invalid or missing values defaulting to a neutral bucket.The classifier reads each candidate’s candidate_type label and Stage 3 bias-analysis text, returning one value per axis.
Taxonomy limitation: Male-NA and Female-NA combine sex/gender with North-African/Arabic-origin naming cues in single sex_gender levels, simplifying the taxonomy for tractability at the cost of nuance.The paper explicitly presents this collapse as an illustrative simplification rather than a methodological requirement.
Regulatory basis: The taxonomy and qualitative bias-flagging prompt derive from the European Union’s Artificial Intelligence Act [European Parliament and Council of the European Union, 2024], selected as the regulatory starting point.The paper treats this instrument as a starting point rather than a methodological requirement.
Variant construction: Nine illustrative variants V01–V09 combine protected-axis levels for injection into every base resume, while BASE fixes the reference combination Male-FR, Young, Close, FR only, No.The variants are used for the example corpus in Section 4 and represent one specific realization of the taxonomy.

3.3 Ranking Pipeline: LLM Structuring, Embedding, and Cosine Scoring

The pipeline structures resumes and job descriptions with an LLM, then applies bounded semantic similarity scoring and within-variant ranking. Its non-generative embedding model avoids same-model self-bias because it neither authored nor was fine-tuned on the audited resumes.

LLM Structuring: The pipeline uses LLM structuring before semantic ranking to reflect realistic encoder-plus-cosine candidate–job matching behavior.For each pooled resume and job description, the structuring call returns a JSON object at temperature 0.1.
Cosine Scoring: Candidates are ranked by descending bounded similarity separately within each job and variant group, preserving each treatment’s within-variant repositioning.Similarity scores are defensively clipped to [0, 1] before entering the metrics engine.
Embedding Model: A non-generative embedding scorer avoids the audited-resume self-bias risk because it neither authored nor was fine-tuned specifically on those resumes.This contrasts with deployment settings that use an LLM itself to directly rate or rank candidates against the job description.

3.4 Fairness Metric Suite

The fairness suite compares each demographic treatment with a fixed reference across counterfactual, group-fairness, and merit-aware metrics using pooled paired observations. Metrics receive statistically supported confidence intervals and PASS/INVESTIGATE/FAIL classifications whose thresholds are legally grounded, analogical, or pre-registered.

Evaluation design: Pooling each treatment across all base identities yields K × |J| paired observations, providing the statistical power needed for meaningful bootstrap intervals and significance tests.Each variant is evaluated against a fixed reference, BASE by default, on the paired frame restricted to that treatment.
Counterfactual metrics: The counterfactual family measures score delta, mean absolute rank change scaled by shortlist size, and top-K flip rate relative to BASE.Score and rank shifts use paired bootstrap confidence intervals and Wilcoxon tests; flip rate uses Wilson intervals and McNemar’s test.
Group-fairness metrics: The group-fairness family reports macro-averaged top-K retention and an impact ratio extending the legal four-fifths rule to ranking shortlist retention.Retention weights job orders equally, while impact-ratio inference uses paired bootstrap intervals and Fisher’s exact test.
Classification and interpretation: Every metric uses PASS/INVESTIGATE/FAIL thresholds, with statistical bases identified as statutory transpositions, analogical extensions, or pre-registered policy choices.Findings are reported relative to the disadvantaged or deprioritized group, and raw values and confidence intervals contextualize boundary-adjacent classifications.
Merit-aware metrics: When ground-truth relevance labels exist, the merit-aware family adds Recall@K, nDCG@K, equal opportunity, and equalized odds restricted to the top-K prefix.Recall and nDCG assess true-match retrieval and ranked relevance, while equal-opportunity and equalized-odds gaps compare top-K true- and false-positive rates.

3.5 Statistical Validation Layer

The statistical validation layer controls false discoveries across all metric–variant comparisons before report generation and fixes bootstrap randomness for reproducibility.

Multiple-comparison correction: Benjamini–Hochberg false-discovery-rate correction is applied at α = 0.05 across all computed metrics before generating the audit report.This limits inflation of statistically significant findings from conducting up to 90 metric × variant comparisons in the example corpus.
Reproducibility: All bootstrap resampling uses a fixed random seed, ensuring reproducibility of the statistical validation procedure.

3.6 Status Classification and Composite Risk Scoring

The report classifies every metric–variant pair as PASS, INVESTIGATE, or FAIL, then aggregates these outcomes into a weighted composite risk score. The score labels reports HIGH, MEDIUM, or LOW RISK, weighting one FAIL as three INVESTIGATE findings.

Status Classification: Every metric–variant pair receives one of three statuses—PASS, INVESTIGATE, or FAIL—and the report aggregates their counts into a weighted risk score.The score uses nFAIL, nINVESTIGATE, and nPASS across all computed pairs, with ntotal as their sum.
Composite Risk Scoring: A report is labeled HIGH RISK at RiskScore ≥0.30, MEDIUM RISK at ≥0.15, and LOW RISK otherwise.The weighting treats one FAIL as equivalent to three INVESTIGATEs, making many borderline findings without outright failures less severe than even a small number of failures.

3.7 Dual-Threshold, Audience-Aware Reporting

The methodology uses deliberately stringent research/publication thresholds to communicate borderline findings as INVESTIGATE rather than absorb them into PASS. It also requires predeclared thresholds and traceability to a locked experimental configuration and random seed.

Threshold policy: Stringent research/publication thresholds surface borderline observations as INVESTIGATE instead of absorbing them into PASS.The tier is intended to demonstrate rigorous pressure-testing and support legal and regulatory scrutiny.
Integrity safeguards: Thresholds for both tiers are declared before results are computed and documented as policy choices when not directly derived from statute.The four-fifths rule is given as an example of a threshold directly derived from statute.
Integrity safeguards: Every reported result must be traceable to a single, locked experimental configuration and random seed.This traceability requirement is treated as a standing protocol rather than a one-off choice.

3.8 Automated Audit Reporting

The automated report generator creates a single dependency-free HTML executive dashboard that records audit configuration, summarizes risk outcomes, and visualizes metric-level status against thresholds.

The generator outputs a single dependency-free HTML file styled as an executive audit dashboard.
The dashboard records run configuration and summarizes overall outcomes through a color-coded risk banner and KPI cards for evaluations, FAIL, INVESTIGATE, and PASS counts.The recorded configuration includes K, reference variant, α, bootstrap count, and generation timestamp.
It lists variants triggering FAIL and provides color-coded bar charts for every metric with embedded PASS/INVESTIGATE threshold lines.

4 Results and Analysis

On an illustrative 5-job, 100-candidate correspondence-audit corpus, seven of nine metric evaluations pass uniformly, while MARC and nDCG@15 surface targeted findings that score- or retention-only auditing would miss. The results demonstrate methodology and report behavior rather than characterize any deployed matching system.

Scope and limitations: The corpus is a worked example generated to make bootstrap and significance machinery informative, not evidence about any specific deployed matching system.It spans 5 job orders, 100 base candidates, and 10 treatments.
Audit report: The report evaluates 90 metric–variant combinations, classifying 0 as FAIL, 4 as INVESTIGATE, and 86 as PASS.These comprise counterfactual, group-fairness, and merit-aware metric families across ten variants.
Counterfactual metrics: Score shifts range from +0.0013 to +0.0074 across variants, remaining inside the ±0.02 PASS band despite significant V06 testing results.V06 has p = 7.96 × 10−5 uncorrected and padj = 0.0032, illustrating that statistical significance can accompany a sub-threshold shift.
Group fairness: Top-10 retention and four-fifths impact ratios range from 0.84 to 0.92, clearing the 0.80 PASS threshold for every treatment.BASE retention is 1.0 by construction.
Merit-aware metrics: Recall@15 ranges from 0.82 to 0.88 and passes for every variant, whereas nDCG@15 ranges from 0.780 to 0.844 with BASE, V04, and V07 in INVESTIGATE.BASE nDCG@15 is 0.786, V04 is 0.793, and V07 is below the 0.80 PASS threshold; the BASE finding reflects ranking quality rather than a bias effect.
Overall findings: Seven of nine metric evaluations pass for every treatment, while MARC flags V06 and nDCG@15 flags three cells, including baseline BASE.The clean evaluations are score delta, flip rate, retention rate, impact ratio, recall, equal opportunity, and equalized odds.

5 Conclusion and Future Discussion

The paper concludes that its multi-agent, two-phase correspondence-audit methodology supports multi-metric fairness assessment while highlighting reproducibility risks near decision thresholds. Its illustrative findings remain scope-limited by a single seeded run, planned model ablations, and non-public artifacts.

Conclusion: The paper presents a two-phase, multi-agent LLM methodology that separates identity-neutral base-candidate generation from demographic-variant injection for correspondence auditing.This design structurally separates qualification signal from demographic signal, supporting audits that can reveal effects invisible to a single aggregate fairness score.
Limitations: Boundary-adjacent findings and floating-point numerical-tolerance hazards can reclassify exact effect sizes across PASS/INVESTIGATE thresholds, creating reproducibility concerns.These issues are presented as concrete considerations for fixed-threshold audit classification generally.
Limitations: The illustrative results are limited to one locked experimental run with a fixed random seed and no repeated trials, so borderline findings may be unstable under resampling.The passage specifically notes four INVESTIGATE flags, including a boundary-adjacent MARC finding, as potentially sensitive to resampling jobs, base candidates, or generation settings.
Future work: Planned ablations will test whether generation-model choice introduces synthetic-resume artifacts and whether ranking-model capability or family affects LLM-mediated evaluation.The current design cannot distinguish generator-induced artifacts from genuine findings without the first ablation, while the second targets deployments whose ranking step is LLM-mediated.
Implementation and availability: The pipeline code, prompts, example corpus, and generated HTML audit report are not publicly released but may be inspected or replicated case by case subject to organizational review.The artifacts remain internal to the deploying organization and are available only upon direct request to the authors.