Competence-Preserving Resume Perturbations Expose Presentation Sensitivity in LLM Screening

1 Introduction

• Resume presentations can vary in wording, structure, stylistic polish, and extraction artifacts without changing whether the candidate is qualified.The central challenge is distinguishing competence variation from variation in how the same evidence is expressed.Source: §1 · p. 1
• Presentation stability remains a complementary robustness question to prior work on screening validity, demographic bias, AI-written resumes, and AI recommendations.The authors note that presentation variation arises from writing assistance, templates, formatting conventions, and document-conversion pipelines.Source: §1 · p. 1
• 0.781 validity and 29.6% reversals for Llama-3.1-chat, and 0.644 validity with a 41.4% flip rate for Mistral, show that validity and presentation stability can diverge.The audit constructs occupation-grounded profiles at controlled competence levels, renders multiple resume forms, and excludes variants that alter evidence before scoring.Source: §1 · p. 1
• Native chat formatting improves validity for several chat-tuned models but does not eliminate presentation sensitivity.This is consistent with evidence that LLM behavior can depend on prompt format and other semantically incidental choices (Sclar et al., 2024; Chatterjee et al., 2024).Source: §1 · pp. 1–2

2 Benchmark Framework

• The framework evaluates known-superiority validity and presentation invariance on the same underlying candidates, using validated resume text rather than hidden competence labels.Figure 1 presents the benchmark construction and scoring flow.Source: §2 · p. 2, Figure 1 · p. 2

2.1 O*NET-Grounded Candidate Profiles

• 17 occupations yield 102 synthetic profiles, with two qualified, two borderline, and two underqualified candidates per occupation.Rubrics draw on occupation titles, required skills, knowledge elements, and representative core tasks; profiles vary in skill evidence, experience, and achievement strength.Source: §2.1 · p. 2

2.2 Competence-Preserving Resume Perturbations

• Five resume forms vary verbosity, structure, AI polish, and layout while keeping intended job-relevant evidence fixed; 505 of 510 variants pass deterministic validation.For same-tier base pairs, validation also requires exact agreement in the evidence vector; 1,000 matched comparisons support the invariance audit.Source: §2.2 · p. 2

2.3 Independent Screening Protocol

• Resumes are scored independently, and pairwise decisions are computed only after both scores are assigned, avoiding pair-order and comparative prompt-position effects.For the invariance audit, a decision change reflects movement in independently assigned resume scores rather than a changed pairwise prompt context.Source: §2.3 · pp. 2–3

2.4 Evaluation Metrics

• Validity measures recovery of known-superiority ordering, while flip rate measures changes from original decisions under presentation changes; a flip is not necessarily an error.Equal-tier tie rate and occupation-level Kendall’s τ provide complementary views of ties and ranking stability.Source: §2.4 · p. 3

3 Experimental Setup

• The benchmark evaluates BM25, TF–IDF, direct LLM scorers from five model families, and a deterministic phrase-preserving control.Direct LLMs include Qwen2.5-7B, Llama-3.1-8B, Llama-3.2-3B, Phi-3.5-mini, Mistral-7B-v0.3, and Gemma-2-2B.Source: §3 · p. 3
• Native chat templates are tested for Llama-3.1, Llama-3.2, and Gemma, with the main table reporting the stronger native-template condition for Llama-3.2 and Gemma.Source: §3 · p. 3

4.1 Validity and Presentation Invariance

• Validity 0.781 for Llama-3.1-8B with its native chat template coexists with a 29.6% flip rate; Mistral-7B-v0.3 reaches validity 0.644 with a 41.4% flip rate.Source: §4.1 · pp. 3–4, Table 1 · p. 4
• Validity and invariance are complementary: high invariance can preserve poor decisions, so desirable screening is both valid and stable.Source: §4.1 · p. 4
• BM25 and TF–IDF have lower flip rates (0.040 and 0.052), but phrase preservation makes them stability references rather than deployment-quality screeners.The structured control's validity 1.000 and flip rate 0.000 arise by construction and serve only as a sanity check.Source: §4.1 · p. 4

4.2 Prompt-Format Ablation

• Chat templates raise validity for Llama-3.1 (0.622 to 0.781), Llama-3.2 (0.321 to 0.551), and Gemma (0.287 to 0.576).Source: Table 2 · p. 4, §4.2 · p. 4
• Native-template flip rates remain 0.296 for Llama-3.1, 0.285 for Llama-3.2, and 0.456 for Gemma, so prompt mismatch does not fully explain presentation instability.Source: Table 2 · p. 4, §4.2 · p. 4

• The benchmark separates candidate competence from resume realization, enabling validity and presentation stability to be evaluated on the same underlying candidates.Source: §5 · p. 4
• The strongest-validity LLM conditions still reverse a large fraction of decisions under competence-preserving rewrites, and native chat formatting does not eliminate the effect.Source: §5 · p. 4
• Future audits should test both whether systems rank stronger candidates correctly and whether decisions remain consistent across equivalent forms of the same evidence.Source: §5 · p. 4

• The controlled benchmark covers 17 occupations and 102 synthetic O*NET-grounded candidates, trading breadth for attribution.Source: §6 · p. 5
• Generalization beyond the controlled setting remains untested for more occupations, human-authored resumes, and production resume-processing pipelines.Source: §6 · p. 5
• The occupation-cluster analysis has only 17 clusters, so its bootstrap intervals are pilot-scale evidence rather than precise population estimates.Source: §6 · p. 5
• The evaluated systems are open-weight instruction models; broader audits could include proprietary screeners and end-to-end applicant-tracking systems.Source: §6 · p. 5

• Each resume is scored independently against its occupation rubric; pairwise decisions are formed afterward, avoiding pair-order and comparative prompt-position effects.Source: A Implementation Details · p. 6
• Uncertainty estimates use row-level and occupation-cluster percentile bootstraps with 10,000 resamples.Source: A Implementation Details · p. 6
• Because the cluster bootstrap resamples only 17 occupations, its intervals are exploratory small-cluster evidence rather than precise nominal population coverage.Source: A Implementation Details · p. 6

B Axis-Level Presentation Sensitivity

• 0.472 for layout artifacts, 0.404 for AI-polished prose, 0.380 for structural reorganization, and 0.252 for verbosity are Qwen2.5-7B raw flip rates.Figure 2 decomposes flips by presentation axis.Source: B Axis-Level Presentation Sensitivity · p. 6
• The decomposition is illustrative rather than a universal difficulty ranking because Qwen’s validity advantage over lexical baselines does not clear zero under paired occupation-cluster bootstrap.Source: B Axis-Level Presentation Sensitivity · p. 6
• Figure 2 presents this as a directional failure-analysis slice of Qwen2.5-7B raw flips by presentation axis.Source: B Axis-Level Presentation Sensitivity · p. 6