Source-linked AI summary
The Illusion of Debiasing: Persona Steering Redistributes Rather Than Reduces Bias in LLMs
Ziyue Feng, Hongbo Fang, James A. Evans
TL;DR
Persona prompts can reshape what a model says, but whether they affect underlying associations or group disparities is unclear. Across three instruction-tuned models, the paper probes persona effects from QA through open-ended generation to word association. It finds that persona legibility does not reliably translate into bias reduction, with effects weakening at deeper probes.
Problem
It is unclear whether persona prompting changes underlying associations and group disparities, rather than only reshaping self-reports and generated tone.
Source: §1 · p. 1Method
The study probes persona-conditioned bias across three instruction-tuned models using closed-form QA, open-ended generation, and word association, while separating absolute sentiment shifts from between-group disparity.
Source: §1 · p. 2, §3.4 · p. 4Results
Takeaways & Limitations
Persona steering often redistributes or reframes measured bias rather than consistently reducing it, with influence weakening on deeper behavioral association probes.
Source: §5 · p. 8The results bound lightweight trait-descriptor prompting; longer personas, in-context examples, or fine-tuning might move bias where these prompts did not.
Source: Limitations · pp. 8–9
Abstract
from arXiv · showhide
Prompt-based interventions: system prompts, personas, role instructions, reliably reshape what a language model says, but it is unclear which layer they reach. Do they reconfigure internal structure, or only modulate the output channel? We use persona conditioning as a controlled probe, measuring its effects along a depth axis from self-report, through open-ended generation, to word-level parametric association, across three instruction-tuned models. We find a graded dissociation. Personas are legible but not structural: models follow single-trait instructions yet fail to reproduce human inter-trait covariance. The dissociation deepens with depth: personas hold or amplify closed-form QA bias, shift absolute tone while leaving between-group disparity unchanged, and barely perturb an already saturated associative baseline. Prompt-based steering thus operates in the output channel and has a structural reach limit that surface manipulability can mask.
1 Introduction
Persona prompting is widely used to shape model personality, raising the possibility of low-cost debiasing. This paper asks whether steering changes underlying associations and group disparities, finding that persona legibility does not translate into reliable bias mitigation.
1 Introduction
- Configurable characters, companion applications, and synthetic survey respondents make persona conditioning a deployed practice, including for prosocial traits such as agreeableness and honesty.The proposed appeal is that a cheap prompt might also produce fairer behavior without retraining.Source: §1 · p. 1
- Prior work establishes single-trait steerability but leaves open whether prompted profiles reproduce human inter-trait covariance and whether tone shifts alter group disparities.Explicitly unbiased models can still retain stereotype-congruent associations, a possibility not yet connected to persona steering.Source: §1 · p. 1
- The study tests three instruction-tuned models across closed-form QA, open-ended generation, and word association, separating absolute sentiment shifts from changes in between-group disparity.It uses a Between-Group Sentiment Gap and persona-induced Amplification Gap for the distinction.Source: §1 · p. 2
- Personas are legible but not structurally faithful, and they hold or amplify QA bias, shift tone without systematic disparity improvement, and weakly affect associative bias.The analysis separates self-report and surface affect from group disparity and word-level association.Source: §1 · p. 2
2 Related Work
Prior research finds that prompted personalities can be controlled, but questions remain about their structural fidelity and their effects on safety and bias across behavioral layers.
Personality in LLMs
- Personality inventories and prompting methods show that LLM traits can be shaped, while other work finds emulated profiles can have unusually sharp structure or inflated inter-item correlations (Serapio-García et al., 2025; Jiang et al., 2023; Wang et al., 2025c; Hashimoto and Oshio, 2025).Socially desirable responding in evaluation contexts also cautions against treating questionnaire scores as transparent measurements.Source: §2 · p. 2
Persona effects on bias
- Persona effects on safety are mixed: studies report increased toxicity or biased reasoning, while another finds high Agreeableness and Honesty–Humility can reduce measured bias and toxicity (Deshpande et al., 2023; Gupta et al., 2024; Wang et al., 2025a).These studies do not distinguish uniform tone shifts from changes in relative group standing or determine whether effects reach associative bias.Source: §2 · p. 2
Limits of prompt-based debiasing and implicit association
- Research on prompt-based debiasing raises concerns about persistent intrinsic bias, concealed associations, and superficial successes; word-association work probes stereotype-congruent links in parametric memory (Sivakumar et al., 2025; Himelstein et al., 2026; Yang et al., 2025; Bai et al., 2025; Kumar et al., 2024).The paper treats persona conditioning as a candidate intervention and compares that associative layer with more manipulable surface behaviors.Source: §2 · pp. 2–3
3 Methodology
The methodology tests whether prompted HEXACO profiles reproduce human personality structure and how persona conditioning affects bias across complementary tasks and progressively deeper probes.
3 Methodology
- A two-study pipeline first evaluates human-like personality structure, then examines persona effects on bias across task modalities.The design uses the same pipeline to address both questions in sequence.Source: §3 · p. 3
3.1 Models
- The evaluation uses GPT-4o-mini, LLaMA-3.1-8B-Instruct, and DeepSeek-V3.2 with identical prompts and decoding settings.The models span closed commercial, open-weights research, and large open-source systems.Source: §3.1 · p. 3
3.2 Psychometric Framework
- Personality is measured with the 100-item HEXACO-PI-R, whose six dimension scores average reverse-keyed item ratings on a 1–5 agreement scale.The study adopts HEXACO in part because Honesty–Humility is relevant to AI safety, ethical alignment, and deception.Source: §3.2 · p. 3
3.3 Study 1: Structural Fidelity of Synthetic Personality
- Study 1 prompts each HEXACO dimension HIGH, LOW, or UNCONDITIONED and compares model trait-score vectors and inter-trait correlation structure with five human populations.The human reference samples 500 respondents across Taiwan (China), Japan, Germany, Turkey, and the Netherlands; Figure 2 presents the profile projection and structural-agreement metrics.Source: §3.3 · p. 3, Figure 2 · p. 5
3.4 Study 2: Persona-Conditioned Bias
- Study 2 compares each persona condition with its model default across closed-form QA, open-ended generation, and word association.The modalities are intended to probe bias at complementary structural levels.Source: §3.4 · p. 3
- For ambiguous BBQ questions, the correct answer is “unknown”; the bias score scales with error rate and distinguishes stereotype-congruent from counterstereotypical errors, while BOLD continuations are scored for VADER sentiment.The BOLD method uses sentiment rather than toxicity to measure subtle group-dependent tonal differences in the aligned models studied.Source: §3.4 · pp. 3–4
- For BOLD, the Between-Group Sentiment Gap is the mean absolute pairwise difference between group mean sentiments, and the Amplification Gap measures its change from default.These measures distinguish absolute tonal shifts from changes in relative group standing.Source: §3.4 · p. 4
- The Word Association Test removes syntactic context to probe semantic links in parametric memory, using marginalized and mainstream identity categories and negative/violent versus positive/innocent attributes.The supplied equation passage gives the bias-score expression, while the prose specifies the test's role and categories.Source: §3.4 · p. 4
4 Results
Personas reliably move single-trait self-reports, but their effects on personality structure and bias depend on the probe. Across QA, open-ended generation, and word association, prompting does not reliably reduce bias or between-group disparity.
4.1 Single-trait instruction following
- Across every HEXACO dimension and all three models, HIGH prompting raised self-report scores and LOW prompting lowered them relative to DEFAULT.The observed ordering was HIGH > DEFAULT > LOW, establishing single-axis instruction following but not fidelity to human inter-trait structure.Source: §4.1 · pp. 5–6
4.1.1 LLMs partially reproduce human inter-trait structure
- DeepSeek-V3.2 achieved Matrix Correlation 0.432, GPT-4o-mini achieved Sign Agreement 0.691, and LLaMA-3.1-8B-Instruct had Matrix Correlation 0.056 and Frobenius Distance 1.127.DeepSeek-V3.2 and GPT-4o-mini partially reproduced human inter-trait structure; neither reached the human–human baseline.Source: §4.1.1 · p. 6
4.1.2 Conditional coupling under matched target change
- Across matched estimable targets, High-pole MAE rose from 0.34 to 0.69 for GPT-4o-mini, 0.51 to 0.87 for LLaMA-3.1-8B-Instruct, and 0.30 to 0.37 for DeepSeek-V3.2.Directional alignment deteriorated for all three models; LLaMA High · C was excluded because it failed the target-change manipulation screen.Source: §4.1.2 · p. 6
4.2.1 Closed-form QA
- Across all 12 BBQ persona conditions, residual bias remained elevated and often intensified; HonestyLow reached 100.0 on Physical appearance and SES in LLaMA-3.1-8B.The row average was 54.5, the highest condition observed, and no condition yielded systematic mitigation.Source: Figure 4 · p. 6, §4.2.1 · p. 7
4.2.2 Open-ended Generation
- Across many traits, both Low and High prompts produced more positive sentiment than Default, while Agreeableness more often increased monotonically from Low to High.Demographic groups often moved in parallel, but their relative spacing and occasionally their ordering changed.Source: §4.2.2 · p. 7
- In the large majority of BOLD conditions, persona-induced sentiment shifts left between-group disparity within sampling noise; reliable changes were model-specific, not systematic neutralization.Of 144 trait-pole×domain cells, only a small minority had a 95% bootstrap interval excluding zero; Figure 5 shows the Amplification Gaps.Source: §4.2.2 · p. 7, Figure 5 · p. 8
4.3 Implicit Bias
- Default BiasWAT means were 0.98, 1.00, and 0.93 across the three models, and persona conditioning produced no consistent attenuation across concepts or architectures.Most concepts were already at the +1 ceiling, while the vast majority of model×persona×concept changes were zero.Source: §4.3 · pp. 7–8
5 Conclusion
Across two studies and three instruction-tuned models, prompt-induced personas moved individual traits but did not reliably reproduce human inter-trait structure. Their effects on social bias were limited and uneven across the probes studied.
5 Conclusion
- Models followed single-trait instructions, but differed in how well they reproduced human inter-trait structure.The personas were legible but not structurally faithful.Source: §5 · p. 8
- Persona prompts held or amplified residual QA bias, shifted open-ended tone without systematically reducing between-group gaps, and weakly perturbed a saturated word-association baseline.The paper concludes that steering often redistributes or reframes measured bias rather than consistently reducing it.Source: §5 · p. 8
- The observed pattern is consistent with surface-level steering whose influence weakens on deeper behavioral-association probes.Whether representation-level or training-time interventions can alter these deeper patterns remains open.Source: §5 · p. 8
Limitations
The findings bound lightweight persona prompting rather than showing that bias cannot be moved by any means. The study also leaves open how finer trait levels or concrete demographic personas would behave.
Limitations
- Personas used short HEXACO facet descriptors, a light intervention that reliably moved self-reports.Longer descriptions, in-context exemplars, or fine-tuning might move bias where these prompts did not.Source: Limitations · pp. 8–9
- The negative results bound lightweight prompting; stronger persona interventions could produce different bias effects.For word association, this caveat is weaker because the baseline is already at the stereotype-congruent ceiling; it remains relevant to the other tasks.Source: Limitations · p. 9
- Using only high, default, and low trait levels captures the poles but misses the graded middle and possible dose-response effects.The authors expect the qualitative pattern to hold but do not test that expectation directly.Source: Limitations · p. 9
- HEXACO trait-level findings do not obviously generalize to named personas with demographic attributes.Such attributes may engage model associations differently, and the study does not settle whether the same surface–deep dissociation appears there.Source: Limitations · p. 9
A Experimental Prompts and Configuration
The appendix specifies the persona prompt descriptions and task-specific decoding configuration. Persona text uses HEXACO facet descriptors, while decoding differs between classification and open-ended generation tasks.
A.1 Persona Prompt Descriptions
- High- and low-pole persona descriptions adapt standard HEXACO facet definitions.Table 2 lists the descriptors inserted into “Act as someone with the following characteristics: {description}”.Source: §A.1 · p. 10
A.2 Decoding Configuration
- BBQ and WAT use greedy decoding at temperature 0.0 for reproducible single-answer classification.This setting measures the model’s most likely response rather than sampling noise.Source: §A.2 · p. 10
- BOLD and persona generation use sampling at temperature 0.8 to elicit natural, varied text.Source: §A.2 · p. 10
- Unspecified parameters use provider defaults; the locally run LLaMA-3.1-8B-Instruct additionally fixes top-p and maximum token count where noted.Source: §A.2 · p. 10
- GPT-4o-mini and DeepSeek-V3.2 are accessed through their respective APIs.Source: §A.2 · p. 10
- All three models receive identical prompts within each condition; only task-specific decoding settings differ.Source: §A.2 · p. 10
B.1 Structural Similarity Metrics
The appendix compares model and human inter-trait structures using correlation-based measures, then estimates conditional coupling and its uncertainty. It specifies bootstrap procedures and screens out unstable or invalid questionnaire conditions.
B.1 Structural Similarity Metrics
- Each source is represented by a 6 × 6 Pearson inter-trait correlation matrix, with 15 unique off-diagonal entries; human-subpopulation comparisons provide a baseline.Reported 95% confidence intervals use 1,000 bootstrap replicates, with point estimates and intervals shown in Figure 2.Source: §B.1 · pp. 10–11
B.2.1 Estimand and standardization
- Conditional coupling adjusts each off-target trait change for the target-trait change achieved, testing structural correspondence rather than causal effects.Scores are standardized using the full human sample; coupling represents change in a non-target trait per one human-SD of achieved target-trait change.Source: §B.2.1 · p. 11
B.2.2 Overall profile metrics
- For each model and pole, five off-target couplings across six target traits form a 30-element vector; cosine similarity measures its orientation relative to the human vector.Positive values indicate broadly aligned co-movement, values near zero no stable directional correspondence, and negative values predominantly opposing co-movement.Source: §B.2.2 · p. 11
B.2.3 Sampling and bootstrap procedure
- Target-specific intervals use 5,000 bootstrap replicates that resample complete human and model profiles while preserving condition-specific sample sizes.The resulting MAE intervals are percentile ranges; cell-level intervals are descriptive and are not adjusted for multiple comparisons.Source: §B.2.3 · p. 11
B.2.4 Manipulation screening and LLaMA refusals
- Dose-adjusted coupling is omitted when the achieved target change is at most 0.05 SD; LLaMA High · C was excluded after a 0.008 SD Conscientiousness change.Separately, 37 of 1,500 LLaMA Default item responses were explicit refusals; profiles failing the prespecified validity rule were excluded, leaving 10 of 15 Default profiles.Source: §B.2.4 · p. 12
B.2.5 Robustness checks
Supplementary checks retained the primary analysis’s qualitative Low–High asymmetry, while the accompanying task descriptions and BGSG procedure specify how the evaluations and uncertainty estimates were constructed.
B.2.5 Robustness checks
- The supplementary checks produced the same qualitative Low–High asymmetry as the primary analysis; cell-level intervals remain descriptive and are unadjusted for multiple comparisons.The checks included leave-one-Default-out analyses, human-tertile comparisons, and restricted cubic splines.Source: §B.2.5 · p. 12
C.1 BBQ Categories and Example Items
- BBQ uses ambiguous questions whose correct answer is “unknown”; selecting a named referent indicates reliance on the associated stereotype across eleven categories.The evaluation includes Race × gender and Race × SES intersectional categories.Source: §C.1 · p. 12
C.2.1 Domains and Example Generations
- BOLD measures open-ended generation by scoring continuations of demographic-group-associated Wikipedia-style prefixes with VADER compound sentiment across four domains.Source: §C.2.1 · p. 12
C.2.2 BGSG Estimation and Bootstrap Uncertainty
- BGSG uncertainty is estimated by resampling VADER scores within each demographic group, preserving its observed size, with 2,000 bootstrap replicates.The resulting intervals are conditional on the fixed observed group set.Source: §C.2.2 · p. 12
- Amplification Gap intervals compare independently bootstrapped condition and Default BGSG estimates; intervals excluding zero are descriptive and unadjusted for multiple comparisons.Source: §C.2.2 · p. 12
C.2.3 Complete Numerical BGSG Results
The section identifies the displays containing complete BOLD group-gap results and their accompanying sample-size and persona-notation information, but provides no cell-level results.
C.2.3 Complete Numerical BGSG Results
- Table 8 reports BGSG estimates with 95% bootstrap confidence intervals across all 156 model×persona×domain conditions.Its cell shading encodes BGSG magnitude; GPT, DS, and LL abbreviate the three models, and K is the number of demographic groups per domain.Source: §C.2.3 · p. 12
- Figure 5 presents the corresponding Amplification Gaps and marks intervals that exclude zero.Source: §C.2.3 · p. 12
- The reported group counts vary by domain and model: Gender has N = 100, while Religion has N = 290; Race and Political samples are also specified.Race and Political sample sizes vary for DeepSeek-V3.2; political samples are N = 600 for GPT-4o-mini and LLaMA-3.1-8B-Instruct.Source: §C.2.3 · pp. 12–13
- Persona notation uses H, E, X, A, C, and O for the six HEXACO traits, with subscripts marking high or low poles.Source: §C.2.3 · p. 13
C.2.4 Robustness to Alternative Sentiment Scoring
The supplementary sentiment analysis finds cross-scorer agreement for item-level valence and broad absolute shifts, while documenting its limits for relational changes; the section also describes BOLD trajectories and WAT construction and results.
C.2.4 Robustness to Alternative Sentiment Scoring
- VADER and LLM-judge ratings correlate at ρ = .453 overall, and persona-induced mean-valence changes agree in 75.7% of cells, but Amplification Gap directions agree in only 54.2%.The authors interpret this as convergent evidence for item-level sentiment and broad absolute shifts, not scorer-invariant replication of every relational change.Source: §C.2.4 · p. 13
C.2.5 Sentiment Trajectories for All Models
- The trajectory figures cover all model-domain combinations across four domains, plotting demographic-group sentiment over HEXACO Low/Default/High conditions; panel y-scales differ.Source: §C.2.5 · p. 13
C.3.1 Task Construction and Scoring
- In WAT, models assign each valenced word to one of two identity terms without sentential context, and scores are based on a single complete assignment without per-cell confidence intervals.The task format and word pools follow Bai et al. (2025).Source: §C.3.1 · p. 13
C.3.2 Default Baseline Saturation
- Default BiasWAT means are 0.98 for GPT-4o-mini, 1.00 for DeepSeek-V3.2, and 0.93 for LLaMA-3.1-8B-Instruct, with most concepts at the +1 ceiling.The score ranges from −1 to +1, where +1 is fully stereotype-congruent and 0 is random or unbiased assignment.Source: §C.3.2 · p. 13, Table 1 · p. 14
C.3.3 Persona-Conditioned Changes
- Across persona conditions, most BiasWAT changes are zero; downward shifts are sparse and model-specific, with no HEXACO trait consistently attenuating scores across concepts and architectures.The observed changes are relative to each model’s Default condition.Source: §C.3.3 · p. 14
D HEXACO Inventory Items
These appendix materials document the self-report inventory, persona prompts, decoding settings, metric tables, and task-specific displays for the paper’s analyses.
D HEXACO Inventory Items
- Table 2 gives high- and low-pole HEXACO persona descriptions, while Table 3 lists task-and-model decoding settings: greedy decoding for BBQ and WAT, sampling for BOLD and persona generation.The default persona has no trait description; the table caption specifies temperatures 0.0 and 0.8 for the two decoding regimes.Source: Table 2 · p. 15, Table 3 · p. 15
- Tables 4 and 5 report conditional-profile agreement metrics; their notes flag an unestimable LLaMA High Conscientiousness coupling after the target manipulation fell below threshold.The matched five-target LLaMA Low average is 0.51, versus 0.45 across all six Low targets.Source: Table 4 · p. 16, Table 5 · p. 16, D HEXACO Inventory Items · p. 16
- The BBQ and BOLD tables provide category or domain counts and representative examples; Table 8 presents BGSG estimates and 95% bootstrap intervals across model, persona, and domain conditions.Source: Table 6 · p. 17, Table 7 · p. 17, Table 8 · p. 18
- The WAT tables show Default-persona assignments and the shared identity and valence word pools; Table 1 defines BiasWAT’s −1 to +1 range and its stereotype-congruent endpoint.Source: Table 10 · p. 19, Table 11 · p. 20
- Tables 12 and 13 contain the 100 HEXACO-PI-R self-report items, administered in fixed order on a 1–5 agreement scale, with dimension scores averaged after reverse-keying.Source: D HEXACO Inventory Items · p. 14, Table 12 · p. 21, Table 13 · p. 22
- Figures 6 and 7 show GPT-4o-mini sentiment for Political, Race, Gender, and Religion domains, with demographic groups represented by separate lines in Figure 6.The per-domain y-scales differ in Figure 6.Source: Figure 6 · p. 23, Figure 7 · p. 23
- Figures 8–11 show DeepSeek-V3.2 and LLaMA-3.1-8B-Instruct sentiment for Gender, Religion, Race, and Political ideology; Figure 12 maps BiasWAT changes by trait and concept.Figure 12 uses red for stereotype-directed shifts and blue for shifts away; its color scale spans observed values, not the full BiasWAT bound.Source: Figure 8 · p. 23, Figure 9 · p. 24, Figure 10 · p. 24