Source-linked AI summary
Can Fairness Be Prompted? Prompt-Based Debiasing Strategies in High-Stakes Recommendations
Mihaela Rotar, Theresia Veronika Rampisela, Maria Maistro
TL;DR
LLMs can infer sensitive attributes from indirect cues, creating a need for accessible methods to address implicit bias in high-stakes recommendations. The paper evaluates three prompt-based debiasing strategies for user-side group fairness in LLM recommenders and finds fairness improvements up to 74% with comparable effectiveness, alongside possible demographic over-adjustment.
Problem
LLMs may infer sensitive attributes from contextual cues and bias high-stakes recommendations, while existing debiasing methods are costly, weight-dependent, or inaccessible to lay users.
Method
The paper evaluates three lightweight bias-aware prompting strategies using neutral and sensitive prompt variants across LLM recommenders.
Results
Fairness improves by up to 74% while recommendation effectiveness remains comparable, although some prompts over-adjust toward specific demographic groups.
Takeaways & Limitations
Prompting an LLM to avoid bias has promising results for user-side fairness, and BERTScore is encouraged for fairness evaluation in LLM recommenders.
Takeaways & Limitations
SNSR and SNSV do not account for the average similarity between neutral and sensitive outputs, and bias-aware prompts can over-adjust toward specific groups.
Abstract
from arXiv · showhide
Large Language Models (LLMs) can infer sensitive attributes such as gender or age from indirect cues like names and pronouns, potentially biasing recommendations. While several debiasing methods exist, they require access to the LLMs' weights, are computationally costly, and cannot be used by lay users. To address this gap, we investigate implicit biases in LLM Recommenders (LLMRecs) and explore whether prompt-based strategies can serve as a lightweight and easy-to-use debiasing approach. We contribute three bias-aware prompting strategies for LLMRecs. To our knowledge, this is the first study on prompt-based debiasing approaches in LLMRecs that focuses on group fairness for users. Our experiments with 3 LLMs, 4 prompt templates, 9 sensitive attribute values, and 2 datasets show that our proposed debiasing approach, which instructs an LLM to be fair, can improve fairness by up to 74% while retaining comparable effectiveness, but might overpromote specific demographic groups in some cases.
2 datasets
The paper is situated in recommender systems and focuses on large language models and implicit bias.
- The paper concerns recommender systems.
- Its central concepts are large language models and implicit bias.
1 Introduction
The introduction examines how implicitly inferred demographic attributes can bias high-stakes recommendations and asks whether prompting can mitigate those biases. It presents lightweight bias-aware prompting as an alternative to weight-access-dependent methods and frames over-adjustment as a potential concern.
- LLMs can infer gender or age from names, pronouns, and writing style, potentially biasing recommendations against underrepresented groups.The paper gives job recommendations that steer women away from STEM and recommend more leadership roles to men as an example.
- The paper studies user-side group fairness, requiring comparable users with different sensitive attributes to receive similar recommendations.This definition is aligned with various anti-discrimination laws.
- The research question asks how prompt-based strategies affect implicit sociodemographic biases in LLM recommenders, especially in high-stakes scenarios.The study focuses on attributes conveyed indirectly through pronouns or social roles rather than explicitly stated in prompts.
- Existing recommender debiasing methods are computationally costly, require model-weight access, or target item rather than user fairness.The introduction identifies a gap in lightweight, easy-to-use methods for users’ sociodemographic biases.
- Prompt-based debiasing has not been investigated for recommendation, and prior work warns it may exacerbate unfairness or over-favor marginalized groups.
- The paper contributes three lightweight bias-aware prompting methods, reports fairness improvements up to 74%, and introduces BERTScore-based fairness evaluation.It also studies cases where debiased recommenders overpromote underrepresented groups.
2 Methodology
The methodology compares neutral and sensitive prompt variants across baseline and bias-aware prompting designs, LLMs, and high-stakes recommendation datasets. Effectiveness uses BERTScore against ground truth, while fairness measures sensitivity of recommendations to implicit demographic attributes.
- The experiment compares recommendations generated with and without implicit sensitive attributes using neutral and sensitive prompt variants.Fairness is assessed from how much recommendations change between the two variants.
- The baseline prompt uses 10 interacted items and asks an LLM to generate five jobs or news titles.News prompts additionally request categories and subcategories.
- Bias-aware prompts minimally modify the baseline by requesting avoidance of discrimination and bias.The designs include an unbiased role prompt, a bias instruction prompt, and an explicit prompt variant.
- Sensitive variants replace “this user” with gender pronouns or age-associated social roles, while neutral variants contain no sensitive information.Job recommendations omit high school student and retired individual roles.
- The study uses Gemma 2 9B, LLaMa 3.1 8B, and Mistral 7B with greedy decoding.
- Experiments use the MIND news dataset and a job recommendation dataset, sampling 300 users per dataset and 10 history interactions per prompt.The setup runs 66.6K inferences across models, prompts, attribute values, and datasets.
- Effectiveness compares recommended items with five ground-truth items using pairwise BERTScore rather than exact matching.Exact matching is unsuitable when generated news or job titles express similar content with different wording.
- Fairness metrics compare neutral and sensitive outputs using exact-match and semantic similarity measures, then summarize group disparities with SNSR and SNSV.Higher SNSR and SNSV indicate greater disparity or inconsistency across sensitive values.
3 Experiments and Results
Bias-aware prompts generally improve fairness across LLM recommenders while preserving comparable effectiveness, though outcomes vary by model, dataset, sensitive attribute, and prompt. Similarity-based analyses also reveal that age can affect recommendations more than gender and that debiasing may over-adjust demographic representation.
- Recommendation effectiveness: F1 differences between neutral and sensitive prompts remain small for jobs and more pronounced but still low for news.Across prompt types and attributes, F1 averages 0.39 for jobs and 0.55 for news; the neutral–sensitive difference averages −0.020 for jobs and −0.011 for news.
- Fairness across groups: 74.0% SNSV (Jaccard) improvement over Base is achieved by LLaMA with BI on jobs with age attributes, from 0.208 to 0.054.LLaMA with BI and EBI is the best-performing combination on jobs.
- Fairness across groups: Bias-aware prompts generally outperform Base, but the best prompt depends on the model, dataset, sensitive attribute, and fairness metric.BI and EBI are often strongest for LLaMA and Mistral, while Base remains better for some Gemma and news-age conditions.
- Similarity to neutral recommendation: Similarity between neutral and sensitive prompts is generally higher for gender than age, indicating that age values affect LLMRec outputs more.Gender is associated with fairer outputs than age in jobs, whereas the reverse occurs for news.
- Similarity to neutral recommendation: BERTScore yields higher and more widely distributed similarity scores than Jaccard because it captures semantic rather than only word-overlap similarity.This makes BERTScore a potential alternative when recommendations differ in wording but retain comparable meanings.
- Fairness across groups: RaB indicates that Gemma tends to produce more male-gendered words, while BI is most neutral and EBI most sensitive across gender values and LLMs.‘Her’ in EBI disproportionately produces women-related news, whereas ‘him’ and ‘them’ outputs are largely unchanged.
4 Conclusions and Future Work
Across 12 LLM–prompt combinations on two high-stakes recommendation datasets, instructing LLMs to avoid bias improves fairness. However, debiased recommendations can over-adjust toward particular demographic groups, motivating improved evaluation and further comparison with other mitigation strategies.
- Fairness improves when LLMs are instructed to avoid bias across 12 combinations tested on two high-stakes recommendation datasets.
- Debiased LLMRecs can overpromote specific demographic groups, such as women in news recommendations, rather than ensure equality.
- BERTScore distinguishes recommendations generated with and without sensitive attributes better than exact-match-based metrics.
- Future work should compare bias-aware prompting with other mitigation strategies and investigate ways to avoid over-adjustment.