Fairness in Large Language Models: A Taxonomic Survey

• The listed keywords are Large Language Models, AI Fairness, and Social Bias.Source: Miami, FL, USA · p. 1
• The paper is titled “Fairness in Large Language Models: A Taxonomic Survey.”Source: Miami, FL, USA · p. 1
• The listed authors are Zhibo Chu, Zichong Wang, and Wenbin Zhang, and the reference dates the work to 2024.Source: Miami, FL, USA · p. 1

1 Introduction

• LLMs span applications from chatbots and medical diagnosis to legal aid and drug discovery.Source: §1 · p. 1
• Biases inherited from the real world may lead LLMs to discriminate across social dimensions, and a cited example portrayed male-named Joseph as more leader-like than female-named Kelly [191].Source: Miami, FL, USA · p. 1
• Existing work varies in its fairness notions and methods, while prior fairness surveys largely focus on other fields, motivating an LLM-specific synthesis [39] [9].Source: Miami, FL, USA · p. 1

Miami, FL, USA

• The survey reviews LLM bias sources, fairness definitions, metrics, mitigation algorithms, evaluation resources, and research challenges.Source: Miami, FL, USA · pp. 1–2

• The taxonomy groups research into metrics for quantifying bias, algorithms for mitigating it, and resources for evaluating it.Source: §2 · p. 2
• Bias metrics are classified by data format as embedding-based, probability-based, or generation-based.Source: §2 · p. 2
• Mitigation algorithms are organized by workflow stage: pre-processing, in-training, intra-processing, or post-processing.Source: §2 · p. 2
• Evaluation resources include toolkits and datasets, with datasets classified as probability-based or generation-based.Source: §2 · p. 2

3.1 Large Language Models

• Transformers became the backbone of modern language models because they compute tokens in parallel and use input-dependent attention across tokens.The survey notes that these models have expanded in scale and exhibit capabilities across domains.Source: §3.1 · p. 2

3.2 Training Process of LLMs

• Training data should be extensive, high-quality, diverse, and representative, while cleansing can remove discriminatory or harmful content.Source: §3.2 · p. 3
• Model configuration choices—including loss functions, transformer layers, attention heads, and hyperparameters—depend on the use case and data and can influence training duration and bias.Instruction tuning can also perpetuate bias from biased instructions; RLHF incorporates human feedback into the reward function to align model behavior with human values.Source: §3.2 · p. 3

3.3 Factors Contributing to Bias in LLMs

• The survey identifies training-data bias, embedding bias, and label bias, including subjectivity introduced through annotator judgments and human feedback.Source: §3.3 · pp. 3–4

4.1 Group Fairness

• Traditional group-fairness metrics rely on class labels and demographic group counts, assumptions that may not hold for LLM tasks.For LLMs, group fairness can instead examine whether word embeddings encode biased associations, such as linking “doctor” more closely to male-associated than female-associated words.Source: §4.1 · p. 4
• Group fairness aims to prevent algorithmic decisions from favoring or harming subgroups defined by sensitive attributes.Examples include gender, race, religion, age, sexuality, nationality, and health conditions.Source: §4.1 · p. 4

4.2 Individual fairness

• Individual fairness asks that similar people receive similar outcomes, while avoiding harmful associations between sensitive words and personal identities.The section notes that measuring this for LLMs is challenging because their varied outputs complicate the label-based comparisons also required by group fairness.Source: §4.2 · p. 4

5 Quantifying Bias in LLMs

• Bias metrics are grouped into embedding-based, probability-based, and generation-based categories.The categories differ in the data format or model output used for quantification.Source: §5 · p. 4

5.1 Embedding-based Metrics

• WEAT compares target-word and attribute-word sets using cosine similarity, while SEAT applies WEAT to sentence embeddings.SEAT uses sentence templates with social-group and neutral-attribute words, allowing sentence vectors to be evaluated with the WEAT formulation.Source: §5.1 · pp. 4–5

5.2 Probability-based Metrics

• Probability-based metrics assess model-assigned probabilities, including masked-token templates and pseudo-log-likelihood comparisons of counterfactual sentences.DisCo compares top predictions across gendered template slots; LPBS adjusts for inconsistent prior probabilities, while CrowS-Pairs compares stereotypical and less-stereotypical sentence pairs using PLL.Source: §5.2 · p. 5

5.3 Generation-based Metrics

• Generation-based metrics address settings where probabilities or embeddings are difficult to obtain by measuring bias in generated text.Classifier-based metrics compare classifications of outputs from prompts differing by social group, whereas distribution-based metrics compare token co-occurrences with gendered words.Source: §5.3 · pp. 5–6

6 Mitigating Bias in LLMs

• Bias-mitigation algorithms are organized into four categories according to the stage at which they intervene in the processing pipeline.The categories are pre-processing, in-training, intra-processing, and post-processing.Source: §6 · p. 6

6.1 Pre-processing

• Pre-processing adjusts training data or prompts, including counterfactual data augmentation and hard or soft prompt tuning.Hard prompts are static templates, whereas soft prompts are dynamically created embeddings; the section notes limited flexibility for hard prompts and limited interpretability for soft prompts.Source: §6.1 · p. 6

6.2 In-training

• In-training methods modify loss functions or add auxiliary modules, requiring retraining to update model parameters.Examples include fairness constraints that penalize spurious correlations and adapter-based debiasing trained on counterfactual augmentation data.Source: §6.2 · p. 6

6.3 Intra-processing

• Intra-processing mitigates bias at inference without additional training, including model editing and decoding modification.The section describes model editing as targeted updates and DExperts as controlled decoding that combines expert and anti-expert models.Source: §6.3 · pp. 6–7

6.4 Post-processing

• Post-processing modifies generated results, with chain-of-thought prompting and rewriting presented as examples.The section reports that chain-of-thought prompts mitigated occupational gender-label bias in a benchmark and describes rewriting biased output with appropriate terms.Source: §6.4 · p. 7

7.1 Toolkits

• AIF360 supports measuring, diagnosing, and mitigating unfairness in machine-learning models, including LLMs; Perspective API scores toxicity, and Aequitas audits fairness and bias.Perspective API is used in studies [108].Source: §7.1 · p. 7

7.2 Datasets

• Probability-based datasets use templates or counterfactual sentence pairs; examples cover gender, LGBTQ+ stereotypes, and other demographic biases.WinoBias uses two gender categories; CrowS-Pairs has 1,508 pairs across nine bias types, and WinoQueer has 45,540 pairs.Source: §7.2 · pp. 7–8, Table 1 · p. 8
• Table 1 organizes datasets by suitable metric type and reports dataset size, bias types, and related work.Within probability-based evaluation, an asterisk marks counterfactual-based datasets; unmarked datasets are template-based.Source: Table 1 · p. 8
• Generation-based datasets prompt continuations from sentence prefixes: RealToxicityPrompts provides 100,000 toxicity-scored prefixes, while BOLD supplies 23,679 prompts for bias assessment.BOLD covers professions, genders, races, religions, and political ideologies.Source: §7.2 · p. 8

Formulating Fairness Notions

• Fairness notions may be tailored to applications and can conflict, leaving the choice or development of a coherent, nonconflicting set for specific LLMs an open question.Different real-world applications may require additional types of bias and tailored approaches to quantification.Source: §8 · p. 8

Rational Counterfactual Data Augmentation

• Simple attribute substitution in counterfactual augmentation can produce anomalous or irrational training sentences that may harm model performance.The example contrasts a man and a woman described as 1.9 meters tall and weighing 200 pounds; more rational replacements or data filtering are suggested.Source: §8 · p. 8

Balance Performance and Fairness in LLMs

• Fairness constraints can reduce bias, but finding the performance-fairness balance often requires manual trade-off tuning, while LLM training iterations are costly and hardware-intensive.The survey calls for systematic ways to achieve a balanced trade-off.Source: §8 · p. 8

Fulfilling Multiple Types of Fairness

• About half of existing fairness work focuses solely on gender bias, motivating attention to multiple societal demographic biases.The survey argues for broader bias categories to support a more comprehensive understanding.Source: §8 · pp. 8–9

Develop More and Tailored Datasets

• Many bias benchmarks use similar blank-slot templates, which may not capture the tailored characteristics needed to evaluate specific social biases accurately.The survey calls for more and more tailored datasets because template choices may produce discrepancies in bias scores across categories.Source: §8 · p. 9

• The survey categorizes fairness definitions, techniques for enhancing fairness, and evaluation resources, then discusses current challenges and open questions.Source: §9 · p. 9