BiasFreeBench: a Benchmark for Mitigating Bias in Large Language Model Responses

1 INTRODUCTION

• BIASFREEBENCH compares eight prompting- and training-based techniques across seven LLMs and multiple bias-mitigation settings.The study analyzes implementation paradigms, model sizes, and bias types.Source: §1 · p. 2
• The benchmark reformats existing bias datasets into query-response settings, including single-turn multiple-choice QA and open-ended multi-turn conversational QA.The multi-turn dataset supports short- and long-context dialogue evaluation.Source: §1 · p. 2
• Bias-Free Score measures the proportion of responses that are safe, fair, and anti-stereotypical.It is designed as a response-level measure of bias in model outputs.Source: §1 · p. 2
• Prompting-based methods are consistently more effective than training-based methods; Self-Awareness improves with larger models, and DPO shows cross-bias-type generalization.The benchmark compares methods across implementation paradigms, model sizes, and bias types.Source: §1 · p. 2

2 RELATED WORK

• Earlier debiasing approaches include counterfactual-data fine-tuning, representation projection, efficient fine-tuning, prompting, adapters, unlearning, and model editing.Representation projection removes bias representations without fundamentally altering internal biases unless model parameters are modified.Source: §2 · pp. 2–3
• Many earlier evaluations used likelihood-based text modeling, including stereotype datasets scored by likelihoods of bias-associated terms or sentences.The examples named are StereoSet and Crows-Pairs.Source: §2 · p. 3
• Modern chat-LLM work explores response evaluation, prompting, multi-agent collaboration, editing, unlearning, steering, and post-training, but lacks a comprehensive comparison.The cited methods include CEB, BiasAlert, BiasGuard, DPO, SFT, and Safe RLHF.Source: §2 · p. 3

3.1 EVALUATION TASK: MITIGATING BIAS IN LLM RESPONSE

• The evaluation targets harmful bias in interactive LLM responses, using single-turn QA examples such as BBQ and a biased-response example involving productivity.The benchmark focuses on mainstream prompting- and training-based mitigation methods.Source: §3.1 · p. 3

3.2 PROMPTING-BASED MITIGATION

• Self-Awareness appends a bias-type hint to the query, while Self-Reflection asks the model to recheck its initial response and regenerate it without recognized bias.The Self-Awareness cue can name a particular bias type or a general type; reflection aims to maintain consistency with the query and initial response.Source: §3.2 · pp. 3–4
• Self-Help has the model rewrite system prompts and queries to avoid bias before a new-session response; Chain-of-Thought prompts step-by-step thinking to avoid biased responses.Self-Help requires two forward passes.Source: §3.2 · p. 4

3.3 TRAINING-BASED MITIGATION

• SFT trains on curated bias-free responses, DPO favors such responses over stereotypical alternatives, and Safe Alignment uses reward and cost models in safe reinforcement learning.The reward and cost models are trained on helpfulness and harmlessness datasets, respectively.Source: §3.3 · p. 5
• Task Vector edits a model by subtracting the weight difference between a biased SFT model and its pretrained counterpart from the pretrained model.The resulting model is designated bias-free in the described procedure.Source: §3.3 · p. 5

4.1 MODEL AND TRAINING SETUPS

• Seven instruction-tuned, reasoning, and commercial LLMs are evaluated with four prompting and four training methods; StereoSet intersentence data trains SFT, DPO, and Task Vector.DPO treats anti-stereotypical responses as positive and stereotypical responses as negative; SFT and Task Vector use anti-stereotypical responses.Source: §4.1 · p. 5

4.2 EVALUATION DATASETS AND METRICS

• The unified query-response evaluation pairs single-turn, gold-annotated BBQ with five-turn FairMT-Bench, scoring BBQ responses against ambiguous-context annotations and FairMT-Bench responses on the last turn.FairMT-Bench labels responses as biased or UNKNOWN, which includes fair or safe responses, insufficient-information answers, and refusals of potentially harmful queries.Source: §4.2 · pp. 5–6

4.3 EVALUATION WITH LLMS AND TOOLS

• Automated judgments use majority voting, with human checks showing 100% agreement for BBQ and 94% agreement for FairMT-Bench.The reported Cohen’s kappa values are 1.0 for BBQ and 0.7 for FairMT-Bench.Source: §4.3 · p. 6

5.1.1 ANALYSIS: PROMPTING-BASED MITIGATIONS

• CoT achieves the highest BFS in most cases on both BBQ and FairMT-Bench.Table 2 reports BBQ results and Table 3 reports FairMT-Bench results; the prose identifies CoT as the strongest method in most cases.Source: §5.1.1 · p. 6, Table 2 · p. 7, Table 3 · p. 7
• Self-Help improves BFS by up to 43.11 on BBQ and 7.84 on FairMT-Bench, but long FairMT-Bench contexts challenge its query rewriting.The authors report 3.81% of rewritten-query responses as semantically misaligned; Self-Awareness is second-best in most FairMT-Bench cases and costs less computation.Source: §5.1.1 · p. 7

5.1.2 ANALYSIS: TRAINING-BASED MITIGATIONS

• DPO outperforms SFT in most cases, Safe Alignment often reduces BFS, and Task Vector debiases better than alignment methods.The authors conjecture that helpfulness rewards in Safe RLHF can inhibit ambiguous responses; Task Vector also reduces general capabilities after editing.Source: §5.1.2 · pp. 7–8
• DPO, SFT, and Safe RLHF show tiny general-capability differences, whereas Task Vector decreases capabilities on BoolQ, COPA, and TruthfulQA.The reported evaluation uses accuracy differences between vanilla and debiased models.Source: §5.1.2 · p. 8, Table 4 · p. 8

5.2 DEBIASING WITH DIFFERENT MODEL SIZES

• Across five Qwen2.5 sizes, prompting methods generally outperform training methods, with higher variance; prompting BFS rises with model size while training performance stays relatively stable.Figure 4 summarizes mean and standard deviation across four prompting-based and three training-based methods.Source: §5.2 · p. 8, Figure 4 · p. 9

5.3 TRAINING WITH DIFFERENT BIAS TYPE

• DPO generalizes more strongly across unseen bias types than SFT; gender-only DPO performs comparably to DPO trained on all bias types.SFT generalizes best when trained on the full set of bias types; Figure 5 reports the bias-type results and single-type training comparisons.Source: §5.3 · p. 9, Figure 5 · p. 9
• Nearly all SFT and DPO runs on Qwen2.5-7B-Instruct have negative BFS improvements, while most such runs on Llama-3.1-8B-Instruct improve BFS.The authors conjecture that Qwen2.5-7B-Instruct’s low initial BFS makes it more difficult to debias.Source: §5.3 · p. 9

• BiasFreeBench evaluates bias mitigation techniques through LLM responses across two QA-style datasets and seven types of LLMs.The benchmark covers four prompting-based and four training-based techniques.Source: §6 · p. 10
• Bias-Free Score directly measures mitigation effectiveness in LLM outputs at the response level.Source: §6 · p. 10
• The authors aim for BiasFreeBench to serve as a unified testbed and inspire research on more effective bias mitigation for LLM responses.Source: §6 · p. 10

• The authors identify immoral use of debiasing techniques to produce biased LLM responses as a potential societal impact.Source: ETHICS STATEMENT · p. 10
• The statement characterizes such biased responses as harmful to society.Source: ETHICS STATEMENT · p. 10
• The authors advocate using their method responsibly to benefit society and minimize harm.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

• The authors say detailed experimental settings are provided across the paper and appendix, and that all experiment data and code are publicly available.The statement also says the research contains no private information.Source: REPRODUCIBILITY STATEMENT · p. 10

A LLM USAGE

• LLMs were used for experiments involving debiasing, evaluations, and LLM-as-a-judge.Source: A LLM USAGE · p. 19
• LLMs were also used to check grammar and polish the paper's writing.Source: A LLM USAGE · p. 19

B.1 IMPLEMENTATION

• BBQ prompts randomly shuffled answer options, while FairMT-Bench used its official system prompt; inference used greedy decoding except for two reasoning models.DeepSeek-R1-Distill-Llama-8B and Qwen3-8B used their model-card decoding settings; each model ran once per sample.Source: §B.1 · p. 19
• Balanced training data outperformed unbalanced data in 62.5% of cases; weighted sampling used inverse-frequency bias-type probabilities, and SFT, DPO, and Task Vector used balanced data except the analysis.Source: §B.1 · p. 19
• DPO and SFT used LoRA, Task Vector used full SFT, and Safe RLHF omitted its SFT phase and used PKU-SafeRLHF-10K for its RL phase.DPO and SFT used two 48GB RTX A6000 GPUs; Task Vector and Safe RLHF used eight 80GB H100 GPUs.Source: §B.1 · p. 20

B.2 LLM FOR JUDGMENT

• For FairMT-Bench, about 4% of responses were judged not to align with queries, prompting a second judgment call and majority vote before exclusion from BFS.BBQ had about 0.12% such responses, which were discarded; reference order was shuffled to mitigate positional bias.Source: §B.2 · p. 21

B.3 MANUAL CHECKS FOR LLM JUDGMENT

• Four graduate-student reviewers checked 100 samples each per dataset; all checked BBQ judgments were correct, and FairMT-Bench checks compared LLM judgments with human annotations.The FairMT-Bench review also used judgment agreement and Cohen’s Kappa.Source: §B.3 · pp. 21–22

C.1 BIAS TYPE DISTRIBUTION

• Table 9 reports the distribution of ambiguous samples in BBQ.Source: Table 9 · p. 22

C.2 GOLD BIAS ANNOTATION

• BBQ gold labels classify ambiguous-context answers as biased, anti-stereotypical, or UNKNOWN using the target and non-target options and context polarity.Source: §C.2 · p. 22

C.3 ONLY USING AMBIGUOUS CONTEXTS

• The evaluation excludes disambiguated contexts and uses the commonly used query format shared by most bias datasets.The authors state that explicit group information can direct models toward factual details rather than stereotypical assumptions.Source: §C.3 · p. 22

C.4 THE LINK BETWEEN BFSBBQ AND ORIGINAL BBQ METRICS

• The metric definitions use counts of biased, anti-stereotypical, and UNKNOWN outputs, with the original BBQ bias score in disambiguated contexts defined separately.Source: §C.4 · p. 23
• In ambiguous contexts, the proposed score treats a neutral UNKNOWN response as equally preferred as an anti-stereotypical response.The original BBQ-derived score weighted UNKNOWN answers by 0.5.Source: §C.4 · p. 23
• The proposed BFSBBQ is a reweighted version of the original BBQ-context score, and key conclusions remain consistent under both versions.The authors say the proposed weight of 1.0 makes trends clearer; Table 10 reports the original-score comparison.Source: §C.4 · p. 24, Table 10 · p. 24

D.1 MODEL SIZE

• Figure 9 presents Bias-Free Scores for mitigation techniques across Qwen2.5 model sizes.Source: §D.1 · p. 24

D.2 TOKEN COSTS

• Table 11 compares token usage for multi-pass Self-Reflection and Self-Help prompts with single-pass CoT and Self-Awareness prompts on Llama-3.1-8B-Instruct.Source: §D.2 · p. 24, Table 11 · p. 25

D.3 UNKNOWN RATE

• SFT and Safe RLHF produced very low UNKNOWN-response rates on BBQ, with anti-stereotypical-only SFT training reducing UNKNOWN answers and Safe RLHF facing a helpfulness trade-off.Source: §D.3 · p. 25, Table 12 · p. 25

D.4 ANTI-STEREOTYPE RATE

• Table 13 reports the ratio of anti-stereotypical responses on BBQ.Source: §D.4 · p. 25, Table 13 · p. 25

D.5 CASES

• CoT examples show models rejecting age- and socioeconomic-stereotyping, and the authors report that reasoning often surfaces social knowledge or fairness principles in bias-free answers.Source: §D.5 · pp. 25–26