PaperlayerView exact revision on arXiv

Source-linked AI summary

Large Language Models Systematically Favor Popular Options: Evidence and Mitigation Across MCQs

Abdelrahman Abdallah, Mohammed Ali, Bhawna Piryani, Mahmoud Abdalla, Adam Jatowt

arXiv:2608.29257v1cs.CL

TL;DR

The paper asks whether answer-option popularity confounds MCQ evaluation by making LLMs favor familiar but incorrect choices. It introduces PopMCQ to isolate the effect and PopDebias to remove the popularity prior at inference time. Across 22 models and four datasets, popularity bias is widespread, while PopDebias improves accuracy substantially under popularity pressure.

  • Problem

    LLMs may select popular but incorrect MCQ options over less-popular correct answers, threatening evaluation reliability when popularity and truth disagree.

  • Method

    PopMCQ varies distractor popularity across six controlled strategies while keeping questions and correct answers fixed; PopDebias removes a popularity prior during inference without fine-tuning.

  • Results

    PopMCQ finds popularity bias across 22 models and four datasets, including ρ = −0.89 under adversarial conditions, while PopDebias improves all models with gains up to 54.1 pp.

  • Takeaways & Limitations

    Popularity is a systematic decision-time confound in MCQs, and lightweight inference-time correction can substantially reduce its effect across tested settings.

  • Takeaways & Limitations

    The study focuses on English MCQs and open-source models with accessible output probabilities, while PopDebias requires a 10% calibration split.

Abstract

from arXiv · show

Multiple-choice questions (MCQs) are a standard format for evaluating large language models (LLMs), yet the popularity of answer options can confound evaluation. Modern LLMs systematically prefer popular but incorrect options over less popular correct ones, a vulnerability we call \textbf{popularity bias}. This pattern aligns with confidence miscalibration: model confidence remains high even as accuracy collapses for popular options. To systematically isolate this phenomenon, we introduce \textbf{PopMCQ}, a benchmark with six controlled strategies that vary option popularity while keeping the correct answer fixed. In our most adversarial setting, where all distractors are more popular than the correct option, models choose popular but wrong answers 66\% of the time. To mitigate this bias, we propose \textbf{PopDebias}, a lightweight inference-time correction that estimates and removes a popularity prior from model predictions. It requires no fine-tuning, is label-free at test time (using only a small calibration split for parameter fitting), and adds negligible computational cost. Experiments on 22 open-source LLMs (0.5B to 32B parameters) show consistent improvements, with accuracy gains up to 54.1 percentage points under strong popularity pressure. The code and data are available https://github.com/DataScienceUIBK/PopMCQ

1 Introduction

LLMs can favor familiar but incorrect MCQ options when popularity conflicts with truth, threatening evaluation reliability. PopMCQ isolates this bias with controlled strategies, and PopDebias mitigates it at inference time.

  • Popularity bias occurs when models select well-known incorrect options over less-known correct answers.The paper illustrates this with DeepSeek-V2-Lite choosing Marguerite of Valois over Marie de’ Medici.
  • Changing distractor popularity while holding questions fixed produces large accuracy swings and negative correctness–popularity correlations.This controlled comparison distinguishes popularity effects from changes in question content.
  • 22 models across four datasets show popularity bias is widespread and predictable, with ρ = −0.89 under adversarial conditions.The pattern is strongly associated with confidence miscalibration.
  • PopMCQ measures popularity bias through six controlled strategies across four QA datasets.The benchmark keeps questions and correct answers fixed while varying distractor popularity.
  • PopDebias is a training-free, label-free-at-test-time correction that removes a popularity prior with little extra computation.It is intended to mitigate the confound without retraining and works across tested settings.

2 Dataset and Strategy Construction

PopMCQ constructs controlled MCQ variants by manipulating distractor popularity while preserving each question and correct answer. Six strategies span baseline, popularity-pressure, reverse-control, and abstention settings.

  • Source data, candidates, and filtering: PopMCQ samples questions from four QA datasets and retains items meeting gold-answer, candidate-availability, and strategy-construction requirements.The sources are MuSiQue, Natural Questions, EntityQuestions, and WebQuestions.
  • Source data, candidates, and filtering: The pipeline generates 20 candidate incorrect answers with GPT-4o, then deterministically selects strategies using Wikipedia popularity scores.This separates strategy manipulation from judgment by the candidate-generating and evaluated LLMs.
  • Strategy construction: Each base question yields six four-option MCQ variants that differ only in distractor popularity, with option positions randomized.There is one correct answer and three distractors in every variant.
  • Strategy construction: S1 uses random distractors, while S2 and S3 place all distractors above the correct answer in popularity.S2 maximizes pressure with the three most popular candidates; S3 imposes a decreasing popularity gradient.
  • Strategy construction: S4 combines one highest-popularity candidate with two medium-popularity distractors, whereas S5 makes the correct answer most popular.S5 serves as a reverse control for popularity pressure.
  • Strategy construction: S6 uses “None of the above” as the zero-popularity correct answer against one highest- and two lowest-popularity distractors.This strategy stress-tests abstention under popularity pressure.

3 Investigation on Popularity Bias

Across controlled strategies, LLMs show popularity bias: popularity pressure produces negative correctness–popularity correlations, high-popularity errors, and confidence miscalibration. The effect persists across model scales and families, while reverse controls and open-recall diagnostics indicate a decision-time preference rather than simply missing knowledge.

  • Experimental Setup: 22 decoder-only LLMs from 0.5B to 32B parameters were evaluated with standardized zero-shot A/B/C/D selection across PopMCQ strategies.The evaluation covers multiple Llama, Qwen, Gemma, Falcon, Mistral, Phi, DeepSeek, and Zephyr models.
  • Systematic Bias: Under popularity pressure, correctness–popularity correlations become strongly negative, while reverse control produces positive alignment.For MuSiQue, mean ρ is −0.864 under S2 and +0.202 under S5.
  • Systematic Bias: 66.0% of selections on MuSiQue fall in the high-popularity category under S2, compared with approximately 50% under S1 and baseline under S5.Accuracy can sometimes rise under S2 because globally famous distractors may be easier to eliminate than locally plausible distractors.
  • Confidence Miscalibration: Confidence remains approximately 0.54–0.59 across popularity buckets while accuracy falls from 0.45 for Low to 0.03 for High popularity.The confidence–accuracy gap therefore increases sharply with selected-option popularity.
  • Uncertainty Handling: Under S6, the correct “None of the Above” option is selected only 11% of the time, while the most popular entity is chosen 34% of the time.For Qwen2.5-7B on MuSiQue, S6 accuracy is 3.2% and high-population selection reaches 67.5%.
  • Scaling and Model Families: Popularity bias does not diminish with scale: Qwen2.5 models show ρ ∈ [−0.82, −0.89] and HPSR > 55% across 0.5B–32B.Frontier models retain negative S2 correlations, while reasoning models attenuate but do not eliminate the bias.

4 Debiasing Method

PopDebias models popularity as a separable prior that can distort option probabilities, then removes that prior and tempers confidence in a single inference pass. Its bias estimate is fitted from a small calibration split using inference-available features, with label-free test-time operation and O(K) overhead.

  • Method Overview: PopDebias transforms a base model’s option probability vector using popularity annotations to reduce over-selection of common entities while retaining useful popularity–truth alignment.The method targets content popularity rather than option position.
  • Bias Estimation: The method estimates bias strength from inference-time features, including model confidence and properties of the current item’s popularity distribution.Bias is expected to rise for confident predictions on very common options when distractor popularities cluster.
  • Popularity-Prior Correction: PopDebias builds a within-item popularity prior, reweights and renormalizes option probabilities, then tempers them toward a more uniform distribution.The reweighting factor implements the inverse popularity prior, while adaptive tempering limits overcorrection.
  • Adaptive Tempering: The popularity-pressure factor ψ increases when one or two options dominate in popularity, indicating a sharper popular trap.When bias is small or popularity is flat, γ ≈1 and the debiased distribution remains close to the original.
  • Efficiency and Calibration: Parameters are fitted on a small calibration split, while test-time inference uses no labels, one forward pass, and O(K) per-item overhead.This contrasts with permutation averaging, which requires K! passes.

5 Main Results

PopDebias improves accuracy and calibration under popularity pressure across models, while producing minimal gains when popularity aligns with truth. It also outperforms prompting and position-based debiasing methods on the adversarial S2 setting.

  • Debiasing Results: +27.6 pp mean accuracy gain is achieved under S2 across the eight models shown, whereas S5 gains are only +2.2 pp.Falcon-7B improves from 26.5% to 76.2%, and Phi-2 from 53.3% to 81.4%.
  • Calibration: 67% ECE reduction is reported for Llama-3-8B, from 0.194 to 0.064, while Gemma-2-9b improves by 60%, from 0.447 to 0.178.Reliability diagrams show reduced overconfidence under S2 on EntityQuestions.
  • Prompting Comparison: Chain-of-thought and bias-warning prompts improve accuracy by only 0–4 pp, while PopDebias improves it by 21–34 pp on S2.Prompting leaves correlations near ρ ≈−0.85 to −0.88, whereas PopDebias reduces them to −0.65 to −0.75.
  • Debiasing Comparison: PopDebias delivers 21.5–34.1 pp accuracy gains and reduces HPSR by 22–35 pp, compared with PrideDebias gains of only 0.2–2.0 pp.The difference reflects PopDebias’s explicit modeling of entity popularity versus PrideDebias’s focus on answer-position bias.

6 Conclusion

PopMCQ demonstrates that LLMs systematically favor popular but incorrect MCQ options, and PopDebias mitigates this vulnerability across models and datasets. The benchmark links popularity-driven selection to confidence miscalibration and shows that content-aware correction outperforms permutation-based alternatives.

  • Conclusion: ρ = −0.89 under S2 and 66% popular-but-wrong selections summarize the strongest observed popularity bias.S5 reverses the correlation, isolating popularity as the driver in the controlled setting.
  • Conclusion: PopDebias improves all 22 models, with accuracy gains up to +54.1 pp and ECE reductions up to 67%.It outperforms permutation-based debiasing methods in the reported comparisons.

Limitations

The study is limited to English MCQ evaluations on open-source models and uses Wikipedia page views as an imperfect popularity proxy. PopDebias also requires calibration data and does not fully eliminate popularity bias.

  • The analysis focuses on MCQs, leaving open-ended generation and free-form question answering for future work.
  • The experiments use open-source models with accessible output probabilities; text-only closed-source APIs require adaptation.
  • PopDebias requires a 10% calibration split, which may be unavailable in zero-resource settings.
  • The evaluation covers English datasets only, so popularity patterns may differ across languages and cultures.
  • Wikipedia page views may miss familiarity encoded from non-English, domain-specific, or synthetic training data.
  • PopDebias improves accuracy substantially but does not fully eliminate negative correctness–popularity correlation under adversarial conditions.

Appendices

The appendices document dataset construction, strategy design, popularity measurement, evaluation details, robustness analyses, and expanded results. They also provide supporting diagnostics for calibration, scaling, family-level bias, and debiasing.

  • Appendix overview: The appendices cover related work, dataset construction, strategy design, popularity scoring, evaluation metrics, and the PopDebias algorithm.
  • Robustness and additional analyses: Robustness analyses examine scoring methods, prompting interactions, scaling, family-level bias, and relations to training-time debiasing.
  • Diagnostics: The open-recall diagnostic and known-answer analysis separate popularity bias from lack of knowledge.
  • Results: Appendix I quantifies confidence–accuracy gaps, while Appendices J and M report aggregate and full PopDebias results.
  • Results: Appendix K provides per-dataset results, and Appendix L presents reliability diagrams across strategies.
  • Tables: Supporting tables report dataset statistics, popularity mappings, proxy validation, known-answer diagnostics, strategy flips, calibration gaps, and aggregate debiasing results.
  • Dataset construction: The study retains questions with gold answers, at least four candidates, and satisfiable core strategies; selection does not use model outputs, difficulty, or accuracy.
  • Strategy design: Each question produces six four-option variants differing only in distractor popularity, with randomized answer positions and a random-distractor baseline.

G.1 Scoring Method Robustness

The robustness analyses show that popularity bias persists across scoring protocols, model scales, families, and prompting conditions. PopDebias consistently improves accuracy and reduces popularity-driven selection, while residual bias and evaluation-scope constraints remain.

  • Experimental controls: Identical prompts, scoring code, and models across strategies isolate option content as the source of the cross-strategy difference.
  • Scoring robustness: The probability distributions are well-spread, with entropy above 1.2 bits and fewer than 15% near-degenerate predictions.
  • Scoring robustness: Next-token and full-sequence scoring preserve the strong negative popularity-bias pattern, with the S1→S2 correlation shift virtually identical.
  • Prompting and debiasing: Few-shot prompting adds only 1–2 pp accuracy, raises HPSR from 51.7% to 54.8%, whereas PopDebias reaches 70.0% with 10-shot prompting and HPSR 23.4%.
  • Prompting and debiasing: PopDebias improves prompting combinations substantially; Qwen2.5-7B rises from 42.0% to 76.1% under Standard prompting, a +34.1 pp gain.
  • Scaling: Across Qwen2.5, accuracy rises from 37.4% to 57.8% as scale increases, but ρ changes only from −0.895 to −0.845.
  • Family-level bias: Every model family shows negative S2 correlations, ranging from −0.76 for Phi to −0.95 for Falcon.
  • Decision-time attraction: Known-answer diagnostics show sharply reduced S2 accuracy, with 99.8% of errors selecting more-popular distractors and about 29% recoverable under reverse control.

J Aggregate PopDebias Results Across Strategies

Across 22 models and four datasets, PopDebias is most effective when popularity conflicts with correctness, while avoiding substantial overcorrection when popularity aligns with truth.

  • Aggregate strategy results: +31.9 pp is the largest aggregate accuracy gain under S2, where popularity pressure is strongest.Under S5, gains are minimal at +2.5 pp; HPSR reductions are consistent across strategies, with the largest drop under S2 at −32.0 pp.
  • Aggregate strategy results: S6’s increasingly negative ρ after debiasing is desirable because correct responses require selecting the zero-popularity “None of the Above” option.
  • Dataset-level results: Under S2, PopDebias improves accuracy by +31.7 pp on EntityQuestions and changes correlation by +0.270.
  • Dataset-level results: Under S2, PopDebias improves accuracy by +38.0 pp on MuSiQue, with HPSR decreasing by 22.3 pp across all 22 models.
  • Dataset-level results: Under S2, PopDebias improves accuracy by +30.2 pp on average for NQ, where ρ reaches −0.871.
  • Dataset-level results: WebQuestions shows ρ = −0.875 under S2 and a +27.9 pp accuracy gain with PopDebias, alongside S5 reversal and S6 abstention failures.

L Calibration Analysis

PopDebias substantially improves calibration when popularity drives overconfidence, with smaller effects when popularity aligns with truth; S6 retains high residual calibration error.

  • Calibration by strategy: S1 begins with ECE = 0.237–0.497, and PopDebias reduces ECE by 16–21%.
  • Calibration by strategy: 60–67% ECE reduction occurs under S2, the most severely miscalibrated strategy.
  • Calibration by strategy: S3 and S4 show ECE reductions of 48–49% and 26–39%, respectively.
  • Calibration by strategy: S5 shows the smallest ECE reduction, at 7–11%, validating that PopDebias primarily targets popularity-induced overconfidence.
  • Confidence–accuracy relationship: Confidence remains high across popularity buckets while accuracy collapses for high-popularity selections, producing a large overconfidence gap.
  • Calibration by strategy: S6 reduces ECE by 28–30%, but absolute ECE remains high at 0.386–0.516, indicating abstention needs mechanisms beyond popularity debiasing.

M Full Debiasing Results (All 22 Models)

Across all 22 models, PopDebias improves performance under adversarial popularity conditions, with the largest gains under S2 and benefits spanning model scales and datasets.

  • Key findings: PopDebias improves all 22 models under adversarial conditions S2–S4 and S6.
  • Key findings: +24.7 pp is the average S2 accuracy gain across all four datasets, with individual gains reaching +54.1 pp.
  • Key findings: Performance is maintained or slightly improved under S5, where popularity aligns with truth.
  • Key findings: Benefits appear across models ranging from 0.5B to 32B parameters, and HPSR decreases consistently across conditions.
  • Reporting scope: Table 43 reports Accuracy, Correlation ρ, and HPSR in Before →After format across all strategies, datasets, and models.
Loading 2608.29257v1…