Source-linked AI summary
Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions
Pouya Pezeshkpour, Estevam Hruschka
TL;DR
LLMs can be sensitive to answer-option order in multiple-choice questions, complicating fair assessment despite their strong capabilities. This paper measures that sensitivity across benchmarks, analyzes positional bias and top-choice patterns, and evaluates calibration methods, finding gaps of up to 75% and improvements of up to 8 percentage points.
Problem
Prior work shows LLM sensitivity to prompt wording and demonstration order, but the extent, causes, and mitigation of sensitivity to multiple-choice option order require investigation.
Method
The paper evaluates GPT-4 and InstructGPT across five benchmarks, analyzes reordered predictions and top-choice placement patterns, and tests majority vote and multiple evidence calibration.
Results
Sensitivity gaps reach up to 75% across benchmarks, while two calibration approaches improve performance by up to 8 percentage points.
Takeaways & Limitations
Positioning the top two choices first and last amplifies bias, whereas placing them adjacent mitigates it; calibration can improve robustness to option order.
Takeaways & Limitations
The analysis focuses on top-two choices, while the origins of order sensitivity require further exploration of training data.
Abstract
from arXiv · showhide
Large Language Models (LLMs) have demonstrated remarkable capabilities in various NLP tasks. However, previous works have shown these models are sensitive towards prompt wording, and few-shot demonstrations and their order, posing challenges to fair assessment of these models. As these models become more powerful, it becomes imperative to understand and address these limitations. In this paper, we focus on LLMs robustness on the task of multiple-choice questions -- commonly adopted task to study reasoning and fact-retrieving capability of LLMs. Investigating the sensitivity of LLMs towards the order of options in multiple-choice questions, we demonstrate a considerable performance gap of approximately 13% to 75% in LLMs on different benchmarks, when answer options are reordered, even when using demonstrations in a few-shot setting. Through a detailed analysis, we conjecture that this sensitivity arises when LLMs are uncertain about the prediction between the top-2/3 choices, and specific options placements may favor certain prediction between those top choices depending on the question caused by positional bias. We also identify patterns in top-2 choices that amplify or mitigate the model's bias toward option placement. We found that for amplifying bias, the optimal strategy involves positioning the top two choices as the first and last options. Conversely, to mitigate bias, we recommend placing these choices among the adjacent options. To validate our conjecture, we conduct various experiments and adopt two approaches to calibrate LLMs' predictions, leading to up to 8 percentage points improvement across different models and benchmarks.
1 Introduction
This paper examines whether LLMs are sensitive to the order of answer options in multiple-choice questions and how that sensitivity affects fair assessment. It identifies substantial order sensitivity, proposes positional bias as an explanation, and evaluates patterns and calibration approaches for improving robustness.
- LLMs’ sensitivity to prompt components motivates a broader analysis of factors that can affect fair assessment and real-world adoption.
- Reordering answer options can change GPT-4’s prediction on a CSQA question, illustrating sensitivity even when the question itself is unchanged.
- The study asks how strongly LLMs respond to option order, what contributes to that sensitivity, and how robustness can be improved.
- 75% sensitivity gap was observed in zero-shot evaluation across five multiple-choice benchmarks, while few-shot demonstrations yielded only marginal robustness improvements when performance increased.
- The authors conjecture that positional bias emerges when models are uncertain among top choices, and identify option placements that amplify or mitigate it.
- Two calibration approaches improved LLM performance by up to 8 percentage points, offering practical solutions beyond simple bootstrapping.
2 Background and Experimental Details
The experiments evaluate InstructGPT and GPT-4 on multiple-choice benchmarks spanning different domains and numbers of options. The setup uses standardized prompts, few-shot demonstrations, and benchmark-level comparisons of order sensitivity.
- The task asks models to select the correct answer from provided options using a standardized multiple-choice prompt.
- The evaluation uses InstructGPT and GPT-4 to assess how model size relates to robustness against option-order changes.
- Five benchmarks cover commonsense, abstract algebra, chemistry, professional law, and logical deduction tasks.
- Table 1 reports zero-shot order sensitivity for GPT-4 and InstructGPT across multiple benchmarks.
- The benchmark selection includes questions with 3, 4, and 5 options and varying model-performance levels.
3 Sensitivity to Order
The paper quantifies option-order sensitivity as the performance range induced by oracle reordering and examines it in zero-shot and few-shot settings. Sensitivity remains substantial despite demonstrations, although it can shrink as performance improves.
- The sensitivity gap is the difference between maximum and minimum performance under oracle orderings when the ground truth is known.
- 13.1% sensitivity gap remains even on tasks where GPT-4 exceeds 90% accuracy, showing that high accuracy does not eliminate order sensitivity.
- GPT-4 has a significantly lower sensitivity gap than InstructGPT, while poorly performing tasks do not necessarily have larger gaps.
- Domain and option count affect model performance, but neither shows a clear relationship with the sensitivity gap.
- Few-shot demonstrations are selected using Sentence-RoBERTa representations and Euclidean distance over question vectors.
- The sensitivity gap remains substantial with more demonstrations, shrinks as performance improves, and does not necessarily decrease when demonstrations are added.
4 Why Do LLMs Show Sensitivity to the Order of Options?
The paper attributes option-order sensitivity to uncertainty among top choices interacting with positional bias, then identifies option-placement patterns that amplify or mitigate this effect.
- Uncertainty Meets Positional Bias: LLM sensitivity arises from uncertainty among top possible answers interacting with positional bias tied to option order.The paper presents this as a conjecture and empirically investigates both factors.
- Uncertainty Meets Positional Bias: The sensitivity gap strongly correlates with error rate across GPT-4 and InstructGPT benchmarks.The sensitivity gap measures performance differences caused by option reordering.
- Uncertainty Meets Positional Bias: More than 60% of GPT-4-sensitive samples are also sensitive for InstructGPT, often involving semantically similar top choices.One example pairs “overpriced” and “expensive” as the top choices.
- Uncertainty Meets Positional Bias: Over 94% of sensitive cases received a “yes” response when models were asked whether multiple choices could be highly probable answers.This self-verification result supports uncertainty in instances where reordering changes predictions.
- What Patterns Amplify or Mitigate the Positional Bias?: The analysis limits pattern investigation to placements of the top-2 choices and defers patterns involving later options to future research.The limitation is stated as a scope boundary of the pattern analysis.
- What Patterns Amplify or Mitigate the Positional Bias?: For amplifying positional bias, placing the top two choices first and last is generally most effective; adjacent placements better mitigate bias.The paper reports that the preferred mitigating placements are often the first two or second and third options.
- What Patterns Amplify or Mitigate the Positional Bias?: Amplifying patterns cover 20%–72% of the initial sensitivity gap, while mitigating patterns cover 0.9%–38%.Higher coverage indicates better amplification, whereas lower coverage indicates better mitigation in this analysis.
5 Calibrating LLMs for MCQ Tasks
The paper evaluates calibration methods for reducing option-order sensitivity in MCQ tasks. Majority voting improves performance by up to 8 percentage points, whereas MEC produces inconsistent or detrimental outcomes.
- Calibration Methods: The study calibrates predictions using majority voting over outputs from 10 random option reorders and Multiple Evidence Calibration.MEC prompts models to generate an explanation before making a prediction.
- Majority Voting: Up to 8 percentage points of performance improvement result from majority voting across benchmarks.GPT-4 remains more stable than InstructGPT after calibration.
- Multiple Evidence Calibration: MEC consistently decreases InstructGPT performance and produces mixed results for GPT-4 across benchmarks.For GPT-4, improvement differs considerably from majority voting on some tasks, raising concerns about MEC reliability.
- Multiple Evidence Calibration: Asking models to explain their reasoning can amplify uncertainty, particularly when initial confidence is low.The paper partly attributes this behavior to hallucination.
6 Related Work
Related work studies LLM capability and robustness across multiple-choice question answering and broader prompt or interaction settings. Prior research has documented sensitivity to prompt choices, demonstration order, and sequential interaction histories.
- Sensitivity of LLMs: The paper situates its MCQ analysis within efforts to evaluate LLM dependability under subtle input changes.The related work emphasizes both capabilities and vulnerabilities of these models.
- LLMs and Multiple-Choice Questions: Multiple-choice questions are used to assess models’ reasoning and fact-retrieval capabilities.LLMs have achieved human-like performance across several MCQ benchmarks.
- Sensitivity of LLMs: Prior robustness research examines how modifications or adversarial attacks affect individual instances.This instance-level perspective includes prompt choice and demonstration-order sensitivity in in-context learning.
- Sensitivity of LLMs: LLMs have demonstrated sensitivity to prompt wording and the ordering of few-shot demonstrations.These findings motivate broader investigation of factors affecting LLM performance.
7 Discussion and Conclusion
The paper finds pronounced sensitivity to option order in multiple-choice questions, with only slight reduction in specific few-shot circumstances. It links this sensitivity to positional bias in uncertain instances, proposes placement patterns and calibration methods, and identifies boundaries for future work.
- LLMs exhibit pronounced sensitivity to multiple-choice option order, and demonstrations reduce this sensitivity only slightly under specific few-shot circumstances.The paper frames option order as a broader instance of sensitivity to multiple input fragments.
- The observed phenomenon extends beyond multiple-choice questions to tasks involving multiple fragments, including odd word detection, list sorting, and document ranking.These related tasks show varying degrees of sensitivity and are reserved for further exploration.
- The authors conjecture that positional bias particularly manifests in uncertain instances and identify patterns that magnify or mitigate it.Detailed experiments validate the conjecture, while the paper notes that understanding the issue’s origin more deeply requires examining training data.
- For calibration, the paper considers majority vote and multiple evidence calibration, both of which show promising performance improvements but have important limitations.Majority voting is computationally expensive, while MEC diverges significantly from majority voting and raises applicability concerns for multiple-choice tasks.
A Detailed Results
The detailed-results section directs readers to tables reporting few-shot order sensitivity for InstructGPT and GPT-4, plus sensitivity gaps under bias-amplifying and bias-mitigating option patterns.
- Tables 6 and 7 provide detailed few-shot order-sensitivity results for InstructGPT and GPT-4, respectively.
- The detailed results are presented as a reference for the few-shot order-sensitivity analysis.
- Table 8 reports sensitivity gaps after applying identified patterns to amplify and mitigate positional bias.