On Robustness and Reliability of Benchmark-Based Evaluation of LLMs

1 Introduction

• Fixed benchmark wording does not capture the linguistic variability of real-world interactions, where users may express the same intent in different ways.Source: §1 · p. 1
• The open question is whether benchmark evaluations remain reliable under systematic rewording, given prior reports of LLM sensitivity to minor wording changes [35] [47].The introduction asks whether scores reflect reasoning ability or performance on a particular wording of a task.Source: §1 · p. 1
• The study systematically paraphrases benchmark questions to test model robustness and whether relative rankings remain stable across rewordings.It frames this as a test of generalization beyond specific phrasings seen during training.Source: §1 · p. 1
• Absolute performance drops significantly on paraphrased questions, while model rankings remain relatively stable.The authors say static-wording benchmarks may overestimate models’ generalization abilities.Source: §1 · p. 1

2 Background and Related Work

• IR evaluation research has studied efficient, reliable comparisons, including ranking stability under incomplete or noisy assessments.Prior work also warns that benchmarks may lose discriminative power as systems overfit narrow interpretations of relevance [29].Source: §2 · p. 2
• LLM benchmarks support reproducible comparisons but use fixed wording, unlike real-world interactions where the same question may be phrased in varied ways [37].Benchmark quality also varies, with reported concerns including poor design, limited replicability, and weak statistical rigor.Source: §2 · p. 2
• Minor prompt changes can shift LLM performance, and prompt optimization does not itself establish robustness [35] [47].Prior work links high scores in fixed conditions to possible reliance on surface patterns or dataset artifacts.Source: §2 · p. 2
• Benchmark Agreement Testing studies ranking consistency across benchmarks, but stability under paraphrasing of the questions themselves has received less attention.The paper identifies systematic question paraphrasing as an open issue for evaluating robustness and benchmark reliability.Source: §2 · p. 2
• LLM judges offer scalable evaluation but may favor similar architectures or training distributions, while human evaluation is costly and difficult to scale.These constraints help explain continued reliance on structured benchmarks despite their limitations [18] [48] [50].Source: §2 · p. 2

3 Methodology

• The methodology describes the benchmarks, evaluated models, paraphrase generation, and prompts used in the experiments.Source: §3 · p. 2

3.1 Benchmarks

• Six established benchmarks are used; the full test splits and benchmark-size-dependent samples of training instances are paraphrased and evaluated uniformly.Each question has four choices; 13 ARC-C questions with three or five options were excluded, and several other datasets were omitted for format, task, or evaluation constraints.Source: §3.1 · pp. 2–3

3.2 Large Language Models

• The study evaluates 34 LLMs spanning model sizes, architectures, and training paradigms, with Table 2 reporting four-class accuracy across the six benchmarks.The set includes compact and large-scale models as well as a closed-source ChatGPT reference model.Source: §3.2 · p. 3, Table 2 · p. 3

3.3 Paraphrases Generation

• Questions are automatically paraphrased while preserving intended meaning, reasoning requirements, and answer-choice order; five alternatives are generated per question.The paraphrases are validated for semantic fidelity, and 31 instances were excluded after the generator refused to paraphrase them.Source: §3.3 · pp. 3–4

3.4 Model Prompting

• All models receive a standardized multiple-choice prompt with constrained numerical answers, and predictions use deterministic top-1 token probability.The evaluation is zero-shot; additional Chain-of-Thought tests had typically below 3% average accuracy improvement and about a 7% consistency drop, so the main analysis focuses on zero-shot prompting.Source: §3.4 · p. 4

4.1.1 Consistency of Models Across Paraphrases

• Across models, about 70%–85% of questions receive one consistent answer, while 15%–30% receive two or more distinct answers across paraphrases.Only a minority of models maintain the same answer across all paraphrases; three- and four-answer cases are less common.Source: §4.1.1 · pp. 4–5
• Consistency is lower on OpenBookQA and especially RACE; the authors suggest greater question complexity and ambiguity may contribute.Some models are highly consistent, often at the expense of low accuracy, and even the most robust models are not perfectly consistent.Source: §4.1.1 · p. 5

4.1.2 Model Accuracy, and Relationship with Consistency

• Accuracy and consistency correlate negatively for 0–15B models (ρ = −0.51) but positively for 16–150B models (ρ = 0.79), so consistency is not always a proxy for correctness.For less capable models, repeated answers may be wrong; for larger models, consistency more meaningfully indicates robustness.Source: §4.1.2 · p. 5
• A single average accuracy on original questions may not capture semantic understanding or robustness to surface-level wording changes.The authors warn that original-formulation benchmarks may overestimate real-world robustness.Source: §4.1.2 · p. 5

4.1.3 Evaluation of Generated Paraphrases

• As more paraphrases are added, distinct answers increase; nearly identical plots with reverse ordering support the paraphrases’ semantic consistency.Only three benchmarks are plotted because the others showed similar trends.Source: §4.1.3 · p. 6

4.2 RQ2: Are LLMs Robust to Question Paraphrases?

• Across benchmarks, most models score worse on paraphrased inputs, yet Kendall’s τ between original- and paraphrase-based rankings exceeds 0.9 (p < 0.01).The comparison uses median paraphrased performance; Figure 6 shows original scores and sampled paraphrase distributions.Source: §4.2 · pp. 6–7, Figure 6 · p. 7
• Over-category counts trend downward with benchmark release date (ρ = −0.33, not significant), while In (ρ = 0.28) and Under (ρ = 0.48) trend upward.The authors conjecture that models may overfit older benchmarks, potentially through pretraining data contamination; Under-category counts are very low.Source: §4.2 · p. 7

• Model rankings remain relatively stable under paraphrasing, but absolute accuracy drops significantly, suggesting static-wording evaluations overestimate generalization.Source: §5 · p. 7
• Consistency is not a reliable correctness proxy: less capable models may repeat wrong answers, while stronger models tend to be consistent and accurate.The authors caution against relying solely on average benchmark scores as a proxy for reasoning ability.Source: §5 · p. 7
• Older benchmarks show evidence of potential data contamination, raising memorization concerns and emphasizing evaluation on fresh, unseen, linguistically diverse inputs.Source: §5 · p. 7
• The study covers only multiple-choice benchmarks, and automatically generated paraphrases may not capture the full range of real-world linguistic variation.Source: §5 · p. 7
• Future work will further validate paraphrases, explore more naturalistic generation, extend analysis to complex reasoning, and release paraphrase-augmented benchmark suites.Source: §5 · p. 7