PaperlayerView exact revision on arXiv

Source-linked AI summary

SCORE: Systematic COnsistency and Robustness Evaluation for Large Language Models

Grigor Nalbandyan, Rima Shahbazyan, Evelina Bakhturina

arXiv:2503.00137v1cs.CL

TL;DR

Single-score evaluations leave a gap in systematic evidence about how LLM predictions vary across non-adversarial conditions. SCORE evaluates models across several such conditions and finds that accuracy and consistency are related but not interchangeable.

  • Problem

    LLM evaluations commonly report one accuracy metric from an optimized setup, while robustness analyses are scattered and difficult to compare.

    Source: §1 · p. 1

  • Method

    SCORE standardizes non-adversarial robustness evaluation across prompt variations, random seeds, and accuracy and consistency measures.

    Source: §1 · pp. 1–2, §3.2 · p. 3, §3.4 · p. 4

  • Results

    Accuracy and consistency are correlated, but higher accuracy or smaller accuracy ranges do not always guarantee better consistency, and model size alone is not a reliable robustness indicator.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Evaluating models under varied conditions and reporting accuracy ranges and consistency rates provides a more nuanced assessment than relying on a single-point evaluation.

    Source: §1 · p. 2, §5 · p. 6

    The datasets and robustness tests may not capture the breadth of LLM capabilities; the benchmark relies heavily on MCQ datasets and does not explicitly evaluate subjective creative tasks such as summarization.

    Source: §6 · p. 6

Abstract

from arXiv · show

Typical evaluations of Large Language Models (LLMs) report a single metric per dataset, often representing the model's best-case performance under carefully selected settings. Unfortunately, this approach overlooks model robustness and reliability in real-world applications. For instance, simple paraphrasing of prompts on the MMLU-Pro dataset causes accuracy fluctuations of up to 10\%, while reordering answer choices in the AGIEval dataset results in accuracy differences of up to 6.1\%. While some studies discuss issues with LLM robustness, there is no unified or centralized framework for evaluating the robustness of language models. To address this gap and consolidate existing research on model robustness, we present SCORE ($\mathbf{S}$ystematic $\mathbf{CO}$nsistency and $\mathbf{R}$obustness $\mathbf{E}$valuation), a comprehensive framework for non-adversarial evaluation of LLMs. The SCORE framework evaluates models by repeatedly testing them on the same benchmarks in various setups to give a realistic estimate of their accuracy and consistency. We release the code publicly and start an LLM robustness leaderboard to facilitate further development and research.

1 Introduction

Single optimized accuracy scores miss how responses vary across input changes, while robustness evaluations remain scattered and difficult to compare. SCORE is an open framework for systematically evaluating non-adversarial robustness alongside accuracy.

1 Introduction

  • Typical evaluations report one accuracy metric from an optimized setup, which provides an incomplete picture of capabilities and misses response consistency under input changes.Existing robustness evaluations are described as scattered, ad hoc, and difficult to compare between models.Source: §1 · p. 1
  • SCORE is an open framework that standardizes and unifies non-adversarial robustness evaluation while assessing consistency alongside accuracy.The paper investigates prompt variations, random seeds in non-greedy inference, and choice order, and argues that accuracy ranges and consistency rates provide a more accurate assessment than a single metric.Source: §1 · pp. 1–2
  • Accuracy and consistency are correlated, but higher accuracy or narrower accuracy ranges do not always guarantee better consistency, and model size alone is not a reliable indicator of robustness.Source: §1 · p. 2

2 Related Work

Prior evaluation efforts offer centralized comparisons, adversarial robustness tools, or broad multi-metric assessment, but each has limitations for evaluating robustness consistently.

2 Related Work

  • Open LLM Leaderboard-v2 centralizes consistent model evaluation and fair comparisons but relies on a single metric.Source: §2 · p. 2
  • PromptBench (Zhu et al., 2023a,b) evaluates worst-case accuracy under adversarial prompts, which may be unrealistic and substantially change input semantics.Source: §2 · p. 2
  • HELM (Liang et al., 2023) uses multiple metrics across scenarios, but its robustness analysis is limited to character-level perturbations, typos, and a small subset of Contrast Sets (Gardner et al., 2020).Source: §2 · p. 2

3 Benchmark

SCORE benchmarks open LLMs using selected factual datasets and three non-adversarial robustness tasks, reporting accuracy and prediction consistency across varied conditions.

3.1 Datasets

  • Dataset selection prioritizes factuality, topic diversity, scale, challenge for current open-source models, and minimal contamination; the benchmark uses MMLU-Pro, AGIEval, and MATH.These datasets do not fully encompass the range of real-world use cases.Source: §3.1 · p. 2

3.2 Tasks

  • Prompt robustness tests accuracy and consistency across ten semantically similar, non-adversarial prompts, including CoT and non-CoT formulations with the question in different positions.For MCQ datasets, prompts request the correct option letter; for MATH, they request a solution.Source: §3.2 · p. 2, Figure 2 · p. 3
  • Non-greedy robustness tests whether factual predictions remain consistent across five random seeds at temperature 0.7, using a fixed prompt to reduce computational cost.Source: §3.2 · p. 3, Figure 2 · p. 3
  • Choice-order robustness reorders options for MMLU-Pro and AGIEval while keeping the prompt and question fixed; generative evaluation also measures prediction consistency alongside accuracy.Fixing the correct answer to a particular letter may introduce evaluation bias, while helping identify preferences for answer options.Source: §3.2 · pp. 3–4, Figure 2 · p. 3

3.3 Models and Inference Setup

  • The evaluation uses instruction-tuned open models from several families and sizes, applying generative evaluation across tasks.The included models span Llama, Mistral, Qwen, and Yi families.Source: §3.3 · p. 4

3.4 Metrics

  • Accuracy is category-level macro accuracy for MMLU-Pro and micro accuracy for AGIEval and MATH, reported as a mean and minimum-to-maximum range.Source: §3.4 · p. 4
  • Consistency rate compares all pairs of predictions for each question, using exact answer agreement for MCQ and symbolic equivalence for MATH.For MATH, symbolic equivalence is evaluated with SymPy; consistency rate does not account for the correctness of individual predictions.Source: §3.4 · p. 4

4 Results

Across robustness tasks, accuracy and consistency are correlated but not interchangeable, and model size alone does not reliably predict either. Prompt wording, answer-choice order, and inference seeds produce different accuracy ranges and prediction consistency across models and datasets.

4.1 Prompt Robustness

  • Prompt robustness varies by dataset: excluding an outlier prompt, accuracy varies by 1.5%–10.6% on AGIEval and 1.3%–15.2% on MMLU-Pro, while MATH varies by 2%–7.9%.Across models, consistency is higher on AGIEval than MMLU-Pro and reaches a maximum of 69.8% on MATH.Source: §4.1 · pp. 4–5

4.2 Choice Order Robustness

  • Choice-order changes produce 4%–13.5% accuracy fluctuations on MMLU-Pro and 2%–7.5% on AGIEval, reaching 29.2% for Mistral 12B.On MMLU-Pro, Llama-3.1 405B is more accurate and consistent than Llama-3.1 70B, while similar consistency can accompany different mean accuracy across other model comparisons.Source: §4.2 · p. 5

4.3 Non-Greedy Inference

  • Non-greedy inference can leave accuracy nearly stable while consistency remains low, and harder MATH problems have fewer identical predictions across seeds.On MMLU-Pro, Llama-3.1 405B has 83.3% consistency but only 73.4% of predictions are identical across all seeds; identical predictions fall from 85% for Level 1 problems to 29.6% for Level 5.Source: §4.3 · p. 5

4.4 Aggregated Analysis

  • Model size alone does not reliably predict accuracy or consistency; across datasets, mean accuracy and consistency are correlated, but single-metric comparisons can mislead.Llama-3.1 405B has accuracy variance below 5% across datasets, while the aggregated analysis also finds model-size comparisons that do not align with accuracy and consistency.Source: §4.4 · p. 6

5 Conclusion

SCORE evaluates models under varied conditions and reports accuracy ranges and consistency rates, providing a more nuanced view than single-point evaluation. The framework supports standardized analysis of non-adversarial robustness.

5 Conclusion

  • SCORE evaluates models under varied conditions and reports accuracy ranges and consistency rates rather than relying on a single performance point.The paper presents this approach as a more nuanced and informative understanding of model performance.Source: §5 · p. 6
  • The framework establishes a foundation for standardized analysis and research of non-adversarial robustness.Source: §5 · p. 6
  • Single-point evaluation provides an incomplete assessment of LLM capabilities.Source: §5 · p. 6

6 Limitations

The authors note that their datasets and robustness tests may not cover the breadth of LLM capabilities. Their evaluation relies heavily on multiple-choice datasets and publicly available data, with broader evaluation carrying potential computational costs.

6 Limitations

  • The datasets and robustness tests may not capture the full breadth of LLM capabilities.Source: §6 · pp. 6–7
  • The evaluation relies heavily on multiple-choice datasets, which provide ease of evaluation and factual clarity, and does not explicitly cover subjective creative tasks such as summarization.Source: §6 · p. 6
  • Adding datasets and robustness tasks could provide a more complete picture, but may significantly increase computational costs.Source: §6 · p. 7
  • Reliance on publicly available datasets exposes the evaluation to potential data contamination risks.Source: §6 · p. 7

A Datasets Statistics

The appendices describe the benchmark datasets and inference setup used in SCORE, along with the scope of the prompt-robustness results table. The evaluation uses MATH, MMLU-Pro, and selected AGIEval subsets, with repeated model evaluations across datasets and conditions.

A Datasets Statistics

  • MATH comprises around 5,000 challenging competition-level mathematics problems requiring multiple reasoning steps.Its subset statistics are categorized by problem difficulty.Source: A Datasets Statistics · p. 8, Table 1 · p. 8
  • MMLU-Pro contains 12,032 questions across 14 subjects, while the selected AGIEval subsets comprise 2,340 datapoints.MMLU-Pro is described as a multiple-choice benchmark for knowledge and reasoning; AGIEval subsets are derived from standardized exams and test reading comprehension, reasoning, and mathematics.Source: A Datasets Statistics · p. 8

B Inference Setup

  • Evaluations use a 0-shot setting and generate up to 1,024 tokens per model and dataset; over 95% of datasets can be answered within that limit.The authors say further token generation yields diminishing metric returns while significantly increasing computational costs.Source: B Inference Setup · p. 9
  • For SCORE, each model is evaluated 25 times on MMLU-Pro, 19 times on AGIEval, and 15 times on MATH.The repeated evaluations cover prompt robustness, choice-order robustness where included, and non-greedy inference; the number of evaluations varies by dataset and task.Source: B Inference Setup · p. 9

C Prompt Robustness Results

  • The prompt-robustness table reports accuracy ranges and consistency rates across 10 prompts while keeping the context fixed.Source: Table 4 · p. 9

D Choice Order Robustness Results

The choice-order robustness table reports accuracy and consistency rates when answer-choice order changes while the prompt remains fixed.

D Choice Order Robustness Results

  • The table reports accuracy and consistency rates for changed answer-choice orders with the prompt held fixed.Source: Table 5 · p. 10

E Non Greedy Results

Table 6 covers accuracy ranges and consistency rates (CR) for Non-Greedy Robustness tasks, using five random seeds and temperature 0.7.

E Non Greedy Results

  • Table 6 reports accuracy ranges and consistency rates (CR) for Non-Greedy Robustness tasks.Source: Table 6 · p. 10
  • The evaluations use five random seeds with temperature set to 0.7.Source: Table 6 · p. 10
  • The table's stated scope is Non-Greedy Robustness evaluation.Source: Table 6 · p. 10

F Aggregated Results

The aggregated results table combines prompt robustness, choice-order robustness, and random-seed variation for non-greedy inference. The appendix provides prompt templates for MMLU-Pro, AGIEval, and MATH.

F Aggregated Results

  • Table 7 aggregates accuracy ranges and consistency rates across Prompt Robustness, Choice Order Robustness, and random-seed variation for non-greedy inference.Source: Table 7 · p. 11

G.1 MMLU-Pro Prompts

  • MMLU-Pro templates vary the multiple-choice instructions and answer formats, with one fixed prompt specified for Choice Order and Non-greedy Robustness tasks.Source: §G.1 · p. 12

G.2 AGIEval Prompts

  • AGIEval templates present multiple-choice instructions with answer choices A–E, including a fixed prompt for Choice Order and Non-greedy Robustness tasks.Source: §G.2 · p. 13

G.3 MATH Prompts

  • MATH templates ask for problem solutions and commonly require the final answer in a boxed format; the table identifies {question} as an input placeholder.Source: Table 10 · p. 14, §G.3 · p. 14

H Per Topic Analysis

Figure 6 compares accuracy ranges and consistency rates across dataset topics for Llama-3.1 405B. It reports differing patterns across MMLU-Pro, AGIEval, and MATH.

H Per Topic Analysis

  • As MATH question complexity increases, accuracy and consistency decline while variance grows.Source: Figure 6 · p. 15
  • On MMLU-Pro, accuracy varies between 3.8% and 10.9%, and consistency is not uniformly distributed across topics.Source: Figure 6 · p. 15
  • Across MMLU-Pro topics, some have the same consistency but differing accuracy, while others share accuracy but differ in consistency.Source: Figure 6 · p. 15

Related papers

Loading 2503.00137v1…