Source-linked AI summary
State of What Art? A Call for Multi-Prompt LLM Evaluation
Moran Mizrahi, Guy Kaplan, Dan Malkin, Rotem Dror, Dafna Shahaf, Gabriel Stanovsky
TL;DR
Single-template benchmarks may give unstable comparisons because LLM performance depends on instruction phrasing. This paper evaluates paraphrased prompts at large scale, proposes use-case-specific multi-prompt metrics, and finds substantial changes in performance and rankings relative to traditional evaluations.
Problem
Benchmarks commonly use one instruction template per task, although LLM performance can vary with semantically equivalent prompt paraphrases.
Method
The paper evaluates models across diverse instruction paraphrases and defines aggregated metrics suited to robustness assessment or top-performing downstream prompts.
Results
The analysis finds that prompt paraphrases substantially change absolute performance and relative rankings, differing from traditional benchmark results.
Takeaways & Limitations
LLM evaluations should use multiple prompts and select aggregation metrics according to the evaluator’s practical use case.
Takeaways & Limitations
The main evaluation excludes closed API models because scale is expensive and their APIs may manipulate input prompts in undisclosed ways.
Abstract
from arXiv · showhide
Recent advances in large language models (LLMs) have led to the development of various evaluation benchmarks. These benchmarks typically rely on a single instruction template for evaluating all LLMs on a specific task. In this paper, we comprehensively analyze the brittleness of results obtained via single-prompt evaluations across 6.5M instances, involving 20 different LLMs and 39 tasks from 3 benchmarks. To improve robustness of the analysis, we propose to evaluate LLMs with a set of diverse prompts instead. We discuss tailored evaluation metrics for specific use cases (e.g., LLM developers vs. developers interested in a specific downstream task), ensuring a more reliable and meaningful assessment of LLM capabilities. We then implement these criteria and conduct evaluations of multiple models, providing insights into the true strengths and limitations of current LLMs.
1 Introduction
Single-prompt evaluations can be brittle because semantically equivalent instruction paraphrases produce different absolute performance and model rankings. The paper proposes multi-prompt evaluation with metrics tailored to evaluators’ use cases and analyzes it at scale.
- LLM performance depends on the chosen instruction template, even when templates describe the same task.
- 6.5M instances across 20 LLMs, 39 tasks, and 3 benchmarks were analyzed to measure prompt-sensitive evaluation results.
- Semantically equivalent prompts can change both absolute performance and relative model rankings, sometimes making one model best under one instruction and worst under another.The example contrasts GPT-3.5-Turbo’s performance on prompts P1 and P4.
- The paper proposes multi-prompt metrics that aggregate performance across paraphrased instruction templates instead of relying on a single prompt.
- Different use cases call for different aggregation criteria: developers may assess robustness across prompts, whereas downstream integrators may compare models by their top-performing prompt.
- The results indicate that LLaMA-based models can match T5-based models under top-performing instructions but lag when average performance across paraphrases is considered.The gap is attributed to poor performance on many paraphrases.
2 Background and Definitions
LLM benchmarks commonly evaluate every model using one task instruction template, although tasks can be phrased in many ways. The paper distinguishes this issue from adversarial prompt-robustness research and studies both absolute and relative performance across prompt choices.
- An instruction template is a string with placeholders into which input samples are inserted; the same task can have multiple templates.
- MMLU, BIG-bench, and HELM use a single instruction template per task, while LMENTRY reports average performance over three templates.
- Many notable model papers do not disclose evaluation templates, which exacerbates meaningful comparative evaluation despite possible leakage concerns.
- Prior prompt-robustness studies typically test adversarial or perturbed instructions against an original benchmark template treated as a gold-standard reference.
- This work instead analyzes how prompt choice affects both absolute and relative model performance across many models and metrics.
3 Experimental Setup
The experiments cover 39 tasks from three benchmarks, use benchmark-specific evaluation procedures, and assess instruction-tuned models from diverse families. Closed API models are excluded from the main evaluation because of cost and possible prompt manipulation.
- 39 tasks are drawn from LMENTRY, BIG-bench Lite, and BIG-bench Hard.The setup includes 10 LMENTRY tasks, 14 BIG-bench Lite tasks, and 15 BIG-bench Hard tasks.
- LMENTRY contributes 10 challenging linguistic tasks, each accompanied by three instruction templates.
- BIG-bench Lite contributes 14 knowledge-domain tasks, while BIG-bench Hard contributes 15 challenging classification and multiple-choice tasks.
- Performance is measured with LMENTRY’s official evaluation script and exact string matching for Big-Bench tasks.The authors describe exact matching as strict, fair, and straightforward.
- The study evaluates 16 instruction-tuned LLMs from 11 diverse model families.
- Closed API models are omitted from the main evaluation because large-scale use is expensive and APIs may manipulate prompts in undisclosed ways.A small-scale OpenAI evaluation is still conducted to test sensitivity to prompt paraphrasing.
4 Evaluating against a Single Prompt Leads to Instability in Results
The study shows that single-prompt evaluation is brittle: valid paraphrases can substantially change model performance and rankings. Using automatically generated and manually validated paraphrases, the authors quantify this instability across tasks and models.
- 4.1 Paraphrasing Instruction Templates: More than 200 instruction paraphrases were automatically generated per task using three prompting methods, then manually verified and filtered.The methods included instruction-template rephrasing, Chain-of-Thought prompting, and gradual template generation.
- 4.1 Paraphrasing Instruction Templates: More than 85% of 375 doubly annotated instructions were found correct by both annotators, with generally high agreement.The reported agreement measures varied, partly because of pronounced label imbalance.
- 4.1 Paraphrasing Instruction Templates: 90% of LMENTRY paraphrases and roughly 84% of BBH paraphrases were judged correct.Validation yielded 240 validated paraphrases per LMENTRY task and 175 per BBH task on average.
- 4.2 Quantifying Performance Variance due to Instruction Paraphrasing: 10 tasks showed only slight to moderate ranking agreement, while only two showed strong agreement; Friedman tests found significant performance differences on 21 of 25 tasks.The analysis uses Kendall’s W for agreement among instruction-induced rankings and a Friedman test for differences across prompts.
- 4.2 Quantifying Performance Variance due to Instruction Paraphrasing: T0pp ranked first under one BBH paraphrase but ninth under another, while Alpaca-13B and Alpaca-7B moved from last under P1 to top-performing under P2 on LMENTRY.These paraphrases were valid, yet produced vastly different rankings.
- 4.2 Quantifying Performance Variance due to Instruction Paraphrasing: 15 of 25 tasks had paraphrase pairs with negative Kendall’s τ, indicating mostly disagreeing model rankings.Kendall’s τ measures agreement between two rankings, with negative values indicating that most model pairs are discordant.
- 4.2 Quantifying Performance Variance due to Instruction Paraphrasing: Changing “excludes” to “lacks” degraded Flan-T5-large performance by 28% on average but improved Flan-T5-XL performance by 46%.The result demonstrates that substantial performance differences can arise even for very similar paraphrase edits.
5 Different Use Cases Merit Different Metrics
Because performance varies with instruction paraphrases, evaluation should use metrics matched to the evaluation purpose rather than a single catch-all measure. The paper defines metrics for peak performance, average robustness, and their combination.
- Metric choice by use case: Evaluation metrics should depend on the purpose of the evaluation rather than use one coarse catch-all metric.The paper introduces metrics tailored to specific scenarios and realistic user needs.
- Maximum Performance for Particular Downstream Applications: MaxP is the highest performance a model achieves across all instruction templates for a task.It is intended for developers selecting a prompt for a specific downstream task and domain; held-out samples can validate the selected prompt.
- Average Performance for LLM Developers: AvgP is the mean performance across all instruction templates for a task.It is useful for assessing robustness to paraphrases and mitigating performance outliers.
- Combined Performance Score: CPS combines maximum and average performance to capture both peak capability and robustness across prompts.It multiplies MaxP by a saturation score measuring how closely best and average performance align.
- Combined Performance Score: CPS is intended for selecting models that must perform effectively and robustly across diverse tasks and user-visible prompts.The paper gives multi-functional chatbots as an example of this use case.
6 Multi-Prompt Evaluation
Multi-prompt evaluation reveals that model performance and rankings can change substantially across instruction paraphrases and evaluation metrics. The proposed metrics remain stable despite automatically generated paraphrase noise, except for one task affected by an evaluation-script error.
- Evaluation results: For the vast majority of tasks, the top three models under original instructions differed from those ranked first by average and maximum performance.Aggregate metrics diverged from performance on the original instruction templates.
- Evaluation results: Model rankings depended on the metric: Falcon-Instruct-7b and Vicuna-13b ranked first by MaxP on LMENTRY rhyming, but their AvgP values were 0.17 and 0.15.Across LMENTRY, LLaMA-based models were competitive with T5-based models by MaxP but tended to lag by AvgP because of poor performance on many paraphrases.
- Robustness to paraphrase noise: Automatic paraphrase-generation noise had virtually no impact on metric-based model rankings after manual filtering.Kendall’s τ showed near-perfect to perfect agreement across tasks except LMENTRY’s “ends with word” task.
- Robustness to paraphrase noise: The exception in LMENTRY’s “ends with word” task appeared mostly attributable to an error in its evaluation script.The results suggest that metrics may be computed over automatically generated paraphrases without manual verification.
7 Small-Scale Evaluation of OpenAI Models on Prompt Paraphrasing
OpenAI models are sensitive to prompt paraphrasing, with substantial differences between original instructions and paraphrase-based average and maximum performance. These differences also change model rankings, motivating multi-prompt evaluation.
- 7 Small-Scale Evaluation of OpenAI Models on Prompt Paraphrasing: Four OpenAI models were evaluated on LMENTRY tasks using original prompts and paraphrases, with average and maximum performance metrics.The evaluation was conducted at small scale because of budget constraints.
- 7 Small-Scale Evaluation of OpenAI Models on Prompt Paraphrasing: Average performance was estimated by randomly selecting one paraphrase per task sample and repeating the evaluation 20 times.Each repetition scored the full set of task samples.
- 7 Small-Scale Evaluation of OpenAI Models on Prompt Paraphrasing: Maximum performance was estimated with a greedy search over roughly 175 instruction templates per task.The search narrowed candidates across successive groups of task instances before selecting the best-performing template.
- 7.1 Results: Figure 7 compares original-prompt maximum performance with estimated maximum paraphrase performance across LMENTRY tasks.Original performances are shown in solid colors and paraphrase maxima in semi-transparent columns.
- 7.1 Results: Minor phrasing changes produced drastic performance changes for OpenAI models, including nearly identical instruction pairs with notable differences.Table 7 presents representative minimal-distance pairs from LMENTRY.
- 7.1 Results: 72.5% of cases had higher original-prompt performance than estimated paraphrase-average performance.For davinci, original prompts added an average of 21 accuracy points.
- 7.1 Results: Ranking agreement with original instruction templates occurred in only 5 of 10 tasks for the average metric and 4 of 10 for the maximum metric.Figure 6 fixes model order by original-template performance to make ranking changes directly comparable across metrics.
8 Related Work
Related work studies challenges in meaningful and reproducible LLM evaluation, including test-data selection and sensitivity to prompt formatting or in-context-example format. This paper differs by varying task instructions rather than the underlying instances.
- Related work: Prior work addresses the rising cost of exhaustive LLM evaluation by selecting test-data subsets expected to represent the full dataset.The paper identifies extending this approach to instruction templates as a possible efficiency direction.
- Related work: Sclar et al. show that minor prompt-formatting choices, such as punctuation, can affect LLM behavior while preserving prompt meaning.The paper reports a similar phenomenon when automatic paraphrasing incidentally changes formatting.
- Related work: Voronov et al. study sensitivity to the format of in-context examples and its interaction with instruction phrasing, demonstration count, and model size.Their experiments vary how input-output pairs are separated.
- Related work: These studies belong to model-robustness work, but they differ from distribution-shift or adversarial-example settings because the task instruction changes while the classified instance remains fixed.The distinction separates instruction sensitivity from changes to the underlying instance.
9 Conclusions
The paper argues that single-prompt evaluation is inadequate because LLM performance is sensitive to prompt paraphrasing. It proposes diverse-template metrics whose interpretation depends on whether robustness or downstream peak performance is the goal.
- Conclusions: The paper proposes multi-prompt metrics that evaluate performance across diverse instruction templates instead of relying on a single prompt.The proposed metrics are intended to support more robust and meaningful LLM evaluation.
- Conclusions: LLM developers may measure robustness using average performance across many prompts, whereas downstream developers may compare models by their top-performing prompt.The recommended metric depends on the intended use case.
- Conclusions: These metrics reveal differences in absolute performance and relative model rankings compared with traditional evaluations.The paper connects this comparison to consistency and comparability in evaluation.