PaperlayerView exact revision on arXiv

Source-linked AI summary

POSIX: A Prompt Sensitivity Index For Large Language Models

Anwoy Chatterjee, H S V N S Kowndinya Renduchintala, Sumit Bhatia, Tanmoy Chakraborty

arXiv:2410.02185v2cs.CLcs.AIcs.LG

TL;DR

Prompt sensitivity matters for evaluating LLMs, but standard performance measures give it little attention. POSIX measures changes in response likelihood across intent-preserving prompt variants; the authors report different most-sensitive variation types across MCQ and open-ended tasks, and that few-shot examples often reduce sensitivity.

  • Problem

    Standard benchmarks and model reports focus mainly on task performance, leaving prompt sensitivity relatively unexamined despite users preferring less sensitive models when task performance is similar.

    Source: §1 · p. 1

  • Method

    POSIX measures prompt sensitivity through changes in response likelihood across intent-preserving prompt variants.

    Source: §3.3.4 · p. 5

  • Results

    Prompt-template changes have the greatest sensitivity for MCQ tasks, paraphrasing does so for open-ended tasks, and adding few-shot exemplars often reduces sensitivity.

    Source: §1 · p. 3, §5.4 · p. 8, §6 · p. 9

  • Takeaways & Limitations

    Models with lower prompt sensitivity are generally preferred even when their standard benchmark performance is similar.

    Source: §1 · p. 1

    POSIX requires O(MN^2) log-likelihood comparisons for M prompts and N variations per prompt.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Despite their remarkable capabilities, Large Language Models (LLMs) are found to be surprisingly sensitive to minor variations in prompts, often generating significantly divergent outputs in response to minor variations in the prompts, such as spelling errors, alteration of wording or the prompt template. However, while assessing the quality of an LLM, the focus often tends to be solely on its performance on downstream tasks, while very little to no attention is paid to prompt sensitivity. To fill this gap, we propose POSIX - a novel PrOmpt Sensitivity IndeX as a reliable measure of prompt sensitivity, thereby offering a more comprehensive evaluation of LLM performance. The key idea behind POSIX is to capture the relative change in loglikelihood of a given response upon replacing the corresponding prompt with a different intent-preserving prompt. We provide thorough empirical evidence demonstrating the efficacy of POSIX in capturing prompt sensitivity and subsequently use it to measure and thereby compare prompt sensitivity of various open-source LLMs. We find that merely increasing the parameter count or instruction tuning does not necessarily reduce prompt sensitivity whereas adding some few-shot exemplars, even just one, almost always leads to significant decrease in prompt sensitivity. We also find that alterations to prompt template lead to the highest sensitivity in the case of MCQ type tasks, whereas paraphrasing results in the highest sensitivity in open-ended generation tasks. The code for reproducing our results is open-sourced at https://github.com/kowndinya-renduchintala/POSIX.

1 Introduction

LLMs can respond differently to intent-preserving prompt changes, but standard evaluation often focuses on task performance rather than prompt sensitivity. The paper introduces POSIX to measure this sensitivity and compares how it varies across models and prompt changes.

1 Introduction

  • Prompt sensitivity is often overlooked by standard benchmarks and model reports, despite users preferring less sensitive models even when task performance is similar.Users may not formulate an optimal prompt, and a universally applicable optimal prompt may not exist.Source: §1 · p. 1
  • Existing sensitivity measures miss response diversity, response-frequency distributions, semantic similarity, or likelihood changes, and often exclude open-ended generation.Accuracy can treat consistently wrong answers and varying wrong answers alike, while exact-match scoring penalizes minor wording differences.Source: §1 · p. 2
  • POSIX measures sensitivity by comparing response likelihoods across intent-preserving prompt variants, without relying on the ground-truth response.The paper frames a prompt-insensitive model as one whose response probabilities remain nearly unchanged across such variants.Source: §1 · p. 2
  • The paper introduces POSIX as an index intended to incorporate four sensitivity factors and work across prompt variations, tasks, and arbitrary-length open-ended responses.The factors are response diversity, response-distribution entropy, semantic coherence, and variance in confidence.Source: §1 · p. 2
  • Prompt-template changes produce the greatest sensitivity in MCQ tasks, while wording changes do so in open-ended generation tasks.The paper also reports that increasing parameter count or instruction tuning does not necessarily decrease prompt sensitivity.Source: §1 · p. 3

2 Related Work

Prior work studies prompt sensitivity and robustness through perturbations, template variation, and prompt engineering, but these efforts often rely on performance-based measures. The paper argues for a comprehensive measure of sensitivity rather than performance alone.

Sensitivity of LLMs to Prompt Variations

  • Prior studies report sensitivity to changes in prompt structure and wording, as well as to example selection, ordering, labels, and instruction phrasing.The section also describes work evaluating robustness to adversarial prompt variations using performance drop rate.Source: §2 · p. 3
  • Existing evaluations include prompt perturbations, template-based performance spread, and variation-ratio measures as sensitivity surrogates.Many benchmarks report performance for a single prompt template, while LMentry uses three templates per task and averages performance over them.Source: §2 · p. 3
  • The paper advocates benchmarks with multiple variants per prompt and a comprehensive measure that captures sensitivity beyond performance alone.Source: §2 · p. 3

Prompt Engineering

  • Prompt engineering and meta-prompting seek prompts that elicit desired responses, while Chain-of-Thought prompting has been observed to help reasoning tasks.The section also notes that in-context example selection matters, with semantically similar or diverse examples beneficial in different cases.Source: §2 · p. 3

3 POSIX: Prompt Sensitivity Index

POSIX measures how response likelihoods change across intent-aligned prompts and aggregates sensitivity across prompt sets. Its design connects response diversity, response-frequency entropy, semantic coherence, and likelihood variation to sensitivity.

3.1 Preliminaries

  • Intent-aligned prompts vary in wording, template, or minor spelling while retaining the same underlying goal, intent, or meaning.A prompt set is intent-aligned when every pair of prompts in it meets this criterion.Source: §3.1 · p. 4

3.2 Defining the Prompt Sensitivity Index

  • POSIX aggregates model sensitivity across a dataset of intent-aligned prompt sets.The dataset-level definition is introduced after defining sensitivity for a model on one prompt set.Source: §3.2 · p. 4

3.3 What does ψM,X Capture?

  • For each prompt set, ψ compares response likelihoods under the original and alternate prompts using relative changes, absolute logarithms, and response-length normalization.The likelihood ratios are used to remove dependence on the overall scale of a model’s probability distributions.Source: §3.3 · p. 4, §3.2 · p. 4

3.3.1 POSIX and Response Diversity

  • Greater response diversity can raise ψ because substantially different responses tend to have substantially different likelihoods under a prompt.Source: §3.3.1 · p. 4

3.3.2 POSIX and Response Distribution Entropy

  • Higher entropy in response frequencies tends to raise ψ because divergent responses occur more often.Source: §3.3.2 · p. 4

3.3.3 POSIX and Semantic Coherence

  • Higher semantic similarity between responses tends to lower ψ when similar responses have similar likelihoods across intent-aligned prompts.Semantically equivalent responses can still have different likelihoods, even when they are identical strings, so cosine similarity alone is insufficient as a sensitivity measure.Source: §3.3.3 · pp. 4–5

3.3 What does ψM,X Capture?

  • Figure 1 presents MMLU correlation plots relating ψ to response diversity, response-distribution entropy, semantic coherence, and variance in confidence.Source: Figure 1 · p. 5

3.3.4 POSIX and Variance in Confidence

  • ψ can capture sensitivity even when all prompts produce the same response, if that response’s log-likelihood varies across prompts.Source: §3.3.4 · p. 5

4 Experimental Setup

The experiments evaluate POSIX on classification and open-ended generation across eight LLMs, using intent-aligned prompt variants that vary spelling, template, or wording. They compare sensitivity across these variation types and a mixed set.

4 Experimental Setup

  • Experiments use MMLU for multiple-choice classification and 5,000 sampled Alpaca questions for open-ended generation.MMLU contains about 14,000 prompt-response pairs across 57 domains.Source: §4 · p. 5
  • The study evaluates eight models from the LLaMA, Mistral, and OLMo families, including base and chat or instruct variants.Experiments were run on eight NVIDIA A100-SXM4-80GB GPUs.Source: §4 · pp. 5–6
  • Each original prompt has 60 intent-aligned variants, equally divided among spelling errors, template changes, and paraphrases.Spelling variants use token-level edits; templates preserve core meaning while changing structure; GPT-3.5-Turbo generates paraphrases preserving intent and meaning.Source: §4 · p. 6
  • POSIX is computed separately for each variation type from the original prompt and its 20 variants, and for a mixture from 21 uniformly sampled variants.The mixture contains equal proportions of the three variation categories.Source: §4 · p. 6, Table 1 · p. 6
  • Experiments generate five tokens for MMLU and 30 for Alpaca; POSIX is normalized by response length and is described as comparable across lengths.Source: §4 · p. 6

5 Results and Analysis

POSIX tracks prompt sensitivity across response diversity, response distributions, semantic coherence, and confidence, and the results show that sensitivity does not reliably fall with model scale or instruction tuning. Few-shot exemplars reduce sensitivity, while the most sensitive prompt variation differs between multiple-choice and open-ended tasks.

5.1 Evaluating the Efficacy of POSIX

  • POSIX varies positively with unique-response count, response-distribution entropy, and log-probability variance, and negatively with average response cosine similarity.The authors present these correlations as empirical validation that POSIX incorporates the four proposed factors.Source: §5.1 · p. 7

5.2 Effect of Instruction Tuning on Sensitivity

  • Instruction-tuned models are generally less sensitive for MMLU template variations, but tend to be more sensitive for other variation categories, especially on Alpaca.The comparison of FLAN-only and chat models favors chat in most cases, except for MMLU prompt templates, where FLAN-only models perform better in sensitivity; the authors hypothesize that FLAN's emphasis on MCQs and prompt templates may contribute.Source: §5.2 · pp. 7–8, Table 3 · p. 8

5.4 Few-shot Exemplars and Sensitivity

  • A single few-shot exemplar significantly reduces prompt sensitivity on MMLU, while adding a second or third yields smaller gains or little further change.The comparison is reported for Llama-2 and Mistral models in few-shot settings.Source: Table 2 · p. 8, §5.4 · p. 8

5.3 Impact of Model Scale on Sensitivity

  • OLMo-7B is more sensitive than OLMo-1B on MMLU but comparable on Alpaca, while Llama-2-13B is not guaranteed less sensitive than 7B.These results reinforce that accuracy and prompt sensitivity are distinct aspects of model behavior.Source: Figure 3 · p. 8, §5.3 · p. 8, Figure 4 · p. 9

5.5 Impact of Various Variation Categories

  • Prompt templates are the most sensitive variation for MMLU multiple-choice tasks, while paraphrases are usually most sensitive for Alpaca open-ended generation.The exception is OLMo, for which paraphrases are not the most sensitive variation; the authors suggest focusing prompt engineering on templates for MCQs and query phrasing for open-ended questions.Source: §5.5 · pp. 8–9

6 Conclusions

POSIX measures LLM sensitivity to intent-preserving prompt variations and enables comparisons across variation types and tasks. The paper also identifies computational complexity as a limitation and reports removing MMLU prompts flagged during paraphrasing.

6 Conclusions

  • POSIX measures sensitivity to intent-preserving prompt variations, and comparisons find template changes most sensitive for MCQ tasks and paraphrasing most sensitive for open-ended generation.The authors also report that parameter count and instruction tuning do not necessarily decrease sensitivity.Source: §6 · p. 9

Limitations

  • POSIX requires O(MN^2) log-likelihood comparisons for M prompts and N variations, making computational complexity a main limitation.The authors state that the method can handle different prompt variations and tasks, including open-ended generation with arbitrary response lengths.Source: Limitations · p. 9

Ethical Considerations

  • Several MMLU prompts flagged as violent or biased during GPT-3.5-Turbo paraphrasing, mostly from moral_scenarios, were removed from the analyses.Source: Ethical Considerations · p. 9

A Code and Data

The paper releases its prompt-variant generation and sensitivity-computation code and documents the templates used for MMLU and Alpaca experiments.

A Code and Data

  • The code for generating prompt variants and computing POSIX is open-sourced, using HuggingFace for inference and PyTorch as the underlying framework.The passage identifies PyTorch's BSD license and HuggingFace's Apache 2.0 license.Source: A Code and Data · p. 12

B Prompt Templates

C Sensitivity of OLMo-1B

The appendix reports OLMo-1B POSIX values for MMLU and Alpaca across prompt-variation types.

D Sensitivity of Llama-2-13B

The appendix reports POSIX values for Llama-2-13B base and chat variants on MMLU and Alpaca.

E Sensitivity on the BBH dataset

The BBH analysis compares model sensitivity and model scale under prompt-template variation, while examples illustrate generated paraphrases and POSIX is evaluated against four sensitivity factors for open-ended Alpaca prompts.

E Sensitivity on the BBH dataset

F Examples of generated paraphrases

G Efficacy of POSIX for Open-ended Generation

  • For Alpaca, Figure 7 relates POSIX to four factors as evidence of its effectiveness in capturing prompt-sensitivity nuances.The factors include response diversity, response-distribution entropy, semantic coherence, and variance in confidence.Source: G Efficacy of POSIX for Open-ended… · p. 14

H Distribution of POSIX Values for All Variation Types

The section presents POSIX distributions across variation types for MMLU and Alpaca and separately plots its correlations with four factors for Alpaca.

H Distribution of POSIX Values for All Variation Types

Related papers

Loading 2410.02185v2…