Source-linked AI summary
Benchmarking Prompt Sensitivity in Large Language Models
Amirhossein Razavi, Mina Soltangheis, Negar Arabzadeh, Sara Salamat, Morteza Zihayat, Ebrahim Bagheri
TL;DR
LLMs can respond differently to slight prompt variations, creating a gap in reliable prompt design and evaluation. The paper introduces Prompt Sensitivity Prediction and PromptSET, built from TriviaQA and HotpotQA variations, and benchmarks related methods. Existing baselines do not fully capture the task’s complexity, motivating further research into reliable prompt variation methods.
Problem
Prompt sensitivity makes it difficult to know whether slight prompt variations will still satisfy an information need, while existing methods do not provide a complete solution.
Method
The paper constructs PromptSET from TriviaQA and HotpotQA by generating semantically similar prompt variations, then benchmarks LLM-based, text-classification, and query-performance-prediction methods.
Results
Existing methods struggle to fully capture prompt sensitivity, with QPP specificity metrics performing weakest and LLM self-evaluation showing inconsistent performance across datasets.
Takeaways & Limitations
Reliable prompt evaluation requires methods tailored to how information needs are phrased across prompt variations.
Abstract
from arXiv · showhide
Large language Models (LLMs) are highly sensitive to variations in prompt formulation, which can significantly impact their ability to generate accurate responses. In this paper, we introduce a new task, Prompt Sensitivity Prediction, and a dataset PromptSET designed to investigate the effects of slight prompt variations on LLM performance. Using TriviaQA and HotpotQA datasets as the foundation of our work, we generate prompt variations and evaluate their effectiveness across multiple LLMs. We benchmark the prompt sensitivity prediction task employing state-of-the-art methods from related tasks, including LLM-based self-evaluation, text classification, and query performance prediction techniques. Our findings reveal that existing methods struggle to effectively address prompt sensitivity prediction, underscoring the need to understand how information needs should be phrased for accurate LLM responses.
1 Introduction
LLMs can be highly sensitive to small prompt changes, motivating systematic study of whether prompt variations preserve answerability. The paper introduces Prompt Sensitivity Prediction and PromptSET to benchmark this problem.
- Motivation: Small changes in prompt wording, structure, or punctuation can substantially alter an LLM’s output, including turning a correct answer into a failure.The examples involve TriviaQA and HotpotQA prompts answered correctly in their original form but not after slight modifications.
- Motivation: Prompt sensitivity describes the difficulty users face when crafting prompts that reliably satisfy their information needs.
- Background: Prompt engineering studies modifications such as structural changes, adversarial prompts, and different levels of specificity.
- Contribution: The paper introduces Prompt Sensitivity Prediction and PromptSET to predict whether an LLM can effectively respond to a prompt or its variation.
- Contribution: The benchmark draws on text classification, query performance prediction, and LLM-based methods to study the task’s complexity.
2 Methodology
The methodology defines prompt sensitivity prediction over semantically similar prompts sharing an information need, then constructs PromptSET from controlled variations and deterministic QA sources. It benchmarks text-classification, query-performance-prediction, and LLM-based approaches.
- Task Definition: Prompt Sensitivity Prediction asks whether an LLM will accurately satisfy the information need underlying a given prompt.
- Task Definition: Each prompt variation preserves the original information need and exceeds a predefined similarity threshold while differing only slightly from the original.
- Dataset Construction: The dataset pipeline selects prompts, generates semantically similar variations, filters misaligned or hallucinated variations, and collects LLM responses.
- Source Data: PromptSET uses TriviaQA and HotpotQA because they provide human-annotated answers and deterministic evaluation, including multi-hop and comparison questions in HotpotQA.
- Dataset Construction: 11,469 unique questions were retained after filtering, then split into 8,028 training and 3,441 testing questions.
- Baselines: The benchmark includes pre-retrieval QPP methods, embedding-specificity metrics, text classification, and LLM-based baselines.
3 Experiments and Findings
The experiments compare baselines for predicting prompt-variation answerability and examine how original-prompt correctness and prompt similarity relate to variation performance. Existing methods show uneven effectiveness, while closely resembling variations are more likely to receive correct answers and accurate predictions.
- Baseline Performance: Specificity-based QPP methods perform lowest on PromptSET, likely because original prompts and variations have similar specificity levels.The evaluated methods are CC, DC, IEF, and PageRank.
- Impact of original prompt correctness on variation answerability: Figure 1(a) counts correctly answered variations according to whether the original prompt was answered correctly or incorrectly.Figures 1(b) and 1(c) show distributions of correct and incorrect responses and predictions based on similarity to the original prompt.
- Baseline Performance: BERT-PE is competitive with text-classification methods on PromptSET-TriviaQA and significantly outperforms other baselines on PromptSET-HotpotQA.LLM self-evaluation performs reasonably on TriviaQA but inconsistently on HotpotQA.
- Impact of choice of LLM on variation answerability: Figure 2 shows the answerability distribution for variations of questions that both LLaMA and Mistral failed to answer correctly.The figure concerns test-set questions whose original prompts were answered incorrectly by both LLMs.
- Impact of prompt similarity: Prompt variations more similar to the original are more likely to produce correct responses and accurate answerability predictions.Similarity is measured using cosine similarity between MiniLM embeddings of prompt-variation pairs.
4 Concluding Remarks
The paper introduces Prompt Sensitivity Prediction and PromptSET to study how LLMs respond to prompt variations sharing the same information need. Results indicate that existing methods do not fully capture prompt sensitivity, motivating further research on reliable prompt generation.
- 4 Concluding Remarks: The paper introduces the Prompt Sensitivity Prediction task and PromptSET, built from TriviaQA and HotpotQA.The dataset contains variations of questions used to examine LLM sensitivity.
- 4 Concluding Remarks: Benchmarking results show that existing methods do not fully capture the complexities of prompt sensitivity.The conclusion identifies further research into prompt-variation sensitivity as necessary.
- 4 Concluding Remarks: The findings underscore the need for methods that help users generate more reliable prompts.