Source-linked AI summary
Calibrate Before Use: Improving Few-Shot Performance of Language Models
Tony Z. Zhao, Eric Wallace, Shi Feng, Dan Klein, Sameer Singh
TL;DR
Few-shot performance can vary sharply with prompt format, examples, and ordering, limiting the reliability of manually designed prompts. The paper diagnoses answer biases and calibrates them using a content-free input, improving accuracy and reducing prompt-related variance across tasks. Calibration mitigates rather than eliminates the need for prompt engineering.
Problem
GPT-3’s few-shot accuracy is unstable across prompt formats, training examples, and example orderings, so manually designed prompts cannot reliably deliver good accuracy.
Method
Contextual calibration estimates answer bias from predictions on a content-free test input and fits parameters making that input’s answer scores uniform.
Results
Up to 30.0% absolute accuracy gains were reported across GPT-3 and GPT-2 tasks, alongside reduced variance across prompt choices.
Takeaways & Limitations
Contextual calibration makes few-shot language models more stable and helps users obtain higher accuracy with less prompt-engineering effort.
Takeaways & Limitations
Calibration mitigates but does not eliminate prompt engineering, and generation-task calibration is limited to the first output token.
Abstract
from arXiv · showhide
GPT-3 can perform numerous tasks when provided a natural language prompt that contains a few training examples. We show that this type of few-shot learning can be unstable: the choice of prompt format, training examples, and even the order of the training examples can cause accuracy to vary from near chance to near state-of-the-art. We demonstrate that this instability arises from the bias of language models towards predicting certain answers, e.g., those that are placed near the end of the prompt or are common in the pre-training data. To mitigate this, we first estimate the model's bias towards each answer by asking for its prediction when given the training prompt and a content-free test input such as "N/A". We then fit calibration parameters that cause the prediction for this input to be uniform across answers. On a diverse set of tasks, this contextual calibration procedure substantially improves GPT-3 and GPT-2's average accuracy (up to 30.0% absolute) and reduces variance across different choices of the prompt.
1. Introduction
Few-shot in-context learning offers practical advantages because language models can perform tasks without parameter updates, but GPT-3’s accuracy is highly unstable across prompt choices. The paper attributes this instability to answer biases and proposes contextual calibration, which improves accuracy and stability.
- Motivation: Few-shot learning lets language models perform tasks from limited examples without finetuning or parameter updates.It also enables rapid prototyping, natural-language model interfaces, and reduced serving complexity across tasks.
- Problem: Changing the prompt format, training examples, or example order can shift GPT-3 accuracy from 54% to 93%.Users who design prompts manually therefore cannot consistently expect good accuracy.
- Analysis: Majority-label, recency, and common-token biases make the model favor answers frequent in the prompt, near its end, or common in pre-training data.For example, a prompt ending with a Negative example can bias predictions toward Negative, while “United States” may be preferred over “Saint Lucia.”
- Method: Contextual calibration estimates answer bias using a content-free input and fits parameters that make its scores uniform across answers.Using “N/A” as the test input produced 62% Positive before calibration in the example.
- Results: Up to 30.0% absolute accuracy gains and greater prompt stability were observed for GPT-3 and GPT-2 across diverse tasks.The procedure reduces the need for prompt engineering and helps end users obtain higher accuracy with less effort.
2. Background and Experimental Setup
The paper evaluates in-context prediction across classification, fact retrieval, and information extraction using varied prompt components and GPT-family model sizes. Prompts combine a format, training examples, and their ordering, with task-specific label or generation procedures.
- Prompt design: Each prompt consists of a format, a set of training examples, and a permutation of those examples.The permutation matters because autoregressive language models update hidden states left to right.
- Prompt design: A prompt format is a template containing placeholders for training and test examples and possibly a natural-language task description.Alternative formats can frame the same task differently, including as question answering.
- Evaluation: For classification, class probabilities come from the probabilities of associated label names, while generation proceeds greedily until a newline.Information extraction is scored by Exact Match between generated output and the ground-truth span.
- Datasets: The evaluation covers text classification, fact retrieval, and information extraction datasets.Classification includes SST-2, TREC, CB, RTE, AGNews, and DBPedia; fact retrieval uses LAMA; extraction uses ATIS and MIT Movies.
- Models: The experiments use GPT-3 models with 2.7B, 13B, and 175B parameters plus GPT-2 with 1.5B parameters.GPT-3 is accessed through the OpenAI API, and replication code is released.
3. Accuracy Varies Highly Across Prompts
GPT-3 accuracy varies substantially with training-example selection, ordering, and prompt format. The paper links this instability to majority-label and recency biases, which skew predictions toward frequent or recently presented answers.
- Training examples and order: Permutation changes alone moved SST-2 accuracy for GPT-3 2.7B from 54.3% to 93.4%.This makes example order as important as, or more important than, which examples are selected.
- Training examples and order: High variance persists with 16 training examples and larger models, while adding one example reduced DBPedia mean accuracy from 36.0% to 25.9%.More data does not necessarily reduce variance or improve accuracy.
- Biases: Unbalanced prompts make GPT-3 predictions favor the majority class, while balanced prompts can still show recency bias from class repetitions near the prompt’s end.For example, prompts ending with repeated Negative examples bias predictions toward Negative.
- Prompt format: Different prompt formats produce different average accuracies, but all tested formats still exhibit high variance across example sets.The study manually designed additional formats including question-answer, conversation-style, Web-page-like, and label-name variants.
4. What Causes the High Variance?
GPT-3’s few-shot predictions vary because majority-label, recency, and common-token biases shift its output distribution toward particular answers. These biases affect classification and generation tasks by favoring frequent, recent, or pretraining-common labels and entities.
- GPT-3’s predictions are biased toward answers that are frequent in the prompt, near its end, or common in pretraining data.These three biases explain why training examples, their ordering, and label names influence prediction distributions.
- Majority Label Bias: 50.2% of 4-shot LAMA predictions repeat one of the four training answers, compared with a correct repeat rate of 24.7%.This majority-label bias shifts generation outputs toward answers present in the prompt.
- Recency Bias: Recency bias can outweigh majority-label bias: the P P P N training set produces nearly 90% Negative predictions despite three Positive examples.In 4-shot LAMA, answers from the 1st through 4th training examples are overpredicted by 8.5%, 8.3%, 14.3%, and 16.1%, respectively.
- Impact on Predictions: For a sentiment prompt, intrinsic and prompt-induced biases frequently produce high-confidence Positive predictions, causing false positives at the default 50% threshold.Setting p(Positive) = 0.68 yields 94% validation accuracy for this classifier.
5. Contextual Calibration
Contextual calibration estimates prompt- and model-induced answer bias from content-free inputs, then adjusts output probabilities to make those inputs uniform. Across GPT-3 and GPT-2 evaluations, it improves accuracy and generally reduces sensitivity to training examples and prompt formats.
- Calibration method: Contextual calibration applies an affine transformation to model output probabilities, using diagonal vector scaling to avoid quadratic parameter growth for large output spaces.For classification, the probabilities correspond to label names; for generation, they cover the first output token.
- Calibration method: The method estimates answer bias with a content-free input, then sets calibration parameters so its class scores become uniform.For a sentiment prompt, the content-free input “N/A” received 61.8% Positive before correction; calibration uses W = diag(ˆpcf)−1 and a zero bias vector.
- Results: 30.0% absolute is the maximum reported improvement in GPT-3 average and worst-case accuracy across classification and generation tasks.Contextual calibration sometimes let GPT-3 2.7B outperform the GPT-3 175B baseline by up to 19.3%.
- Results: Contextual calibration considerably reduces variance across training-example choices in a majority of tasks, while increasing it only modestly in the remaining cases.The reported variance is the standard deviation of accuracy across different training-example selections.
- Results: For GPT-2, contextual calibration works without task-specific modification, improving mean accuracy and reducing variance for most tasks.The paper presents this as evidence that few-shot prompt variance is a general problem for language models.
- Results: Across varied prompt formats, contextual calibration improves average and worst-case accuracy for SST-2 and LAMA, while reducing SST-2 variance.The evaluation varies 15 prompt formats for SST-2 and for three LAMA relations.
6. Discussion
Contextual calibration mitigates, but does not eliminate, the need to engineer prompts. It makes best-, average-, and worst-case prompt accuracy more similar and higher while retaining in-context learning without finetuning.
- Contextual calibration does not eliminate prompt engineering, but it makes the best, average, and worst-case prompts more similar and higher in accuracy.
- The authors use a fixed language model without finetuning, preserving rapid prototyping, a natural-language interface, and lower memory and serving complexity.
7. Related Work
The paper builds on work showing that language models can solve many tasks through in-context learning and that prompt choices affect few-shot performance. It adds contextual calibration and relates observed failures to broader language-model biases.
- Few-shot Learning with Language Models: Prior work showed that large language models can solve a wide range of tasks in a few-shot manner through in-context learning.
- Contextual calibration achieves similar AGNews accuracy to oracle calibration while using no training data, across different training-example choices.
- Few-shot Learning with Language Models: The paper introduces a simple modification to in-context learning that improves performance and suggests probing methods may underestimate model accuracy.
- Volatility of Few-shot Learning in NLP: Earlier studies found that prompt formats and training-example choices can affect zero-shot or few-shot performance, and this paper shows similar instability for in-context learning.
- Failures of Language Models: The paper identifies recency bias in in-context learning alongside previously documented repetition, overconfidence, recency, and generic-response failures in text generation.
8. Conclusion and Future Work
The paper argues that small methodological choices such as calibration can strongly affect few-shot results, complicating method development and comparison. It also calls for deeper analysis of what models learn from prompts.
- Methods: Small but non-trivial decisions such as calibration can greatly influence few-shot results, making new methods harder to develop and compare correctly.
- Methods: The authors hope to make few-shot methods more robust and extend calibration to a wider range of tasks, including open-ended generation.
- Analysis: The results motivate studying how GPT-3 learns from prompts, including its ability to improve with more examples and its tendency to learn superficial answer-repetition patterns.
A. Additional Results on Variance and Calibration
Additional results illustrate sensitivity to example ordering and content-free calibration inputs, while comparing calibration across prompt formats and model sizes. These results clarify where calibration choices matter and how broadly the method helps.
- Variance: Table 2 demonstrates that simply reversing the order of two examples can move GPT-3 2.7B from good accuracy to near-random performance.
- Variance: 88.5% accuracy drops to 51.3% when two training examples are reversed, illustrating strong sensitivity to example ordering.
- Calibration: The choice of content-free input affects accuracy, but many good choices exist; experiments ensemble N/A, [MASK], and the empty string.
- Prompt Format: Figure 9 compares GPT-3 accuracy with and without calibration as the prompt format varies for LAMA.
- Model Comparison: Table 4 reports the effect of contextual calibration for GPT-2.
B. Prompt Formats Used
The paper specifies default prompts for classification and generation, alongside multiple formats used to study prompt-format effects. Figure 9 examines contextual calibration across 15 formats for three LAMA relations, while Table 4 reports analogous GPT-2 results.
- Default prompts are specified for all tasks, with separate prompt tables for text classification and generation tasks.Tables 5 and 6 illustrate one training example per task.
- Figure 9 plots GPT-2 2.7B’s mean accuracy across 15 formats for three LAMA relations: place of death, Headquarter Location, and place of birth.The figure caption states that contextual calibration improves accuracy across these prompt formats.
- Table 4 presents GPT-2 XL results analogous to Table 1 and describes contextual calibration accuracy results for GPT-2.
- The prompts show label names, and predictions are made by checking the language model’s probability for the corresponding tokens.
- 15 different prompt formats are used to study the effect of format on SST-2.Table 7 shows one illustrative training example for these formats.