PaperlayerView exact revision on arXiv

Source-linked AI summary

Prompting GPT-3 To Be Reliable

Chenglei Si, Zhe Gan, Zhengyuan Yang, Shuohang Wang, Jianfeng Wang, Jordan Boyd-Graber, Lijuan Wang

arXiv:2210.09150v2cs.CL

TL;DR

The paper addresses the under-explored problem of improving GPT-3 reliability for real-world use, focusing on generalizability, social biases, calibration, and factuality. It develops simple prompting strategies for these facets and reports that appropriately prompted GPT-3 outperforms smaller supervised models across the evaluated reliability metrics. The findings offer practical guidance while emphasizing that GPT-3 remains imperfect and not ready for deployment solely on this evidence.

  • Problem

    GPT-3 reliability is under-explored despite biased generations, false statements, outdated information, and the need for trustworthy confidence estimates.

  • Method

    The paper systematically evaluates prompting strategies across generalizability, social biases, calibration, and factuality, including source examples, balanced demographics, natural-language interventions, and added knowledge.

  • Results

    Appropriately prompted GPT-3 outperforms smaller supervised models across the paper’s evaluated reliability facets and metrics.

  • Takeaways & Limitations

    The prompting strategies provide practical recommendations for using GPT-3 more reliably while revealing insights about prompting-based LLM reliability.

  • Takeaways & Limitations

    The authors stress that GPT-3 remains far from perfect, with ample room for improvement especially in social-bias reduction and calibration, and is not established as deployment-ready.

Abstract

from arXiv · show

Large language models (LLMs) show impressive abilities via few-shot prompting. Commercialized APIs such as OpenAI GPT-3 further increase their use in real-world language applications. However, the crucial problem of how to improve the reliability of GPT-3 is still under-explored. While reliability is a broad and vaguely defined term, we decompose reliability into four main facets that correspond to the existing framework of ML safety and are well-recognized to be important: generalizability, social biases, calibration, and factuality. Our core contribution is to establish simple and effective prompts that improve GPT-3's reliability as it: 1) generalizes out-of-distribution, 2) balances demographic distribution and uses natural language instructions to reduce social biases, 3) calibrates output probabilities, and 4) updates the LLM's factual knowledge and reasoning chains. With appropriate prompts, GPT-3 is more reliable than smaller-scale supervised models on all these facets. We release all processed datasets, evaluation scripts, and model predictions. Our systematic empirical study not only sheds new insights on the reliability of prompting LLMs, but more importantly, our prompting strategies can help practitioners more reliably use LLMs like GPT-3.

1 INTRODUCTION

The paper frames GPT-3 reliability across generalizability, social bias, calibration, and factuality, developing prompting strategies rather than only evaluating intrinsic model properties. These strategies reportedly improve reliability across all four facets and outperform smaller supervised models.

  • GPT-3 reliability is under-explored despite biased generations, false statements, and outdated information posing risks in real-world deployment.
  • The paper studies prompting strategies that improve reliability under these facets rather than only measuring intrinsic model characteristics.
  • The study organizes reliability around four facets: generalizability, social biases, calibration, and factuality.The framework is inspired by unsolved problems in ML safety.
  • Balanced demographic examples and natural-language interventions reduce social biases, while appended up-to-date knowledge can replace memorized knowledge or reasoning chains.
  • The proposed prompting strategies beat smaller-scale supervised models on all considered reliability metrics.

2 FACET 1: GENERALIZABILITY

This section tests whether few-shot GPT-3 prompting remains robust under domain shifts, perturbations, and spurious correlations. GPT-3 generally achieves stronger out-of-distribution generalization and smaller performance gaps than smaller supervised models.

  • 2 FACET 1: GENERALIZABILITY: Domain shift, text perturbations, and spurious correlations can cause substantial performance drops when models encounter real-world distribution changes.
  • Experiment Setup: The experiments evaluate GPT-3 on MRQA, AdvGLUE, Contrast Sets, HANS, and PAWS using demonstrations sampled from source or clean training data.The setup covers domain shift, adversarial perturbations, and shortcut reliance.
  • Results: F1 77.2 is achieved on MRQA target domains using either source-domain or target-domain demonstrations.
  • Results: GPT-3 achieves higher OOD accuracy and smaller generalization gaps than supervised BERT and RoBERTa, even when its in-domain accuracy is slightly lower.
  • Takeaway: Few-shot GPT-3 is more robust than fine-tuned BERT and RoBERTa across domain shift, perturbations, and spurious-correlation settings.
  • Takeaway: Source-domain demonstrations perform the same as target-distribution demonstrations in the reported generalization comparison.

3 FACET 2: SOCIAL BIAS AND FAIRNESS

The study finds that GPT-3's social-bias behavior is strongly shaped by prompt-example distributions and ordering, while explicit fairness instructions can further reduce bias. Balanced demonstrations and random shuffling generally produce fairer predictions across WinoBias and BBQ evaluations.

  • WinoBias: Balanced WinoBias demonstrations produce the smallest bias gaps, while prompts containing only Pro-Bias examples increase Pro-Bias predictions.The balanced prompt uses four examples from each Type I-Pro, Type I-Anti, Type II-Pro, and Type II-Anti subset, totaling 16 demonstrations.
  • WinoBias: Randomly shuffling WinoBias demonstrations reduces bias gaps more than placing Pro-Bias or Anti-Bias examples at the prompt's end.Ending the prompt with either group increases bias gaps.
  • Broader social dimensions: The BBQ balanced prompt best trades off accuracy and bias across ambiguous and disambiguated questions.BBQ evaluates accuracy, where higher is better, and bias score magnitude, where lower is better.
  • Broader social dimensions: Biased BBQ demonstrations yield high bias scores, whereas balanced demonstrations increase accuracy while maintaining low bias scores.Prompts containing only neutral answers can achieve extremely low bias scores but sacrifice accuracy on disambiguated questions.
  • Broader social dimensions: A natural-language fairness instruction makes GPT-3 choose neutral answers more often, increasing ambiguous-question accuracy and significantly reducing bias scores.The intervention instructs equal treatment across protected groups and choosing the unknown option when information is insufficient.
  • Takeaway: The authors conclude that balanced answer distributions, shuffled demonstrations, and explicit fairness instructions can reduce social biases in GPT-3 prompts.These strategies address demographic balance, example ordering, and intended model behavior.

4 FACET 3: UNCERTAINTY CALIBRATION

The paper evaluates GPT-3’s uncertainty calibration using intrinsic confidence measures and selective prediction. GPT-3 is better calibrated than supervised DPR-BERT, although calibration remains imperfect and does not necessarily improve with more demonstrations.

  • Evaluation Setup: The evaluation measures exact-match correctness, ECE, Brier score, reliability diagrams, and selective prediction on closed-book NQ, TriviaQA, and HotpotQA.It compares raw confidence scores with supervised DPR-BERT and focuses on intrinsic rather than post-hoc calibration.
  • Results: GPT-3 is better calibrated than supervised DPR-BERT on both in-domain and out-of-distribution settings.This comparison is reported using accuracy, ECE, and Brier scores.
  • Confidence Scoring: LM-Prob uses normalized language-model probability, while Self-Con uses the frequency of the most common answer across 10 temperature-0.7 samples.These provide two confidence estimates for GPT-3’s generated answers.
  • Results: Most GPT-3 calibration errors arise from overconfidence, and increasing demonstrations improves accuracy without necessarily improving calibration.For example, 2-shot prompting is 5.8 points less accurate than 64-shot prompting but better calibrated.
  • Results: Confidence scores support effective selective prediction: on NQ, the most confident 10% of GPT-3 predictions achieve 83.1% accuracy versus 60.1% for DPR-BERT.This enables trusting high-confidence predictions while routing the remainder for human verification.

5 FACET 4: FACTUALITY VIA KNOWLEDGE UPDATING

The paper improves factuality by supplying updated evidence and reasoning structure in prompts. Counterfactual passages enable knowledge updating, retrieval improves open-domain QA, and human decomposition improves multi-hop reasoning.

  • Motivation: Factuality is limited because LLMs can produce wrong or outdated answers despite storing extensive knowledge in their parameters.The section targets factual reliability through improved prompting methods.
  • Knowledge Updating: Counterfactual passage-question-answer triples test whether GPT-3 retains memorized answers, adopts substitution answers, or produces another answer.The setup uses questions from NQ and SQuAD and compares GPT-3 with a supervised reader baseline.
  • Knowledge Updating: About 85% of the time, GPT-3 updates its answer when prompted with counterfactual triples, outperforming the supervised baseline.Larger GPT-3 models also update better to new answers in counterfactual passages.
  • Retrieval-Augmented QA: Adding retrieved Wikipedia passages consistently improves GPT-3’s accuracy on all three open-domain QA datasets.Better retrieval recall is associated with better performance, with particularly large gains on SQuAD.
  • Reasoning-Augmented Multi-Hop QA: Human-written question decomposition improves multi-hop QA by guiding GPT-3’s subsequent reasoning steps and intermediate answers.Adding the first decomposed sub-questions helps both overall and sub-question accuracy, while adding the first human-written QA pair yields a further multi-hop gain.
  • Takeaways: The factuality results show that retrieval, in-context knowledge updating, and human decomposition address complementary failures in factual and multi-hop QA.The paper’s takeaway links retrieved evidence to factual QA, conflicting passages to knowledge updating, and decomposition to corrected reasoning chains.

6 CONCLUSION

The paper studies GPT-3 reliability across generalizability, fairness, calibration, and factuality, developing prompting strategies that outperform smaller supervised models on these facets. It presents these strategies as practical recommendations while identifying further reliability challenges and research directions.

  • Conclusion: The study systematically examines four reliability facets: generalizability, fairness, calibration, and factuality.Its prompting strategies are designed to improve reliability across these dimensions.
  • Conclusion: The proposed prompting strategies make GPT-3 outperform supervised models by large margins across the studied reliability facets.The conclusion characterizes the result as both a systematic empirical finding and a source of practical recommendations for GPT-3 users.
  • Conclusion: Future work should examine additional reliability facets, apply prompting to real-world applications, and explore more effective strategies such as post-hoc calibration.The paper specifically mentions harmful generations, human feedback for collaborative planning, and calibration of language-model probabilities.

ETHICAL STATEMENT

The paper frames ethical use as improving GPT-3 reliability while acknowledging that fairness remains incomplete and broader reliability questions remain open.

  • GPT-3 exhibits biased predictions, especially when prompt demonstrations have skewed demographic distributions.
  • The study covers four reliability facets, while recognizing that adversarial examples, prompt injection, toxicity, and hallucinations are additional concerns.
  • The authors caution that effective prompting strategies do not establish that GPT-3 is reliable or ready for deployment.
  • The causes of several observed prompting behaviors remain unclear, including whether they arise from prompting, training data, objectives, or architecture.

B ADDITIONAL RESULTS: GENERALIZABILITY

Additional experiments reinforce that GPT-3 prompting can generalize better than supervised baselines under several distribution shifts, while model size and demonstration choices affect outcomes.

  • GPT-3 shows better generalization than supervised baselines on AdvGLUE and Contrast Sets.
  • HANS retains significant performance gaps between bias-supporting and bias-countering subsets for subsequence and constituent features.
  • GPT-3 significantly outperforms supervised baselines on MRQA out-of-distribution datasets, and in-domain prompts transfer successfully.
  • GPT-3 significantly outperforms supervised models on adversarial AdvGLUE sets despite lagging on clean test sets.
  • Using fewer QQP demonstrations preserves robustness on PAWS, whereas smaller GPT-3 variants perform far worse on both QQP and PAWS.

C ADDITIONAL RESULTS: SOCIAL BIASES

The additional social-bias results show substantial variation across bias categories and do not establish that matching demonstration categories improves bias scores.

  • Bias scores differ substantially across categories, and same-category demonstrations do not correlate with lower bias scores.
  • The Contrast Sets results provide an additional generalization comparison, with GPT-3 incurring a smaller gap than supervised models.
  • Table 15 is an ablation of demonstration count and GPT-3 model variants on MNLI-HANS.

D ADDITIONAL RESULTS: CALIBRATION

Calibration analyses show that confidence ranks predictions meaningfully, but errors commonly reflect overconfidence and calibration remains consequential even at matched accuracy.

  • The most confident predictions have much higher accuracy, indicating that confidence scores can rank model predictions.
  • Most calibration errors arise from overconfidence, with confidence exceeding expected accuracy; GPT-3 behaves similarly with in-domain and OOD prompts.
  • When DPR-BERT and GPT-3 have equal accuracy, DPR-BERT achieves much higher accuracy among its most confident predictions, highlighting better calibration.

E ADDITIONAL RESULTS: KNOWLEDGE UPDATING

The section compares prompt designs for in-context knowledge updating. Counterfactual passage-question-answer demonstrations substantially improve GPT-3’s ability to replace memorized answers.

  • Prompt Design: Using 16 randomly sampled demonstrations, the study compares original and answer-substituted prompt designs for knowledge updating.The designs include original passage-question-answer triples, question-answer pairs with substitution entities, and answer-substituted passages with updated answers.
  • Results: 85% of the time, counterfactual triples ⟨P’, Q, A’⟩ enable GPT-3 to update answers with much lower memorization ratios than a supervised model.Original triples ⟨P, Q, A⟩ still produce high memorization ratios.

E.2 TARGETED IN-CONTEXT KNOWLEDGE UPDATING

Targeted in-context knowledge updating tests whether GPT-3 changes answers for relevant questions while preserving answers for unrelated ones. Mixing original, edited relevant, and edited irrelevant examples yields high editing success with minimal irrelevant-question drawdown.

  • Motivation and Experiment Setup: Targeted updating evaluates whether a knowledge edit changes answers to related questions without changing answers to unrelated questions.The setup uses FEVER and zsRE, measuring editing success rate and accuracy drawdown on unrelated questions.
  • Prompt Design: The study compares prompts containing original examples, edited relevant examples, and edited irrelevant examples, using 16 demonstrations in each case.Edited irrelevant examples pair an update with the original answer when the question is unrelated to that update.
  • Results: Adding edited relevant examples improves success rate but can make the model over-rely on updates for irrelevant questions.Using only original examples produces relatively poor success rates, especially on FEVER.
  • Results: 99.9% and 98.8% editing success rates are achieved with only 0.5% drawdown on irrelevant questions when all three example types are mixed.These results are reported for FEVER and zsRE, respectively.
  • Results: GPT-3 achieves high editing success and low drawdown on irrelevant questions when prompts incorporate original, edited relevant, and edited irrelevant examples.This finding summarizes the targeted updating behavior under the mixed-example design.
Loading 2210.09150v2…