PaperlayerView exact revision on arXiv

Source-linked AI summary

PaLM 2 Technical Report

Rohan Anil, Andrew M. Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, Eric Chu, Jonathan H. Clark, Laurent El Shafey, Yanping Huang, Kathy Meier-Hellstern, Gaurav Mishra, Erica Moreira, Mark Omernick, Kevin Robinson, Sebastian Ruder, Yi Tay, Kefan Xiao, Yuanzhong Xu, Yujing Zhang, Gustavo Hernandez Abrego, Junwhan Ahn, Jacob Austin, Paul Barham, Jan Botha, James Bradbury, Siddhartha Brahma, Kevin Brooks, Michele Catasta, Yong Cheng, Colin Cherry, Christopher A. Choquette-Choo, Aakanksha Chowdhery, Clément Crepy, Shachi Dave, Mostafa Dehghani, Sunipa Dev, Jacob Devlin, Mark Díaz, Nan Du, Ethan Dyer, Vlad Feinberg, Fangxiaoyu Feng, Vlad Fienber, Markus Freitag, Xavier Garcia, Sebastian Gehrmann, Lucas Gonzalez, Guy Gur-Ari, Steven Hand, Hadi Hashemi, Le Hou, Joshua Howland, Andrea Hu, Jeffrey Hui, Jeremy Hurwitz, Michael Isard, Abe Ittycheriah, Matthew Jagielski, Wenhao Jia, Kathleen Kenealy, Maxim Krikun, Sneha Kudugunta, Chang Lan, Katherine Lee, Benjamin Lee, Eric Li, Music Li, Wei Li, YaGuang Li, Jian Li, Hyeontaek Lim, Hanzhao Lin, Zhongtao Liu, Frederick Liu, Marcello Maggioni, Aroma Mahendru, Joshua Maynez, Vedant Misra, Maysam Moussalem, Zachary Nado, John Nham, Eric Ni, Andrew Nystrom, Alicia Parrish, Marie Pellat, Martin Polacek, Alex Polozov, Reiner Pope, Siyuan Qiao, Emily Reif, Bryan Richter, Parker Riley, Alex Castro Ros, Aurko Roy, Brennan Saeta, Rajkumar Samuel, Renee Shelby, Ambrose Slone, Daniel Smilkov, David R. So, Daniel Sohn, Simon Tokumine, Dasha Valter, Vijay Vasudevan, Kiran Vodrahalli, Xuezhi Wang, Pidong Wang, Zirui Wang, Tao Wang, John Wieting, Yuhuai Wu, Kelvin Xu, Yunhan Xu, Linting Xue, Pengcheng Yin, Jiahui Yu, Qiao Zhang, Steven Zheng, Ce Zheng, Weikang Zhou, Denny Zhou, Slav Petrov, Yonghui Wu

arXiv:2305.10403v3cs.CLcs.AI

TL;DR

PaLM 2 addresses how to improve language-model quality across multilingual, reasoning, and other tasks while making inference more efficient. It combines compute-optimal scaling, broader data mixtures, and architectural and objective improvements, and reports stronger performance than PaLM across diverse evaluations. The results support using smaller, higher-quality models for more efficient deployment, while reported model results do not necessarily transfer directly to evolving user-facing products.

  • Problem

    Language models need stronger multilingual, reasoning, and natural-language-generation evaluation, while memorization and downstream harms complicate assessment across diverse uses.

  • Method

    PaLM 2 combines compute-optimal scaling, a more multilingual and diverse pre-training mixture, Transformer architecture, and varied training objectives, with evaluations of capabilities and responsible AI risks.

  • Results

    PaLM 2 significantly outperforms PaLM across a wide range of tasks, including English and multilingual understanding, reasoning, generation, and translation, while using less inference compute.

  • Takeaways & Limitations

    Training smaller models with more tokens and higher-quality data can improve capability and inference efficiency, supporting broader deployment and faster serving.

  • Takeaways & Limitations

    Reported results concern pre-trained models, whereas fine-tuned variants and user-facing products add processing and may evolve over time.

Abstract

from arXiv · show

We introduce PaLM 2, a new state-of-the-art language model that has better multilingual and reasoning capabilities and is more compute-efficient than its predecessor PaLM. PaLM 2 is a Transformer-based model trained using a mixture of objectives. Through extensive evaluations on English and multilingual language, and reasoning tasks, we demonstrate that PaLM 2 has significantly improved quality on downstream tasks across different model sizes, while simultaneously exhibiting faster and more efficient inference compared to PaLM. This improved efficiency enables broader deployment while also allowing the model to respond faster, for a more natural pace of interaction. PaLM 2 demonstrates robust reasoning capabilities exemplified by large improvements over PaLM on BIG-Bench and other reasoning tasks. PaLM 2 exhibits stable performance on a suite of responsible AI evaluations, and enables inference-time control over toxicity without additional overhead or impact on other capabilities. Overall, PaLM 2 achieves state-of-the-art performance across a diverse set of tasks and capabilities. When discussing the PaLM 2 family, it is important to distinguish between pre-trained models (of various sizes), fine-tuned variants of these models, and the user-facing products that use these models. In particular, user-facing products typically include additional pre- and post-processing steps. Additionally, the underlying models may evolve over time. Therefore, one should not expect the performance of user-facing products to exactly match the results reported in this report.

1 Introduction

PaLM 2 combines modeling, data, and scaling advances to improve multilingual, coding, reasoning, and language-generation capabilities over PaLM. Its smaller, higher-quality models also improve inference efficiency, while responsible-AI features address toxicity, memorization, and downstream harms.

  • PaLM 2 unifies modeling advances, data improvements, and scaling insights in a successor to PaLM.
  • Compute-optimal scaling indicates that data and model size should grow roughly 1:1 for a fixed training-compute budget.This contrasts with earlier trends that scaled models faster than datasets.
  • A more multilingual and diverse pre-training mixture spans hundreds of languages and domains without reducing English understanding as models grow.The mixture includes programming languages, mathematics, and parallel multilingual documents, with deduplication to reduce memorization.
  • PaLM 2 significantly outperforms PaLM across natural language generation, translation, reasoning, and advanced language-proficiency evaluations.It passes exams in all evaluated languages, with some results reaching proficiency sufficient to teach that language.
  • PaLM 2-L is significantly smaller than the largest PaLM model while using more training compute, improving inference efficiency and reducing serving cost.The report links this efficiency to broader downstream deployment and access for more applications and users.
  • PaLM 2 adds inference-time toxicity control and evaluates memorization, multilingual toxicity, harms, biases, and representation across downstream-use contexts.The report also notes that not all PaLM 2 capabilities are exposed through its APIs.

2 Scaling law experiments

The study derives compute-optimal scaling laws at very large model scales and finds that model size and data size should grow in roughly equal proportions. However, minimizing training loss does not guarantee optimal downstream task performance, so throughput and serving latency also matter.

  • 2.1 Scaling laws: Four compute budgets—1×10^19, 1×10^20, 1×10^21, and 1×10^22 FLOPs—were used to train differently sized models and derive the scaling curves.The procedure uses FLOPs ≈6ND to determine training tokens and fully decays each model’s learning rate at its final training token.
  • 2.1 Scaling laws: Model size and data size should grow in equal proportions as the training-compute budget increases.The study independently derives this relationship for very large models, while noting that these scaling-law models are separate from the deployed PaLM 2 models.
  • 2.1 Scaling laws: Quadratic fits to smoothed validation losses identify optimal model sizes for each compute band, while the FLOPs heuristic determines the corresponding data amounts.The resulting optimal parameter and token counts are then fit with a power-law curve.
  • 2.1 Scaling laws: Models approximately following the predicted optimal parameter counts achieve the lowest training loss at their respective FLOPs budgets.The reported model sizes and FLOPs apply only to the scaling-law study, not to the PaLM 2 models.
  • 2.2 Downstream metric evaluations: A 9.5B model is predicted to be optimal at 1×10^22 FLOPs, but it slightly underperforms a 16.1B model on downstream tasks.This indicates that training loss is not a perfect proxy for downstream metrics.
  • 2.2 Downstream metric evaluations: Optimal training loss for a fixed FLOPs budget does not necessarily produce optimal task performance, and model-size decisions also depend on throughput and serving latency.The paper therefore treats scaling laws as useful for training-loss optimization rather than a complete rule for deployment choices.

3 Training dataset

PaLM 2 uses a larger, more diverse pre-training corpus with substantially more non-English and parallel multilingual data than earlier large language models. The training pipeline also applies data-quality filtering, toxicity control tokens, memorization canaries, and increased context length.

  • 3 Training dataset: PaLM 2’s pre-training corpus combines web documents, books, code, mathematics, and conversational data, and is larger than PaLM’s corpus.It also contains a higher percentage of non-English data than previous large language models.
  • 3 Training dataset: Parallel source-target text pairs covering hundreds of languages improve multilingual understanding and generation while providing an inherent translation ability.One side of each pair is in English.
  • 3 Training dataset: Data de-duplication, sensitive-PII removal, and filtering are used to improve corpus quality.The authors partially attribute PaLM 2’s English evaluation gains to the higher quality of its data mixture despite its smaller proportion of English data than PaLM.
  • 3 Training dataset: Toxicity-marking control tokens support inference-time toxicity conditioning without negatively affecting performance on unrelated tasks.The tokens are added to a small fraction of pre-training data using signals from a fixed version of the Perspective API.
  • 3 Training dataset: PaLM 2 uses a substantially longer context length to support long dialog, long-range reasoning, comprehension, and summarization without hurting generic benchmark performance.The cited generic benchmarks may not require longer contexts.

4 Evaluation

PaLM 2 is evaluated across English, multilingual, reasoning, coding, translation, and natural-language-generation tasks, using multiple model sizes and few-shot settings. It generally outperforms PaLM and competing systems, while showing strong multilingual, reasoning, and coding results with some translation variation and benchmark-overlap caveats.

  • Classification and question answering: PaLM 2-L achieves large improvements over PaLM across almost all English classification and question-answering tasks, especially ANLI, ReCoRD, and RACE.The comparison uses one-shot evaluation, with PaLM 2 variants compared against PaLM 540B.
  • Classification and question answering: All PaLM 2 variants outperform PaLM on multilingual TyDi QA in both Gold Passage and no-context settings, with the largest model strongest in the harder no-context setting.The gains are particularly pronounced for languages with less training data.
  • Reasoning: PaLM 2 outperforms PaLM across reasoning datasets, achieves results competitive with GPT-4, and establishes a multilingual XCOPA state of the art without chain-of-thought prompting.On BIG-Bench Hard, it outperforms PaLM on every task; several multi-step and hierarchical reasoning tasks improve by more than 100%.
  • Reasoning: Some reasoning comparisons have a data-exposure caveat because WinoGrande, ARC, DROP, and COPA appear in the Flan mixture used to instruction-tune PaLM 2.The authors state that training data may have been seen by PaLM 2, while dev and test data were not, and exposure of other models is unknown.
  • Coding: PaLM 2-S* significantly improves coding performance over PaLM 2-S while preserving natural-language performance, and outperforms the much larger PaLM-540B-Coder on all evaluated coding benchmarks.PaLM 2-S* is trained with an extended, code-heavy, heavily multilingual mixture and is evaluated on HumanEval, MBPP, ARCADE, and translated coding tasks.
  • Translation: PaLM 2 improves translation quality over PaLM and Google Translate in MQM and regional-translation evaluations, while gender-agreement gains vary across languages.It improves gender agreement in Spanish, Polish, and Portuguese but scores lower than PaLM for Telugu, Hindi, and Arabic.

5 Responsible usage

PaLM 2’s responsible-usage analysis evaluates inference-time toxicity control while emphasizing that downstream applications require context-specific risk assessment and mitigation. The section also identifies boundaries around memorization, prompting, and generalization to user-facing systems.

  • Deployment scope: Pre-trained-model evaluations may not match user-facing products because fine-tuning, additional processing, and model evolution can change performance.Application-specific evaluation remains necessary when deploying PaLM 2.
  • Inference-time toxicity control: Control tokens significantly influence toxic-continuation probability and can either reduce or increase toxicity from non-toxic prompts.The method modifies only a small fraction of pre-training tokens.
  • Inference-time toxicity control: 12% versus 30% on the standard dataset and 7% versus 18% on the adversarial dataset are the toxic-response percentages reported with PaLM 2 control.These comparisons describe reductions from the corresponding baselines.
  • Inference-time toxicity control: Dialog prompting alone reduces toxic generation more effectively than control tokens, with small additional gains from combining them only on the standard dataset.The adversarial dataset measures a construct distinct from the pre-training toxicity tags.
  • Application-specific mitigation: Specialized downstream mitigation methods remain more effective than general-purpose inference-time mitigations for dialog toxicity.The authors point to application-specific fine-tuning, filtering, and sample-and-rank methods as examples of specialized mitigation.
  • Application-specific mitigation: Responsible-use evaluations provide a signal for downstream systems, but developers must assess harms and bias in each application context.The authors caution that results may not generalize across representational harms, prompting methods, or use contexts.

6 Conclusion

PaLM 2 concludes that strong performance gains can come from data, architecture, objectives, and compute allocation rather than scaling model size alone. The resulting models outperform PaLM across tasks while improving inference efficiency.

  • Conclusion: PaLM 2 significantly outperforms PaLM across English, multilingual, and reasoning tasks while using significantly less inference compute.The conclusion characterizes PaLM 2 as a new state-of-the-art model.
  • Scaling: Training tokens should grow at roughly the same rate as model parameters, consistent with compute-optimal scaling at larger scales.This conclusion reports the paper’s scaling-law finding.
  • Design choices: Data mixture, architecture, and varied objectives all contribute to performance gains, while more training compute for a smaller model can improve efficiency.The authors state that, for fixed inference and training budgets, training a smaller model with more tokens is generally more efficient.
  • Future directions: Further scaling of model parameters and dataset size and quality, alongside architectural and objective improvements, is expected to yield additional gains.This is presented as the authors’ forward-looking conclusion.

7 Authorship, attribution, and acknowledgements

This section attributes PaLM 2 to multiple workstreams and contributors, then defines the project’s contribution labels and acknowledges supporting teams, reviewers, and colleagues.

  • Workstreams: The listed workstreams cover pretraining, design, data, evaluation, responsible AI, long context, optimization, code, serving, and leadership.Each workstream names leads and contributors.
  • Contribution roles: Lead denotes responsibility for an entire workstream throughout the project.The section defines this role explicitly.
  • Contribution roles: Core Contributor denotes significant contributions to a workstream throughout the project.The role definition does not establish an ordering of contribution magnitude.
  • Contribution roles: Contributor denotes project contributions with partial involvement in the effort.Individuals within each role are ordered alphabetically by first name.
  • Acknowledgements: The authors acknowledge infrastructure teams, reviewers, and colleagues who provided support, input, discussion, and responsible-AI experimentation.Acknowledged groups include infrastructure, resource-management, and responsible-AI collaborators.

A Detailed results

The detailed-results section describes scaling-law methodology, instruction tuning, multilingual reasoning, coding evaluations, and examples of model outputs. It reports that PaLM 2 improves multilingual reasoning over PaLM, including on under-represented languages.

  • Scaling-law methodology: IsoFLOP curves estimate compute-optimal parameter and token counts by fitting quadratic validation-loss curves across four compute scales.The methodology uses scales from 1×10^19 to 1×10^22 FLOPs.
  • Scaling-law methodology: Models trained on 1×10^22 FLOPs are evaluated on downstream tasks at a single compute scale.The corresponding results are reported in Table 15.
  • Instruction tuning: Flan instruction tuning uses more than 1,800 tasks and at least 20 instruction templates per task to improve instruction following and unseen-task performance.The section reports results in Table 16.
  • Multilingual reasoning: PaLM 2 significantly outperforms PaLM in standard multilingual XCOPA prompting across all evaluated languages.The strongest improvements occur for Swahili, Quechua, and Haitian, and standard PaLM 2 prompting exceeds PaLM with chain-of-thought prompting.
  • Coding: Coding evaluations use HumanEval, MBPP, ARCADE, and BabelCode, with pass@1 and pass@k settings where specified.HumanEval and MBPP test natural-language-to-Python generation, while ARCADE tests notebook-cell completion.
  • Model examples: The section includes sample model outputs, most from an instruction-following fine-tuned PaLM 2 variant.These examples are presented as demonstrations of model capabilities.

B.1 Multilinguality

PaLM 2 demonstrates multilingual capabilities across explanation, translation, dialect conversion, transliteration, style transfer, creative generation, and coding tasks. Examples show zero-shot or prompted generation that follows task instructions across languages, dialects, and scripts.

  • PaLM 2 explains jokes, puns, and haikus in Japanese and clarifies idiomatic translations in German and Swahili.
  • PaLM 2 converts between regional varieties by identifying dialect-specific terms and replacing them with equivalent expressions.Examples include European and Argentinian Spanish and Brazilian and European Portuguese.
  • PaLM 2 performs multilingual text normalization and transformation, including abbreviation expansion, typo correction, register conversion, and transliteration to Latin script.
  • PaLM 2 generates and transforms coherent text across languages, including Persian email, Arabic translation, Pinglish transliteration, and Tajiki-to-Persian conversion.The examples report successful generation across different scripts and languages while following the requested transformations.
  • PaLM 2 produces creative multilingual outputs, including an Armenian-name-based children’s game, a Taiwanese speech, and explanations in German and Badisch.The examples describe outputs as realistic, fluent, natural, or appropriate to the target audience.
  • PaLM 2 demonstrates coding capabilities in multilingual settings, including bug fixing with Korean comments and function generation with Malayalam comments.

C Language proficiency exams

The paper evaluates PaLM 2 with publicly available or obtained human language-proficiency exams across multiple languages under simulated, zero-shot conditions. Scores are converted into pass/fail outcomes using exam-specific guidelines, with cross-contamination also assessed.

  • The evaluation uses advanced language-proficiency exams in Chinese, Japanese, French, Spanish, and Italian.The cited passages identify the exams and describe their intended proficiency levels or uses.
  • Questions are presented zero-shot, using multiple-choice or free-text responses across reading, listening, and writing components.
  • Exam scores weight reading and writing equally and apply exam-specific pass thresholds, including 60% for HSK, DELE, and PLIDA.The reported outcomes are simulated and are not official or accredited exam results.
  • The study measures potential evaluation-data contamination through normalized 25-gram substring overlap with pre-training data.
  • The language distribution of multilingual web documents and responsible-AI analyses are included alongside the proficiency evaluation.

D.1 Dataset analysis

The dataset analysis examines how people and identity-related signals are represented in PaLM 2’s pre-training data and how toxicity varies across identity-referencing subsets. It uses automated tools and classifiers while emphasizing measurement limitations and downstream complexity.

  • The analysis focuses on representations of people in pre-training data as part of a responsible-AI assessment.
  • Automated analyses use Know Your Data and classifiers to measure relative signal frequencies within optimized training-data formats.
  • The measured signals include identity mentions, English gender pronouns, grammatical person, and English toxicity probability.
  • Marked sexuality references occur in 3% of documents, while “gay” accounts for 53% and “homosexuality” for 22% of those references.
  • References to Americans comprise 42% of mentions in the broad race, religion, ethnicity, and nationality category, indicating a Western skew in representation.
  • Documents mentioning “white people” have the highest reported toxicity probability at 0.250 among qualifying identity subsets.
  • Automated identity measures are limited for assumed or unmarked identities and may vary in validity across analyses and social contexts.

D.2 Evaluation approach

The evaluation approach treats language-model harms as dependent on downstream uses and sociotechnical contexts. It prioritizes multilingual, automated assessments across capabilities, applications, and potential harms while acknowledging that these measures are incomplete.

  • Evaluations begin by reviewing core PaLM 2 capabilities and extending Responsible AI assessments toward multiple languages.
  • The study prioritizes downstream uses by considering how developers build language-model systems and which application types are likely to matter.
  • Potential harms are conceptualized as adverse lived experiences arising from misuse, system failures, or harmful effects during proper use.
  • The evaluation measures observable task-level harms and treats them as limited approximations rather than exhaustive or holistically complete assessments.
  • The work prioritizes automated evaluations while sharing measurement-quality rubrics intended to support improvements in validity, relevance, and inclusivity.
  • The analysis considers potential harms across dialog, classification, translation, question answering, language modeling, and memorization-related privacy risks.
  • Dialog evaluations approximate common assistant systems, although real systems may include fine-tuning and additional safety-mitigation methods.

D.3.1 Toxic language harms in dialog generation

The evaluation measures toxic degeneration in dialog systems using standard and adversarial prompts, comparing dialog-prompted PaLM 2 with language-modeling baselines. Prompting reduces toxicity for single responses, but substantial toxic responses remain when sampling multiple continuations.

  • Evaluation setup: The evaluation samples 20 continuations per prompt from balanced standard and adversarial toxic-interaction datasets, counting prompts with at least one Perspective API toxicity score above 0.5.Standard interactions are explicitly toxic, whereas adversarial interactions contain implicit and subtly harmful language.
  • Prompting effects: Dialog-prompted PaLM 2 has significantly lower toxic degeneration than language-modeling evaluation, showing that toxicity measurements depend on context and prompting.The comparison includes PaLM 2 and PaLM language-modeling baselines and an early LaMDA dialog system.
  • Prompting effects: With 20 samples, toxic responses occur for over 30% of adversarial queries and almost 60% of standard queries.Thus, prompting reduces toxic degeneration for one sampled response but is less effective when multiple responses are considered.

D.3.2 Multilingual toxic language harms and representational bias

PaLM 2’s multilingual toxicity and bias evaluation spans 16 languages and identity groups. Dialog prompting controls toxicity for most languages, but English, German, and Portuguese remain outliers with higher toxicity and identity-group disparities.

  • Evaluation scope: The evaluation extends representational-bias testing to toxic language harms across 16 languages and adapts it for dialog context.Queries target ethnicity, religion, and gender identity terms.
  • Evaluation method: The multilingual evaluation samples 10 continuations per prompt with top-k = 40 and temperature = 1.0, then scores toxicity with Perspective API.Results are disaggregated by language and identity group; classifier performance varies across languages.
  • Toxicity results: Toxic-response rates range from 1% to 17.9% in worst-case English, German, and Portuguese evaluations, where toxicity and variance across identity groups are significant.Dialog prompting controls toxicity for most other evaluated languages.
  • Representational bias: In English, German, and Portuguese, toxicity varies across identity groups, including higher rates for selected racial, religious, and gender groups.For most other languages, toxicity remains below 5% across identity groups.
  • Implications and limitations: Developers should continue investing in application-specific mitigations because prompted PaLM 2 systems still produce toxic harms and language- and identity-dependent bias.The authors limit this analysis to dialog-prompting methods and one specific dialog prompt.
  • Toxicity classification: PaLM 2 significantly improves toxicity-classification performance over PaLM in zero-shot and 10-shot settings, including across the analyzed languages.The advantage is larger in zero-shot evaluation and narrower after 10-shot prompting.
  • Implications and limitations: Automated toxicity measures contain noise and bias, while classification-bias evaluation covers only a biased subset of English identity terms.These evaluations may not generalize to other responsible-AI constructs, and fine-tuned classifier workflows are not evaluated.

D.5.2 Translating from English

The paper evaluates PaLM 2’s zero-shot translation from English using human ratings of gender agreement and translation quality across multiple languages. PaLM 2 improves gender agreement over PaLM in some languages and exceeds Translate in three high-resource languages, while data representation broadly relates to quality.

  • Evaluation method: Because gender is encoded differently across languages, the evaluation uses human ratings rather than automatic gender-pronoun matching for translations out of English.Two raters score gender agreement from 0 to 3 and assess translation quality on passages of 1–3 sentences.
  • Results: PaLM 2 outperforms PaLM on gender agreement in some languages and outperforms Google Translate in Spanish, Polish, and Portuguese.The evaluation reports average gender-agreement and translation-quality scores for zero-shot translations.
  • Results: PaLM 2 shows improved translation quality in Italian and Japanese despite lower gender-agreement scores in those languages.This indicates that translation quality and gender agreement do not always improve together.
  • Training-data relationship: Across PaLM and PaLM 2, languages representing over 2% of web-document training data generally achieve average translation-quality scores above 2.50, with Arabic an exception.The authors characterize multilingual training data as one important part of mitigating potential misgendering harms.
  • Limitations: The evaluation excludes non-binary gender harms and contested cross-cultural pronoun perspectives, and assessing gender agreement out of English remains challenging and time-intensive.It also covers only a subset of potential risks and focuses on model outputs without the wider sociotechnical context.

D.6 Question answering uses

The paper adapts BBQ to evaluate representational harms in generative question answering, analyzing free-form responses rather than multiple-choice selections. PaLM 2 performs well on disambiguated contexts but remains unreliable and bias-prone when contexts are ambiguous or contain unexpected stereotypes.

  • Evaluation design: The benchmark covers nine social dimensions and distinguishes ambiguous contexts, where the answer is unknown, from disambiguated contexts, where one named person is answerable.Responses are categorized as biased, anti-biased, or other relative to each context and question.
  • Evaluation design: The evaluation adapts BBQ for generative QA by presenting PaLM 2 with a context and question and analyzing generated text without multiple-choice options.This design targets harms that free-form generation may introduce beyond the original three-option setup.
  • Disambiguated contexts: 91.4% overall accuracy is achieved on disambiguated questions, while 38.2% of errors reinforce harmful social bias.Such errors represent 3% of all disambiguated questions.
  • Disambiguated contexts: In correct disambiguated responses, answers aligned with social bias occur 2.2 percentage points more often than anti-biased answers.Among incorrect answers, Other responses account for 22.5% of errors.
  • Additional harms: Unexpected biases appeared in 7 of 9 sensitive-topic categories, affecting 86 queries, or about 0.5% of all queries.Examples included stereotypes unrelated to the bias explicitly tested by the prompt.
  • Additional harms: The analysis motivates systematic measurement of representational harms that may introduce stereotypes unrelated to the original query.The authors connect this concern to the need for evaluating harms in good-faith use cases.

D.7.1 Open-ended generation

The evaluation examines toxic degeneration in PaLM 2 using multiple prompt conditions and evaluation variants. PaLM 2 shows minor aggregate improvements over PaLM, while toxicity patterns remain stable across prompt-toxicity strata and training.

  • The small evaluation filters 50k prompts to 38k prompts with toxicity probability below 0.5 and uses greedy decoding.
  • PaLM 2 shows slightly lower toxic degeneration than PaLM across both evaluation variants.
  • Continuations have consistently lower toxicity probability than inputs across all prompt-toxicity strata and remain below the human baseline.
  • Mean TPC was 0.0676±0.0026 during 87% of training epochs, with worst-case performance of 0.0725 in the tiny evaluation.
  • The authors recommend the small evaluation task and better-constructed evaluation sets to improve measurement quality and enable continuous evaluation and ablations.
  • Toxicity judgments vary with comment content, surrounding context, and annotators’ cultural familiarity with language referring to social groups.
Loading 2305.10403v3…