PaperlayerView exact revision on arXiv

Source-linked AI summary

PaLM 2 Technical Report

Rohan Anil, Andrew M. Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, Eric Chu, Jonathan H. Clark, Laurent El Shafey, Yanping Huang, Kathy Meier-Hellstern, Gaurav Mishra, Erica Moreira, Mark Omernick, Kevin Robinson, Sebastian Ruder, Yi Tay, Kefan Xiao, Yuanzhong Xu, Yujing Zhang, Gustavo Hernandez Abrego, Junwhan Ahn, Jacob Austin, Paul Barham, Jan Botha, James Bradbury, Siddhartha Brahma, Kevin Brooks, Michele Catasta, Yong Cheng, Colin Cherry, Christopher A. Choquette-Choo, Aakanksha Chowdhery, Clément Crepy, Shachi Dave, Mostafa Dehghani, Sunipa Dev, Jacob Devlin, Mark Díaz, Nan Du, Ethan Dyer, Vlad Feinberg, Fangxiaoyu Feng, Vlad Fienber, Markus Freitag, Xavier Garcia, Sebastian Gehrmann, Lucas Gonzalez, Guy Gur-Ari, Steven Hand, Hadi Hashemi, Le Hou, Joshua Howland, Andrea Hu, Jeffrey Hui, Jeremy Hurwitz, Michael Isard, Abe Ittycheriah, Matthew Jagielski, Wenhao Jia, Kathleen Kenealy, Maxim Krikun, Sneha Kudugunta, Chang Lan, Katherine Lee, Benjamin Lee, Eric Li, Music Li, Wei Li, YaGuang Li, Jian Li, Hyeontaek Lim, Hanzhao Lin, Zhongtao Liu, Frederick Liu, Marcello Maggioni, Aroma Mahendru, Joshua Maynez, Vedant Misra, Maysam Moussalem, Zachary Nado, John Nham, Eric Ni, Andrew Nystrom, Alicia Parrish, Marie Pellat, Martin Polacek, Alex Polozov, Reiner Pope, Siyuan Qiao, Emily Reif, Bryan Richter, Parker Riley, Alex Castro Ros, Aurko Roy, Brennan Saeta, Rajkumar Samuel, Renee Shelby, Ambrose Slone, Daniel Smilkov, David R. So, Daniel Sohn, Simon Tokumine, Dasha Valter, Vijay Vasudevan, Kiran Vodrahalli, Xuezhi Wang, Pidong Wang, Zirui Wang, Tao Wang, John Wieting, Yuhuai Wu, Kelvin Xu, Yunhan Xu, Linting Xue, Pengcheng Yin, Jiahui Yu, Qiao Zhang, Steven Zheng, Ce Zheng, Weikang Zhou, Denny Zhou, Slav Petrov, Yonghui Wu

arXiv:2305.10403v3cs.CLcs.AI

TL;DR

PaLM 2 addresses limited multilingual, reasoning, and broad generation evaluation in large language models. It combines scaling, data, architecture, and objective improvements, achieving stronger performance than PaLM across diverse tasks while using less inference compute.

  • Problem

    Large language models have limited evidence for multilingual, reasoning, and broad natural-language-generation capabilities across diverse tasks.

  • Method

    PaLM 2 combines compute-optimal scaling, a more multilingual and diverse training mixture, and improved architecture and objectives.

  • Results

    PaLM 2 significantly outperforms PaLM across English, multilingual language, code generation, and reasoning tasks while using less inference compute.

  • Takeaways & Limitations

    Training a smaller model with more tokens can be more efficient than modifying architecture for inference efficiency under fixed budgets.

  • Takeaways & Limitations

    Reported results concern pre-trained models and may not match fine-tuned variants or user-facing products with additional processing and evolving underlying models.

Abstract

from arXiv · show

We introduce PaLM 2, a new state-of-the-art language model that has better multilingual and reasoning capabilities and is more compute-efficient than its predecessor PaLM. PaLM 2 is a Transformer-based model trained using a mixture of objectives. Through extensive evaluations on English and multilingual language, and reasoning tasks, we demonstrate that PaLM 2 has significantly improved quality on downstream tasks across different model sizes, while simultaneously exhibiting faster and more efficient inference compared to PaLM. This improved efficiency enables broader deployment while also allowing the model to respond faster, for a more natural pace of interaction. PaLM 2 demonstrates robust reasoning capabilities exemplified by large improvements over PaLM on BIG-Bench and other reasoning tasks. PaLM 2 exhibits stable performance on a suite of responsible AI evaluations, and enables inference-time control over toxicity without additional overhead or impact on other capabilities. Overall, PaLM 2 achieves state-of-the-art performance across a diverse set of tasks and capabilities. When discussing the PaLM 2 family, it is important to distinguish between pre-trained models (of various sizes), fine-tuned variants of these models, and the user-facing products that use these models. In particular, user-facing products typically include additional pre- and post-processing steps. Additionally, the underlying models may evolve over time. Therefore, one should not expect the performance of user-facing products to exactly match the results reported in this report.

1 Introduction

PaLM 2 unifies compute-optimal scaling, multilingual and diverse data, and improved Transformer objectives to improve multilingual, reasoning, coding, and language capabilities over PaLM. Its smaller, higher-quality models also improve inference efficiency, while the report evaluates toxicity, memorization, harms, and biases for downstream deployment.

  • PaLM 2 scales data and model size roughly 1:1 for compute-optimal training, rather than following earlier trends that scaled models three times faster than datasets.This scaling strategy is validated at larger compute budgets.
  • PaLM 2 uses a multilingual, domain-diverse and deduplicated pre-training mixture plus a tuned mixture of Transformer training objectives.The data spans hundreds of languages and domains, including programming, mathematics, and parallel multilingual documents; larger models handle more non-English data without reducing English understanding.
  • PaLM 2 models significantly outperform PaLM across natural language generation, translation, and reasoning tasks despite PaLM 2-L being smaller than the largest PaLM model.The results indicate that meticulous data selection and efficient architecture/objectives can improve performance beyond model scaling alone, while reducing serving cost and enabling broader deployment.
  • PaLM 2 shows strong multilingual language, code generation, and reasoning abilities, including better performance than PaLM on advanced language proficiency exams across all evaluated languages.Figure 1 reports simulated professional-exam evaluations, with PaLM 2 reaching C2-level testing targets and attaining level A/special A in Japanese versus level F for PaLM; these are not official results.
  • PaLM 2 supports inference-time toxicity control, lower average verbatim memorization than PaLM, and evaluations of multilingual toxicity, harms, biases, and representation in pre-training data.The report also uses canary sequences to measure memorization across languages and finds tail-language memorization exceeds English only when data is repeated several times across documents.

2 Scaling law experiments

The experiments find that model size and training data should scale in equal proportions as compute increases, consistent with Hoffmann et al. (2022). This compute-optimal training-loss scaling does not always yield the best downstream performance, which also depends on throughput and serving latency.

  • 2.1 Scaling laws: Model size and training data should grow in equal proportions as the FLOPs budget increases, closely matching Hoffmann et al. (2022).The result was obtained across four compute budgets from 1×10^19 to 1×10^22 FLOPs using quadratic fits to smoothed validation loss.
  • 2.1 Scaling laws: Models approximately following the FLOP-dependent optimal parameter sizes achieve the lowest training loss in the scaling-law study.Models from 400M to 15B were evaluated at 1×10^20, 1×10^21, and 1×10^22 FLOPs; these sizes and FLOPs apply only to the scaling-law study, not PaLM 2 models.
  • 2.2 Downstream metric evaluations: The downstream-optimal size for a 1×10^22-FLOP model is around 9.5B, although it slightly underperforms the 16.1B model on downstream tasks.Thus, training loss is a useful but imperfect proxy for task performance, while throughput and serving latency also influence model-size choices.

3 Training dataset

PaLM 2 uses a larger, diverse pre-training corpus with substantially more non-English and parallel multilingual data than prior models, alongside quality filtering and longer-context training. The dataset also supports toxicity control through special tokens without harming unrelated-task performance.

  • PaLM 2’s corpus combines web documents, books, code, mathematics, and conversational data, and is significantly larger than PaLM’s training corpus.
  • A higher proportion of non-English data and parallel source–target pairs spanning hundreds of languages improve multilingual understanding, generation, and translation.The parallel pairs have English on one side, and no languages were explicitly filtered in or out.
  • Significant English-evaluation improvements despite less English data are attributed partly to higher mixture quality after de-duplication, sensitive-PII removal, and filtering.
  • Special toxicity-control tokens derived from a fixed Perspective API version enable inference-time control without negatively affecting unrelated-task performance.
  • PaLM 2 substantially increases context length beyond PaLM, enabling long-dialog, long-range reasoning, comprehension, and summarization capabilities without hurting generic-benchmark performance.

4 Evaluation

This section evaluates PaLM 2 across multilingual language proficiency, academic, reasoning, coding, translation, and responsible-AI tasks. It examines performance gains and efficiency relative to comparison systems alongside memorization, toxicity, representational-harm, and language-specific weaknesses.

  • 4.2 Classification and question answering: PaLM 2-L achieves large improvements over PaLM across almost all English QA and classification tasks, with particularly strong gains on ANLI, ReCoRD, and RACE.The smallest PaLM 2 variant is competitive with PaLM 540B, while PaLM 2-M consistently outperforms PaLM.
  • 4.2 Classification and question answering: All PaLM 2 variants outperform PaLM on multilingual TyDi QA in both Gold Passage and no-context settings, with especially pronounced gains for limited-data and non-Latin-script languages.In no-context evaluation, PaLM 2-L clearly outperforms all comparison models; in Gold Passage evaluation, differences among PaLM 2 sizes are relatively small.
  • 4.3 Reasoning: PaLM 2 outperforms PaLM across all reasoning datasets and achieves results competitive with GPT-4, while establishing a new state of the art on multilingual XCOPA without chain-of-thought prompting.On BIG-Bench Hard, PaLM 2 improves over PaLM on every task, often by a large margin; several tasks improve by more than 100%.
  • 4.3 Reasoning: PaLM 2 dramatically outperforms PaLM on MATH, GSM8K, and MGSM, exceeding Minerva and GPT-4 on GSM8K and surpassing the state of the art on MGSM without self-consistency.On MATH, PaLM 2 is competitive with Minerva’s state-of-the-art performance.
  • 4.4 Coding: PaLM 2-S* outperforms PaLM-540B-Coder on all coding benchmarks, often significantly, despite being dramatically smaller, cheaper, and faster to serve.PaLM 2-S* is created by continuing PaLM 2-S training on a code-heavy, heavily multilingual mixture while preserving natural-language performance.
  • 4.4 Coding: PaLM 2-S* outperforms PaLM on all but two BabelCode languages, improving over PaLM-Coder-540B by 6.3× on Haskell and 4.7× on Julia.Java, JavaScript, and TypeScript performance is higher than Python, while low-resource languages show little degradation.
  • 4.5 Translation: PaLM 2 improves translation quality over both PaLM and Google Translate on WMT21 and improves over both systems in all FRMT locales.WMT21 uses professional-translator MQM evaluations, where lower average errors per segment indicate better results; FRMT evaluates region-specific dialect translation with BLEURT.
  • 4.5 Translation: In translation, zero-shot PaLM 2 outperforms PaLM and Google Translate on gender agreement in Spanish, Polish, and Portuguese, but underperforms PaLM in Telugu, Hindi, and Arabic.The evaluation covers 13 languages and uses human raters for gender agreement and translation quality.
  • 4.6 Natural language generation: PaLM 2-L improves over PaLM on multilingual natural-language generation by 59.4% on XSum to 100.8% on WikiLingua, with even its smallest versions outperforming PaLM.Results use one-shot evaluation across typologically diverse languages and report ROUGE-2 or SentencePiece-ROUGE-2 metrics.
  • 4.6 Natural language generation: PaLM 2 shows slight toxicity improvements over PaLM in RealToxicityPrompts language modeling but slight regressions in conversational language modeling on ParlAI Dialogue Safety.These findings concern open-ended generation without prompting designed to approximate anticipated downstream use.
  • 4.6 Natural language generation: Dialog prompting substantially reduces toxic-language harms relative to decontextualized language modeling, although toxic-response rates range from 0%–3.5% in the best cases to 1%–17.9% in the worst cases.This pattern holds on ParlAI Dialogue Safety and Multilingual Representational Bias, with variation across languages and identity terms.
  • 4.6 Natural language generation: PaLM 2 achieves 91.4% accuracy on disambiguated social-identity questions, but 3% of such questions produce representational harm by reinforcing social bias.Qualitative analysis indicates that hallucinations may create salient harms not captured by classification or regression bias measures.
  • 4.7 Memorization: PaLM 2 significantly reduces average verbatim memorization compared with PaLM, including fewer memorized sequences than PaLM’s least-memorizing model.The comparison uses 10,000 shared English training documents, 50-token prompts, and 50-token target suffixes across three model sizes.
  • 4.7 Memorization: Memorization is much lower for sequences repeated only a few times, but PaLM 2 has a higher likelihood than PaLM of memorizing n-grams repeated more than a few times.The authors hypothesize that deduplication makes repeated n-grams rarer yet present in more unique contexts, increasing memorization likelihood.
  • 4.7 Memorization: Tail-language canaries require fewer repetitions for extraction, whereas real training data shows no strong correlation between language size and extraction rate, despite occasional high memorization for highly repeated tail-language sequences.The discussion cautions that reported rates estimate memorization rather than fully characterize downstream privacy attacks, which may exploit additional safeguards or side-channel context.

5 Responsible usage

PaLM 2 responsible usage requires distinguishing pretrained-model evaluations from evolving, processed products and applying ongoing safety review, monitoring, and mitigation. Inference-time control can reduce toxic generation, but application-specific evaluation and safeguards remain essential because harms and effectiveness vary by context.

  • 5 Responsible usage: Google’s policies exclude harmful technologies, weapons that facilitate injury, surveillance violating accepted norms, and technologies contravening international law and human rights.Where material risk exists, development proceeds only when benefits substantially outweigh risks and appropriate safety constraints are incorporated.
  • 5 Responsible usage: PaLM 2 applications undergo ethics review, mitigation recommendations, rigorous testing, and production monitoring, but pretrained models can generate bias, misinformation, hallucinations, and privacy harms.User-facing products may add processing and evolve over time, so their performance should not be expected to match the reported pretrained-model results.
  • 5.1 Inference-time control: Control tokens significantly influenced toxic-continuation probability, reducing or increasing toxic generation from baseline across input-toxicity strata.The intervention used toxicity labels on only a small fraction of pre-training tokens and showed no clear alignment tax or penalty on other evaluations.
  • 5.1 Inference-time control: Specialized downstream mitigations remained more effective than general-purpose inference-time controls, supporting application-specific safeguards targeting multiple harms beyond toxicity.Relevant methods include additional fine-tuning, response filtering, sample-and-rank classifiers, and classifier-in-the-loop decoding.
  • 5.1 Inference-time control: 30% to 12% and 18% to 7%: control tokens reduced toxic responses in standard and adversarial conversational datasets, respectively.For dialog uses, prompting alone was more effective than control tokens, with small additional gains from combining them only on the standard dataset.
  • 5.2 Recommendations for developers: Downstream developers should assess harms and bias in their specific applications because prompts and decoding can substantially change outputs, and toxicity reductions may not generalize.Application-specific analysis and evaluation of potential harms is critical despite the paper’s model-level evaluations.

6 Conclusion

PaLM 2 is a state-of-the-art model that outperforms PaLM across diverse language and reasoning tasks while using less inference compute. The conclusion attributes these gains to scaling, architecture and objective improvements, and data mixture design.

  • PaLM 2 significantly outperforms PaLM across English, multilingual language understanding, and reasoning tasks while using significantly less inference compute.
  • PaLM 2 independently verifies scaling laws at large scales, with training tokens growing roughly in proportion to model parameters.
  • Architecture improvements, varied objectives, and the data mixture were all important to PaLM 2’s performance gains.
  • Even a minor share of translation pairs brought PaLM 2 to parity with production translation services, while training a smaller model on more tokens was generally more efficient than making it more inference-efficient.
  • Further scaling of model parameters and dataset size and quality, alongside architectural and objective improvements, is expected to yield continued gains in language understanding and generation.

7 Authorship, attribution, and acknowledgements

The section attributes PaLM 2 to contributors organized across specialized workstreams, with roles distinguishing leadership, significant contributions, and partial involvement. It also acknowledges reviewers, colleagues, infrastructure teams, and supporting contributors.

  • Large Model Training: Large Model Training credits Andrew M. Dai, David R. So, Dmitry Lepikhin, Jonathan H. Clark, Maxim Krikun, Melvin Johnson, Nan Du, Rohan Anil, Siamak Shakeri, Xavier Garcia, Yanping Huang, Yi Tay, Yong Cheng, Yonghui Wu, Yuanzhong Xu, Yujing Zhang, and Zachary Nado as core contributors.
  • Pre-training Data and Mixture Workstream: The Pre-training Data and Mixture Workstream names Siamak Shakeri as lead, with core and contributing collaborators including Andrew M. Dai, Bryan Richter, Nan Du, Alex Polozov, and Zhongtao Liu.
  • Architecture and Modeling Workstream: Architecture and Modeling credits Andrew M. Dai, Dmitry Lepikhin, and Yi Tay as co-leads, while separately listing evaluation and Responsible AI workstream leadership and contributors.The listed evaluation co-leads are Gaurav Mishra, Jonathan H. Clark, Mark Omernick, and Sebastian Ruder; Responsible AI co-leads are Kathy Meier-Hellstern and Kevin Robinson.
  • Fine-tuning Workstream: Melvin Johnson leads the Fine-tuning Workstream, supported by core contributors including Abe Ittycheriah, Aditya Siddhant, Andy Coenen, Daniel Sohn, and others.
  • Infrastructure Workstream: Infrastructure contributions span Long Context, Optimization, Code Pre-training, and Serving workstreams, each naming co-leads and additional core or contributing collaborators.Named co-leads include Melvin Johnson and Siamak Shakeri; Emanuel Taropa and Rohan Anil; Emanuel Taropa, Eric Chu, Kefan Xiao, and Rohan Anil; and Yanping Huang, respectively.
  • Technical Report: The Technical Report credits Andrew M. Dai, Jonathan H. Clark, Kevin Robinson, Maysam Moussalem, Sebastian Ruder, Siamak Shakeri, and Jacob Austin.
  • Technical Program Management and Product Management: Technical Program Management and Product Management credits Erica Moreira and Paige Bailey as leads, with Eric Ni and Lucas Gonzalez as program or product managers.
  • Cross-workstream Leadership: Cross-workstream leadership identifies overall leaders, defines Lead, Core Contributor, and Contributor roles, states that contributions are equal within roles, and thanks reviewers, colleagues, and Google infrastructure teams.Individuals are ordered alphabetically by first name within each role, and that ordering does not indicate contribution order.

A Detailed results … B Examples of model capabilities

The detailed results cover scaling laws, instruction tuning, multilingual reasoning, coding, natural-language generation, and illustrative model outputs. PaLM 2 shows especially strong multilingual commonsense reasoning, including gains on under-represented languages and performance exceeding PaLM with chain-of-thought prompting.

  • A.1 Scaling laws: Scaling-law curves estimate optimal parameter and token counts from isoflop experiments spanning 1×10^19 to 1×10^22 FLOPs.Quadratic fits estimate minimum validation loss, while a power-law fit extrapolates to larger compute scales; leave-one-out estimates provide error bars.
  • A.1 Scaling laws: At 1×10^22 FLOPs, models of different sizes are evaluated on downstream tasks, with roughly equal validation loss corresponding to approximately equal downstream performance.The results are reported for an architecture and data mixture different from PaLM 2, using 1-shot evaluation unless otherwise noted.
  • A.2 Instruction tuning: Fine-tuning on the Flan dataset improves instruction following and performance on unseen tasks, with consistent gains across the reported evaluation benchmarks.Flan contains more than 1,800 tasks and at least 20 instruction templates per task.
  • A.4 Coding: BabelCode coding results are reported as greedily sampled pass@1 rates for PaLM and PaLM-2 experiments.The raw pass@1 results are provided in Table 18.
  • A.3 Multilingual commonsense reasoning: PaLM 2 significantly outperforms PaLM across all evaluated XCOPA languages and particularly improves on Swahili, Quechua, and Haitian, even exceeding PaLM with chain-of-thought prompting.The comparison uses standard prompting for PaLM 2 versus both standard and chain-of-thought prompting for PaLM.
  • A.5 Natural language generation: Per-language natural-language-generation results are reported both on the full evaluation data and on filtered data.The per-language results appear in Tables 19 and 20.
  • B Examples of model capabilities: The paper presents sample outputs illustrating PaLM-2 capabilities, mostly from an instruction-following fine-tuned variant.These examples are intended to demonstrate model capabilities rather than provide aggregate benchmark results.

B.1 Multilinguality · B.2 Creative generation · B.3 Coding

PaLM 2 demonstrates multilingual understanding, generation, and transformation across languages, dialects, scripts, and registers. The examples also show creative generation in diverse languages and coding assistance with multilingual comments.

  • B.1 Multilinguality: PaLM 2 explains jokes, puns, and haikus in Japanese in a zero-shot multilingual setting.Figure 12 provides examples of these capabilities.
  • B.1 Multilinguality: PaLM 2 explains translation ambiguities in German and Swahili by identifying underlying meanings and supplying more appropriate translations.The examples focus on idiomatic phrases and translation rationale.
  • B.1 Multilinguality: PaLM 2 performs zero-shot translation between dialects by identifying dialect-specific terms and replacing them with equivalents in another variety.Examples include European-to-Argentinian Spanish and Brazilian-to-European Portuguese.
  • B.1 Multilinguality: PaLM 2 follows multilingual instructions to expand abbreviations, fix typos, transfer formal text into colloquial chat, and transliterate text into Latin script.These demonstrations use short instructions, including cases where the target language is not specified.
  • B.1 Multilinguality: PaLM 2 generates coherent, well-written Persian email text from English instructions, then follows instructions to transliterate and translate it into Arabic.The generated content is described as high quality and consistent with both the original email and requested transformations.
  • B.2 Creative generation: PaLM 2 generates text in Tajiki and Persian despite their different scripts, suggesting understanding of both languages’ grammatical and semantic structures.Tajiki uses Cyrillic, whereas Persian uses a variation of Arabic script.
  • B.2 Creative generation: PaLM 2 generates appropriate and fluent creative text in diverse forms and languages, including an Armenian-name-based children’s game, Taiwanese speech, German dialect explanations, Persian email, Tajiki prose, and Tamil poetry.The examples include languages that are under-represented in pre-training data or less commonly written.
  • B.3 Coding: PaLM 2 demonstrates multilingual coding by designing a simple website, fixing code with line-by-line Korean comments, and generating JavaScript with Malayalam comments and usage examples.The examples cover webpage design, debugging, and a palindrome-checking function.

C Language proficiency exams · D Responsible AI

The paper evaluates advanced multilingual language proficiency through standardized exams under simulated zero-shot conditions, then examines PaLM 2’s responsible AI behavior and cautions against equating pre-trained-model results with user-facing products.

  • C Language proficiency exams: Advanced Chinese, Japanese, French, Spanish, and Italian proficiency was evaluated using publicly available practice or past exams.The exams were HSK 7–9, J-Test Level A, TCF C2, DELE C2, and PLIDA C2.
  • C Language proficiency exams: The Chinese HSK 7–9 evaluation used purchased third-party study material, included reading and listening content, and excluded speaking despite the exam’s broader format.Available listening transcripts were additionally used as reading questions.
  • C Language proficiency exams: All proficiency questions were administered zero-shot, with temperature 0.3 for multiple choice and 0.6 for free-text responses.Three native-speaker contractors graded writing responses, and their scores were averaged.
  • C Language proficiency exams: Scores weighted reading and writing equally and used official pass thresholds, including 60% for HSK, DELE, and PLIDA, 85.7% for TCF C2, and 90% for J-Test Level A.The paper stresses that these are simulated, unofficial, and unaccredited exam results.
  • C Language proficiency exams: Contamination checks excluded German practice and French writing results, while removing contaminated questions yielded 79% Chinese and 87% Spanish scores without changing Italian or Japanese scores.The check used normalized 25-gram overlaps against pre-training data and hand-checked matches.
  • D Responsible AI: The responsible AI section analyzes PaLM 2 pre-training data, multilingual toxicity classification, downstream harms and biases, and representation of people.It presents the section as an evaluation of model behavior across several responsible-AI-focused benchmarks.
  • D Responsible AI: Responsible-AI results apply to pre-trained models rather than necessarily to fine-tuned variants or user-facing products, which may add processing and evolve over time.The analyses are intended to help developers identify application-specific harms and prioritize procedural and technical safeguards early.

D.1 Dataset analysis

PaLM 2’s pre-training data analysis finds substantial differences in representations of people across identity, gender, and toxicity signals, while emphasizing that automated measurements have important limitations. The analysis also identifies Western and male skews alongside elevated toxicity in documents referencing certain identities.

  • Methods: The analysis used Know Your Data and classifiers on optimized training-data formats, reporting relative frequencies and generally omitting subset values at or below 5%.For example, English pronouns occurred in 66% of optimized English documents, and female pronouns occurred in 48% of that subset; single-digit percentages still represent substantial token counts.
  • Methods: The study measured identity mentions, English gender and grammatical-person pronouns, and English toxicity probabilities using string matching and an English-only Perspective signal.Identity signals may include non-person entities and were intended to support multilingual text.
  • Discussion: Female references were 30% versus 10% for male references, while marked non-binary references remained below the 5% reporting threshold.The analysis found women under-represented relative to men across most signals and English gender pronouns.
  • Discussion: Marked sexuality references appeared in 3% of documents, with “gay” comprising 53% and “homosexuality” 22% of those references.The analysis also found a Western skew in race, religion, ethnicity, and nationality references, with “americans” comprising 42% of references.
  • Limitations: Automated identity measurements struggle with assumed or unmarked identities and encoded social contexts, while the validity of some measures varies with signal reliability.The authors therefore call for continuing measurement and research because links between pre-training data and downstream harms vary across designs, systems, and domains.
  • Discussion: Documents mentioning “white people” had the highest toxicity probability at 0.250, followed by “transsexual” at 0.248.The toxicity analysis included identities exceeding overall toxicity and occurring in more than 0.1% of English pre-training data; sexuality-related identities generally had higher toxicity.

D.2 Evaluation approach · D.3 Dialog uses

The evaluation framework prioritizes downstream uses and evaluates potential harms and bias within realistic sociotechnical contexts. For dialog applications, it uses generic Assistant prompting and multiple-response sampling while recognizing additional mitigation methods can change outcomes.

  • D.2 Evaluation approach: The framework prioritizes downstream uses by approximating how developers build systems, including practical tradeoffs involving efficiency, costs, and workflows.It focuses on four system types: dialog, safety classification, translation, and question answering.
  • D.2 Evaluation approach: Potential harms are conceptualized as adverse lived experiences arising from misuse, system failures, or proper use that causes harm or amplifies inequalities.The evaluation emphasizes representational harms, including toxic language, stereotypes, discrimination, and exclusionary norms, plus memorization as a privacy-harm proxy.
  • D.2 Evaluation approach: Evaluations measure harms observable during each task, primarily through extrinsic measures, while separately testing bias proxies and their relationships to downstream-use harms.These measures are limited approximations rather than exhaustive or holistic assessments.
  • D.2 Evaluation approach: The framework prioritizes automated evaluations and offers measurement-quality rubrics to support improvements in validity, relevance, inclusivity, and attention to cumulative harms affecting vulnerable groups.Evaluating risks is difficult because language models operate across complex sociotechnical systems and harms vary across individuals, communities, societies, and sociocultural contexts.
  • D.2 Evaluation approach: The evaluation covers core capabilities and Responsible AI risks across dialog, classification, translation, question answering, language modeling, and memorization.It also begins extending Responsible AI evaluations across multiple languages.
  • D.3 Dialog uses: For dialog applications, PaLM 2 is evaluated with generic “Assistant” prompting adapted from Glaese et al. (2022) and cues intended to mitigate potential harms.This setup approximates a common dialog-system construction approach.
  • D.3 Dialog uses: Some dialog evaluations sample multiple responses per prompt to measure harms across possible outputs before additional response filtering.This approximates a common practical dialog-system workflow (Thoppilan et al., 2022; Glaese et al., 2022).
  • D.3 Dialog uses: Dialog systems may include additional fine-tuning, instruction fine-tuning, or reinforcement learning, so the evaluated prompting setup does not represent every safety-mitigation pipeline.Examples include Thoppilan et al. (2022), Ouyang et al. (2022), Rae et al. (2021), Glaese et al. (2022), Ganguli et al. (2022), and Bai et al. (2022).

D.3.1 Toxic language harms in dialog generation

PaLM 2’s dialog prompting reduces toxic degeneration for single sampled responses, but this mitigation weakens with repeated sampling, especially for adversarial and explicitly toxic prompts.

  • D.3.1 Toxic language harms in dialog generation: The study evaluates toxic degeneration using ParlAI dialog transcripts with balanced standard and adversarial query sets, comparing dialog-prompted PaLM 2 with language-modeling PaLM 2, PaLM, and LaMDA.The standard dataset is explicitly toxic, while the adversarial dataset contains more implicit and subtly harmful language.
  • D.3.1 Toxic language harms in dialog generation: The evaluation samples 20 continuations per query with top-k sampling (k = 40, temp=1.0) and counts prompts producing at least one response with Perspective API toxicity score > 0.5.Results span 1 to 20 samples and approximate repeated usage across end users; the fixed Perspective API may miss implicit, subtle, and contextualized harms.
  • D.3.1 Toxic language harms in dialog generation: For a single sample, dialog-prompted PaLM 2 performs similarly to LaMDA, while prompting substantially lowers toxic degeneration relative to language-modeling evaluation.PaLM 2 and PaLM show no major difference in conversational language-modeling toxic degeneration, indicating the reduction is influenced by prompting techniques.
  • D.3.1 Toxic language harms in dialog generation: With 20 samples, toxic responses occur for over 30% of adversarial queries and almost 60% of standard queries, showing prompting is insufficient for robust harm mitigation.Standard interactions are explicitly toxic, whereas adversarial interactions contain more implicit and subtle harmful language.

D.3.2 Multilingual toxic language harms and representational bias … D.5.1 Translating to English

Across multilingual safety evaluations, PaLM 2 reduces toxicity relative to PaLM but still exhibits language- and identity-dependent harms, while zero-shot translation into English shows no major misgendering differences from PaLM 540b. These findings motivate disaggregated evaluation, application-specific mitigation, and caution about automated cross-language measurements.

  • D.3.2 Multilingual toxic language harms and representational bias: The multilingual bias evaluation extends prior representational-bias benchmarking across 16 languages and dialog contexts using identity-targeting questions and imperatives plus neutral subgroup baselines.Queries cover ethnicity, religion, and gender, and target systems that should produce no toxic harms.
  • D.3.2 Multilingual toxic language harms and representational bias: Toxic responses range from 1% to 17.9% in English, German, and Portuguese, versus 0% to 3.5% in other languages, with identity-group variation also higher in the outlier languages.The corresponding standard deviations across identity groups are 1.2%–6.2% and 0%–1.3%, respectively; Figure 31 presents the cross-language comparison.
  • D.3.2 Multilingual toxic language harms and representational bias: English, German, and Portuguese show higher toxicity for particular identity groups, including Black and White, Judaism and Islam, and Male and Female references.Figure 32 disaggregates toxicity rates by identity group and language.
  • D.4 Classification uses: The safety analysis prioritizes zero-shot and few-shot classifier evaluations because classifier harms vary with measurement, domain, and product context, while fine-tuning and prompt-tuning remain future work.This scope includes toxicity classification but does not cover the more common fine-tuning or parameter-efficient classifier workflows.
  • D.4.1 Toxicity classification with CivilComments: PaLM 2 shows significant performance gains over PaLM in both 0-shot and 10-shot scenarios.The evaluation uses 10,000 random test-split samples, balanced toxic and non-toxic few-shot examples from the training split, and AUC-ROC based on normalized positive-continuation log-likelihood.
  • D.4.2 Multilingual toxicity classification with Jigsaw Multilingual: Multilingual toxicity classification uses Jigsaw comments in six languages, English vehicle prompts, same-language few-shot examples, and normalized positive-continuation likelihood scored by AUC-ROC.The dataset contains 50,261 comments, with more than 7,000 samples in its least represented language.
  • D.4.2 Multilingual toxicity classification with Jigsaw Multilingual: PaLM 2 outperforms PaLM in multilingual toxicity classification across all analyzed languages in zero-shot evaluation and remains better overall in the 10-shot setting.The advantage is larger zero-shot and narrower with 10-shot examples.
  • D.5.1 Translating to English: Zero-shot translation into English shows no major misgendering differences from PaLM 540b, with a minor improvement in disaggregated worst-case performance across 26 source languages.The evaluation contains 1,954 passages and scores whether translations avoid potential gender-expression errors; Hindi-to-English performs worst.

D.5.2 Translating from English

Human evaluation of PaLM 2’s zero-shot translation out of English finds variable gender agreement, higher potential misgendering harms than translating into English, and quality linked imperfectly to multilingual training-data representation.

  • D.5.2 Translating from English: Automated gender-pronoun matching was unsuitable because languages encode gender across different linguistic elements, and early automated evaluations were noisy and unreliable.Consequently, the study used human evaluation to assess gender accuracy and coherence in PaLM 2’s zero-shot translations.
  • D.5.2 Translating from English: Human evaluators rated gender agreement and translation quality for zero-shot translations across 13 languages using two independent 0–3 scales.The evaluation used passages from the gender sets evaluation set, translated by professional translators, with scores averaged across two raters.
  • D.5.2 Translating from English: PaLM 2 outperforms PaLM on gender agreement in some languages and Translate in Spanish, Polish, and Portuguese, but has lower agreement than PaLM in Telugu, Hindi, and Arabic.Italian and Japanese show improved translation quality despite lower gender agreement scores; overall, zero-shot translation out of English presents higher potential misgendering harms than translation into English.
  • D.5.2 Translating from English: Most languages comprising over 2% of training data achieve average translation-quality scores above 2.50, although Arabic is a notable exception.The relationship between translation quality and web-document pre-training representation suggests multilingual data helps mitigate harms but does not ensure quality across languages.
  • D.5.2 Translating from English: The evaluation excludes non-binary gender harms, contested cross-cultural pronoun perspectives, wider sociotechnical context, and many potential risks, while out-of-English assessment remains challenging and time-intensive.The study evaluates only a subset of risks and focuses on model outputs rather than the broader contexts in which instruction-finetuned systems operate.

D.6 Question answering uses · D.7 Language modeling

PaLM 2’s generative QA evaluation adapts BBQ to expose representational harms beyond multiple-choice answers, finding high accuracy in disambiguated contexts but severe failures on ambiguous prompts. The accompanying language-modeling analysis focuses on representational and toxic-language harms and their relationship to downstream-use harms.

  • D.6 Question answering uses: The adapted BBQ evaluation presents contexts and questions without multiple-choice options and analyzes generated text directly to capture additional bias and representational harms.BBQ covers 9 social-dimension categories and was originally designed for multiple-choice QA.
  • D.6 Question answering uses: 91.4% overall accuracy on disambiguated questions coexisted with 38.2% of errors reinforcing harmful social bias, affecting 3% of all disambiguated questions.The evaluation found no systematic tendency for errors always to reinforce harmful bias.
  • D.6 Question answering uses: Correct disambiguated responses aligned with social bias 2.2 percentage points more often than anti-biased responses, while Other responses comprised 22.5% of errors.String search identified responses naming either individual; the remaining 14% of responses were hand-coded.
  • D.6 Question answering uses: 0.6% accuracy on ambiguous contexts showed that PaLM 2 often named a bias target or non-target despite insufficient information, with biased responses occurring 15.3 percentage points more often.Bias-aligned answers exceeded anti-biased answers across every identity category except disability status.
  • D.6 Question answering uses: 86 queries, about 0.5% of all queries, introduced unexpected harmful biases across 7 of the 9 sensitive-topic categories tested.The qualitative analysis found biases unrelated to the explicitly tested bias, motivating broader measurement of representational harms.
  • D.6 Question answering uses: Qualitative examples included responses attributing academic success to “the white students,” using gendered insults, and claiming “not fat people” frequently play sports.These examples illustrate biases introduced beyond the prompt’s explicitly tested social dimension.
  • D.7 Language modeling: The language-modeling analysis focuses on representational and toxic-language harms and examines how these measures relate to potential harms in downstream uses.Prior work has reported weak relationships between these measures, while other work found bias patterns can persist through adaptation, including instruction fine-tuning.

D.7.1 Open-ended generation … D.10 Model Card

This section examines PaLM 2’s safety and translation-related evaluations, including open-ended generation, conversational safety, and gender-agreement annotation. It also documents evaluation and annotation limitations, dataset maintenance and downstream-harm concerns, and model-card status and usage restrictions.

  • D.7.1 Open-ended generation: PaLM 2 has slightly lower toxic degeneration than PaLM across evaluation variants, with continuations less toxic than prompts and the human baseline across prompt-toxicity strata.This pattern is consistent across checkpoints, models, and full versus small variants.
  • D.7.1 Open-ended generation: During training, PaLM 2 (L) maintained mean TPC of 0.0676±0.0026 and worst-case performance of 0.0725 over 87% of epochs in the tiny evaluation variant.The tiny-variant level was slightly lower than evaluation on the small variant.
  • D.8 Measurement quality rubrics: The responsible-AI appendices recommend the small open-ended evaluation and better-designed evaluation sets to improve measurement quality, continuous training evaluation, and ablation signals.The measurement-quality rubrics communicate and improve evaluation practice, drawing on prior evaluation literature and this work’s experience.
  • D.7.2 Conversational language modeling: The conversational evaluation found a slight PaLM 2 (L) degradation relative to PaLM, while adversarial prompts produced less toxic degeneration than explicitly unsafe standard prompts.Unsafe responses were measured from the first dialogue act using Perspective API, with scores above 0.5 deemed unsafe.
  • D.9 CrowdWorksheets: CrowdWorksheets document evaluation datasets as a step toward standardizing transparency, including Civil Comments, which contains approximately 2 million crowdsourced, multilabel comments.Civil Comments includes train, test, and validation splits and labels such as toxicity, identity attack, and sexually explicit content.
  • What are the precise instructions that were provided to annotators?: Civil Comments annotation judgments are subjective and context-dependent, with cultural familiarity affecting identity-label recognition and potentially harmful content posing psychological risk to annotators.Annotators were informed about the risk, and English proficiency was the only eligibility criterion; representation of affected communities was unknown.
  • Platform and Infrastructure Choices: Figure Eight annotation paid 1.5 cents per judgment, yielding reported hourly earnings of $0.90–$5.40 and an average pay-satisfaction score of 3.8 out of 5.The platform supported feedback and contesting incorrect test questions, while payment targeted competitive local wages.
  • Dataset Analysis and Evaluation: The dataset analysis found no disagreement analysis, while noting that toxicity perceptions and annotation utility may change as language and social views evolve; misgendering evaluation used gender sets with zero-shot PaLM 2, PaLM, and Google Translate comparisons.The gender-set evaluation was built from translation-system error analysis and professionally translated examples.
  • (1) Gender Agreement and Coherency Evaluation: Gender agreement is rated 0–3 from completely incorrect to correct; gender-neutral targets score 3 for gender-specific sources when no other gender errors occur.Assessment covers nouns, names, pronouns, possessives, adjectives, and gender-inflected words; annotators answer “Can’t tell” when judgment is impossible.
  • (2) General Quality Evaluation: General translation quality is rated 0–3 from near-total meaning loss to fully preserved meaning with correct grammar.Instructions were iteratively refined to improve inter-annotator agreement, but intermediate scores and non-binary gender agreement remain culturally subjective.
  • Selecting Annotators: Annotators were required only to be proficient in English and the target language, while gender identity and lived experience may influence judgments and affected communities were not represented.The task could expose annotators to misgendering, and researchers assumed sufficient bilingual fluency for accurate scoring.
  • Platform and Infrastructure Choices: Annotations were conducted on Venga, selected for translation-data experience; annotators could request clarifications and discuss suspected annotation errors.The supplier was paid an average of $42 per hour, with language-dependent costs and rates set by location and experience.
  • Dataset Analysis and Evaluation: Quality checks used vendor guidelines, linguist disagreement resolution, and review of languages with outlier aggregate results, but no systematic disagreement analysis was performed.The dataset may become less valid as language and social views change, with no current update plans.
  • D.10 Model Card: The PaLM 2 model card (Mitchell et al. (2019)) describes a static model trained offline from random initialization and warns against harmful or insufficiently analyzed downstream use.PaLM 2 supports open-ended text generation, was not tested outside research settings affecting performance, and should not be used to cause harm.
Loading 2305.10403v3…