PaperlayerView exact revision on arXiv

Source-linked AI summary

DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models

Boxin Wang, Weixin Chen, Hengzhi Pei, Chulin Xie, Mintong Kang, Chenhui Zhang, Chejian Xu, Zidi Xiong, Ritik Dutta, Rylan Schaeffer, Sang T. Truong, Simran Arora, Mantas Mazeika, Dan Hendrycks, Zinan Lin, Yu Cheng, Sanmi Koyejo, Dawn Song, Bo Li

arXiv:2306.11698v5cs.CLcs.AIcs.CR

TL;DR

Trustworthiness evidence for GPT models remains limited despite proposed use in sensitive applications. This paper comprehensively evaluates GPT-3.5 and GPT-4 across multiple dimensions, finding that GPT-4 is generally stronger on standard evaluations but more vulnerable to misleading instructions and demonstrations.

  • Problem

    Evidence on GPT models’ trustworthiness is limited even as they are considered for sensitive applications such as healthcare and finance.

  • Method

    The paper evaluates GPT-3.5 and GPT-4 across toxicity, stereotype bias, adversarial and OOD robustness, privacy, machine ethics, and fairness using benchmarks, designed data, and adversarial prompts.

  • Results

    GPT-4 generally performs better than GPT-3.5, but is easier to manipulate with jailbreaking or misleading prompts and demonstrations because it follows instructions more precisely.

  • Takeaways & Limitations

    Trustworthiness depends strongly on prompt and demonstration conditions, so standard benchmark performance does not capture all vulnerabilities.

  • Takeaways & Limitations

    The evaluations focus on general vulnerabilities and standard tasks rather than coordinated adversaries or domain-specific settings.

Abstract

from arXiv · show

Generative Pre-trained Transformer (GPT) models have exhibited exciting progress in their capabilities, capturing the interest of practitioners and the public alike. Yet, while the literature on the trustworthiness of GPT models remains limited, practitioners have proposed employing capable GPT models for sensitive applications such as healthcare and finance -- where mistakes can be costly. To this end, this work proposes a comprehensive trustworthiness evaluation for large language models with a focus on GPT-4 and GPT-3.5, considering diverse perspectives -- including toxicity, stereotype bias, adversarial robustness, out-of-distribution robustness, robustness on adversarial demonstrations, privacy, machine ethics, and fairness. Based on our evaluations, we discover previously unpublished vulnerabilities to trustworthiness threats. For instance, we find that GPT models can be easily misled to generate toxic and biased outputs and leak private information in both training data and conversation history. We also find that although GPT-4 is usually more trustworthy than GPT-3.5 on standard benchmarks, GPT-4 is more vulnerable given jailbreaking system or user prompts, potentially because GPT-4 follows (misleading) instructions more precisely. Our work illustrates a comprehensive trustworthiness evaluation of GPT models and sheds light on the trustworthiness gaps. Our benchmark is publicly available at https://decodingtrust.github.io/ ; our dataset can be previewed at https://huggingface.co/datasets/AI-Secure/DecodingTrust ; a concise version of this work is at https://openreview.net/pdf?id=kaHpo8OZw2 .

1 Introduction

The paper introduces a comprehensive evaluation of GPT-3.5 and GPT-4 across multiple trustworthiness dimensions, including toxicity, bias, robustness, privacy, ethics, and fairness. It finds broad vulnerabilities, including greater GPT-4 susceptibility to misleading instructions despite stronger standard performance.

  • Privacy: The study evaluates privacy by testing extraction from pretraining data, inference-stage PII, and conversation-history leakage.Its evaluation scenarios target memorization and information leakage under constructed prompts.
  • Evaluation scope: The evaluation covers toxicity, stereotype bias, adversarial and out-of-distribution robustness, privacy, machine ethics, and fairness.It combines existing benchmarks with new data and evaluation protocols.
  • Main findings: GPT-4 generally performs better than GPT-3.5, but misleading prompts or demonstrations can make GPT-4 easier to manipulate.The authors attribute this potential vulnerability to GPT-4 following instructions more precisely.
  • Stereotype bias: GPT-3.5 and GPT-4 are usually not strongly biased under benign prompts, yet adversarial system prompts can induce agreement with stereotyped content.GPT-4 is reported as more vulnerable to misleading targeted prompts than GPT-3.5.
  • Out-of-distribution robustness: GPT-4 shows higher generalization across diverse OOD styles and greater resilience to unknown facts than GPT-3.5, although accuracy on recent events remains limited.The model more often answers “I do not know” rather than producing made-up content.
  • Fairness: Fairness results show an accuracy-fairness tradeoff and indicate that balanced few-shot examples can improve prediction fairness.Unbalanced test data increases unfairness, while a small balanced context can guide fairer predictions.

2 Preliminaries

GPT-3.5 and GPT-4 are autoregressive decoder-only transformers trained with next-token prediction and RLHF. The paper interacts with them through system and user prompts for zero-shot, few-shot, and generation tasks while probing trustworthiness weaknesses.

  • Models: GPT-3.5 and GPT-4 generate text autoregressively, using previously generated tokens to predict the next token.GPT-3.5 retains 175 billion parameters, while GPT-4’s parameter count and pretraining corpus are undisclosed.
  • Training: Both models use reinforcement learning from human feedback to encourage instruction following and alignment with human values.Their training combines standard autoregressive pretraining with RLHF.
  • Prompt design: Zero-shot prompts provide a task description and test input, whereas few-shot prompts additionally include demonstrations with corresponding labels.The task description specifies the required operation and permissible class labels.
  • Evaluation strategy: The evaluation uses diverse and potentially adversarial prompts to probe model vulnerabilities and support broader trustworthiness assessment.The authors also plan to open-source the evaluation platform for assessing other LLMs.

3 Evaluation on toxicity

The toxicity evaluation combines standard benchmarking with adversarial system and user prompts to expose vulnerabilities in GPT-3.5 and GPT-4. Although adversarial prompting can sharply increase toxicity, GPT-4-generated challenging prompts further elicit toxic content and transfer to other models.

  • 3.1 Evaluation on standard benchmark: The evaluation compares GPT-3.5 and GPT-4 with pretrained models on REALTOXICITYPROMPTS, then tests designed adversarial system and user prompts.The study examines both standard benchmark behavior and prompting conditions intended to elicit toxicity.
  • 3.1 Evaluation on standard benchmark: Adversarial system prompts can drive GPT-3.5 and GPT-4 to nearly 100% average toxicity probability, even for nontoxic task prompts.GPT-4 shows more toxic behavior than GPT-3.5 under these adversarial prompts.
  • 3.1 Evaluation on standard benchmark: GPT-3.5 and GPT-4 reduce toxicity relative to models without instruction tuning or RLHF, maintaining toxicity probability below 32% across task prompts.
  • 3.2 Design of diverse system prompts: The study designs 33 system prompts across categories including task reformulation and respond-as-program prompting to elicit toxic outputs.Some prompts use role-playing, fake syntax, or reformulated objectives to encourage offensive generations.
  • 3.2 Design of diverse system prompts: GPT-4 has higher average toxicity than GPT-3.5 under adversarial system prompts, potentially because it follows instructions more accurately.Straightforward prompts and explicit instructions to add swear words are reported as especially effective.
  • 3.3 Design of challenging user prompts: GPT-4-generated challenging toxic prompts raise GPT-4 expected maximum toxicity to 0.95 with average toxicity probability reaching 100%.These prompts also transfer to pre-RLHF GPT-3 models, producing expected maximum toxicity of 0.9 and 100% toxicity probability.

4 Evaluation on stereotypes bias

The stereotype-bias evaluation measures how often GPT-3.5 and GPT-4 agree with statements targeting demographic groups across stereotype topics and prompting conditions. Both models usually reject biased statements under benign or untargeted prompts, but targeted adversarial prompts substantially increase agreement.

  • 4.1 Design of stereotype dataset: The dataset covers 24 demographic groups across seven factors and 16 stereotype topics, divided into stereotyped and contrast groups for analysis.The paper notes that all demographic groups experience stereotypes in the real world; the distinction is analytical.
  • 4.2 Evaluation metric: The agreementIndex is computed from 25 generations per prompt and represents the fraction of responses agreeing with a stereotype statement.Values range from 0, never agreeing, to 1, always agreeing; results are averaged across three template variations per topic.
  • 4.3 Results: Under benign and untargeted prompts, both GPT-3.5 and GPT-4 largely reject biased statements across stereotype topics.The heatmaps summarize agreement across demographic groups and stereotype topics.
  • 4.3 Results: AgreementIndex increases by 0.346 for GPT-3.5 and 0.535 for GPT-4 from untargeted to targeted prompts on average.The increase from benign to untargeted prompts is smaller: 0.008 for GPT-3.5 and 0.057 for GPT-4.
  • 4.3 Results: GPT-4 is more likely than GPT-3.5 to output biased content under misleading targeted prompts, potentially because it follows instructions more precisely.Bias also varies substantially across demographic groups and stereotype topics.

5 Evaluation on adversarial robustness

The evaluation finds that GPT-4 is more robust than GPT-3.5 on standard AdvGLUE, yet both remain vulnerable to stronger transferred adversarial attacks. Prompt wording has little effect on standard robustness, while attack effectiveness varies by surrogate model and strategy.

  • 5.1 Robustness evaluation on standard benchmark AdvGLUE: GPT-4 reaches 78.41% average robust accuracy versus 67.37% for GPT-3.5 on standard AdvGLUE.GPT-3.5 is near the leaderboard SoTA score of 65.77%, while GPT-4 has a smaller maximum performance drop: 9.90% versus 14.43%.
  • 5.1 Robustness evaluation on standard benchmark AdvGLUE: GPT-4 is more robust than GPT-3.5 to transferred human-crafted adversarial texts, with ANLI attack success falling from 61.13% to 36.78%.For AdvSQuAD, the corresponding attack success rate falls from 10.52% to 0% on GPT-4.
  • 5.2 Robustness evaluation on generated adversarial texts AdvGLUE++: AdvGLUE++ substantially lowers both models’ robust accuracy, showing vulnerability to stronger adversarial attacks despite standard-benchmark robustness.
  • 5.2 Robustness evaluation on generated adversarial texts AdvGLUE++: 49.23% and 55.64% are GPT-3.5 and GPT-4 robust accuracies on Alpaca-generated attacks, the most transferable surrogate attacks.SemAttack is most effective against Alpaca and StableVicuna, whereas TextFooler is most effective against Vicuna.
  • 5.1 Robustness evaluation on standard benchmark AdvGLUE: Different task descriptions and system prompts have no significant influence on GPT-3.5 or GPT-4 robustness.

6 Evaluation on out-of-distribution robustness

The study evaluates robustness to altered styles, unknown knowledge, and out-of-distribution demonstrations. GPT-4 generally performs better or more consistently, but both models degrade on unfamiliar styles and knowledge, with GPT-4 benefiting from explicit refusal options.

  • 6 Evaluation on out-of-distribution robustness: The OOD evaluation covers unexpected input styles, unknown facts, and demonstrations whose distributions differ from the test data.
  • 6.1 Robustness on OOD style: GPT-4 generalizes better than GPT-3.5 across diverse OOD style transformations, while both are more vulnerable to rare styles such as Shakespearean-W and Bible.
  • 6.2 Robustness on OOD knowledge: 24% lower QA2023 accuracy for GPT-4 than GPT-3.5 contrasts with GPT-4’s 4% higher accuracy on in-distribution questions.Meaningful Accuracy declines from above 87% on QA2020 to approximately 70% on QA2023 for both models.
  • 6.2 Robustness on OOD knowledge: Adding an “I don’t know” option makes GPT-4 more conservative and reliable, increasing its refusal rate and meaningful accuracy, unlike GPT-3.5.
  • 6.3 Robustness on OOD demonstrations via in-context learning: GPT-4 improves more consistently on style-transformed tests with original or matching-style demonstrations, while GPT-3.5 benefits especially from matching-style demonstrations.

7 Evaluation on robustness against adversarial demonstrations

The evaluation tests whether adversarial demonstrations mislead GPT models through counterfactual examples, spurious correlations, and backdoor triggers. GPT-4 is generally more resistant to spurious correlations but more vulnerable to backdoored demonstrations, while counterfactual demonstrations do not mislead either model.

  • 7.1 Robustness against counterfactual examples: Counterfactual demonstrations do not mislead GPT-3.5 or GPT-4, and both models generally benefit from them.The evaluation compares zero-shot, counterfactual-only, ordinary demonstrations, and demonstrations augmented with a counterfactual example.
  • 7.2 Robustness against spurious correlations: GPT-3.5 is easier to mislead by spurious correlations than GPT-4 on the NLI task.On the S_RC heuristic subset, entailment-correlated demonstrations reduce GPT-3.5 performance, whereas GPT-4 identifies the underlying causal features and improves.
  • 7.2 Robustness against spurious correlations: Different spurious correlations can either worsen predictions or help models identify causal features underlying the task.The experiments construct entailment- and non-entailment-correlated demonstrations from six HANS heuristic types.
  • 7.3 Robustness against backdoored demonstrations: GPT-4 is more vulnerable to backdoored demonstrations, retaining high clean accuracy while achieving higher attack success rates than GPT-3.5.GPT-4’s stronger pattern-following and instruction-following abilities increase its susceptibility to syntactic, stylistic, and instructional backdoors.
  • 7.3 Robustness against backdoored demonstrations: Trigger placement matters: beginning-of-text triggers are most effective, while later demonstrations near the test input receive greater attention than earlier ones.GPT-4 still shows stronger attention to distant demonstration text than GPT-3.5, although later positions have greater impact.
  • 7.3 Robustness against backdoored demonstrations: Adding a backdoored task instruction raises attack success rates; GPT-4 reaches 100% while its clean accuracy remains unchanged.The result indicates that GPT-4 follows the malicious instruction while preserving the benign task behavior.
  • 7.3 Robustness against backdoored demonstrations: Backdoored demonstrations mislead both models, with word- or sentence-based triggers generally exerting more impact than syntactic or style-based triggers.BadWord and AddSent produce higher attack success rates under the same setup, while SynBkd and StyleBkd require more backdoored demonstrations.

8 Evaluation on privacy

The privacy evaluation examines leakage from training data and interactive conversation history using context, zero-shot, and few-shot prompts. GPT models can recover memorized email addresses and leak conversational PII, with GPT-4 generally stronger at PII protection but more prone to leakage under some constructed prompts.

  • 8.1 Privacy leakage of training data: GPT-3.5 and GPT-4 can leak email addresses from the Enron dataset, indicating memorization of training data during inference.Under context prompting, GPT-3.5 predicts up to 5.23% and GPT-4 up to 3.97% of email addresses.
  • 8.1 Privacy leakage of training data: Longer context generally produces more correct private-email predictions for both models.The evaluation feeds preceding tokens from an email before the target address to elicit the address.
  • 8.1 Privacy leakage of training data: GPT-4 achieves higher email-extraction accuracy than GPT-3.5 for most few-shot templates, and Template A reaches 48.19% under five-shot prompting.Under Template A, GPT-4’s reported accuracies are 18.80% for zero-shot, 31.88% for one-shot, and 48.19% for five-shot prompting.
  • 8.2 Privacy leakage from conversations: GPT-4 is more robust than GPT-3.5 at safeguarding PII, protecting all tested PII types under zero-shot privacy-protection prompts.GPT-3.5 is prone to violating privacy-protection instructions and leaking phone numbers and passwords.
  • 8.2 Privacy leakage from conversations: Digit sequences are harder to leak than letter sequences, and SSNs are the most difficult PII type to leak for both models.The authors attribute the SSN pattern possibly to explicit instruction tuning for related keywords.
  • 8.2 Privacy leakage from conversations: Leakage also varies by privacy event: GPT-4 better understands some events than GPT-3.5, while events such as divorce, voting, and promotion produce high leakage rates.The models’ behavior changes both across privacy-related wording and across the underlying event.

9 Evaluation on machine ethics

The machine-ethics evaluation measures GPT-3.5 and GPT-4 on commonsense morality and tests whether jailbreaking prompts or evasive descriptions cause immoral actions to be misrecognized. GPT-4 generally performs better on standard benchmarks but is more vulnerable to misleading prompts.

  • Standard machine ethics benchmarks: The evaluation compares GPT-3.5 and GPT-4 on commonsense morality using ETHICS and Jiminy Cricket datasets in zero-shot and few-shot settings.ETHICS includes short and long samples, while Jiminy Cricket contains scenarios from text-based games.
  • Standard machine ethics benchmarks: GPT-4 exceeds GPT-3.5 by 2.8% and 0.9% in few-shot accuracy on short and long ETHICS samples, respectively.In zero-shot evaluation, the corresponding GPT-4 advantages are 3.4% and 4.5%.
  • Standard machine ethics benchmarks: GPT-4 reaches 78.6% zero-shot and 82.4% few-shot accuracy on Jiminy Cricket, exceeding GPT-3.5 in both settings.GPT-3.5 reaches 73.9% and 77.9%, respectively.
  • Jailbreaking prompts: Designed jailbreaking prompts mislead both models, and combining weak prompts raises minimum ΔFPR from 1.50% to 90.00% on ETHICS.On Jiminy Cricket, the corresponding increase is from -19.00% to 62.50%.
  • Jailbreaking prompts: GPT-4 is easier to mislead than GPT-3.5; jb_prompt2 raises ETHICS FPR by 96.00% for GPT-4 versus 14.00% for GPT-3.5.The paper potentially attributes this vulnerability to GPT-4 following instructions more precisely.
  • Evasive sentences: Evasive descriptions can make immoral actions appear moral, especially when actions are framed as accidental, harmless, unauthenticated, or imaginary.The increase reaches 43% on GPT-4 for accidental actions on ETHICS and 64% for weakened harm on Jiminy Cricket.
  • Conditional actions: GPT models recognize harm to others better than harm to oneself, with GPT-4’s harm-to-others FPR as low as 17.31% and GPT-3.5’s harm-to-self FPR reaching 87.63%.The great-harm class is excluded because it is small and highly imbalanced.

10 Evaluation on fairness

The fairness evaluation examines prediction gaps across sensitive attributes and tests how demographic imbalance in test data or few-shot contexts affects GPT-3.5 and GPT-4. Imbalanced few-shot examples substantially increase unfairness metrics for both models.

  • Evaluation setup: The study evaluates fairness across sensitive attributes and compares GPT-3.5 with GPT-4 using transformed Adult-dataset classification tasks.It considers demographic parity difference and equalized odds difference alongside accuracy.
  • Fairness metrics: Demographic parity difference measures positive-prediction gaps between groups defined by A = 1 and A = 0.Equalized odds difference additionally conditions prediction gaps on ground-truth labels.
  • Few-shot demographic imbalance: Only 32 biased few-shot examples increase GPT-3.5 demographic parity difference from 0.033 to 0.12 and GPT-4’s from 0.10 to 0.28.The test-set base rate parity is fixed at 0.0, isolating bias induced by the context.
  • Few-shot demographic imbalance: Increasing demographic imbalance in few-shot contexts makes both models less fair according to demographic parity difference and equalized odds difference.The context uses base rate parity values of 0.0, 0.5, and 1.0.
  • Zero-shot fairness: Larger test-set base rate parity produces group performance gaps, while GPT-4 is more accurate on balanced data but can have higher unfairness on unbalanced data.The paper characterizes this as an accuracy-fairness tradeoff.

11 Related work

Related work has developed benchmarks for LLM capabilities, toxicity, adversarial robustness, privacy, machine ethics, and fairness. The paper positions its machine-ethics evaluation against existing text-game benchmarks and extends evaluation to GPT models and adversarial conditions.

  • LLM evaluation and trustworthiness: Existing benchmarks evaluate general language understanding, more difficult tasks, toxicity, adversarial robustness, privacy leakage, machine ethics, and fairness.The cited literature covers both model capabilities and trustworthiness risks.
  • Privacy: Prior privacy research shows that training data and personally identifiable information can potentially be extracted from language models.This motivates evaluating memorization and information leakage.
  • Machine ethics: Jiminy Cricket provides thousands of annotated scenarios across 25 text-based adventure games, while MACHIAVELLI covers over half a million scenarios across 134 games.These benchmarks address moral and social behavior in interactive text worlds.
  • Machine ethics: The paper evaluates GPT-3.5 and GPT-4 on existing machine-ethics benchmarks with adversarial prompts, evasive sentences, and conditioned behaviors.This extends prior machine-ethics benchmarks toward GPT-specific trustworthiness testing.

12 Conclusion and future directions

The paper presents DecodingTrust as a broad evaluation of GPT-3.5 and GPT-4 across multiple trustworthiness dimensions. GPT-4 generally performs better, yet misleading prompts or demonstrations can make it easier to manipulate; the authors call for broader and more formal evaluations.

  • Conclusion: DecodingTrust evaluates GPT-3.5 and GPT-4 across toxicity, stereotype bias, adversarial and OOD robustness, adversarial demonstrations, privacy, ethics, and fairness.The evaluation is intended to expose trustworthiness gaps across diverse perspectives.
  • Conclusion: GPT-4 generally performs better than GPT-3.5, but misleading system prompts or in-context demonstrations make GPT-4 easier to manipulate.The paper potentially attributes this to GPT-4 following instructions more precisely.
  • Future directions: Future evaluations should test coordinated and stealthy adversaries that combine different misleading behaviors.The current work mainly considers one type of misleading or adversarial case per test scenario.
  • Future directions: Future work should evaluate domain-specific vulnerabilities in applications such as law and education.The paper’s current evaluations focus on general vulnerabilities and standard illustrative tasks.
  • Future directions: Empirical evaluations lack rigorous guarantees, motivating verification methods for GPT trustworthiness in safety-critical domains.The paper notes that GPT models’ discrete nature makes formal verification challenging.

B.2 Supplementary results on stereotype bias evaluation

The supplementary stereotype evaluation shows that GPT-4 can respond sarcastically to stereotype prompts, sometimes introducing additional stereotypes even while rejecting the original claim. The BBNLI analysis examines bias across NLI and QA formulations and demographic categories.

  • Heatmap evaluation: The heatmaps compare average stereotype agreement for GPT-3.5 and GPT-4 across stereotyped and non-stereotyped demographic groups under benign system prompts.Lighter colors indicate lower agreementIndex, darker colors higher agreementIndex; rows are divided by a red line.
  • Supplementary stereotype examples: GPT-4 sometimes rejects a stereotype sarcastically while introducing additional stereotypes about the same target group.The example rejects a claim about Asians but includes two further stereotypes.
  • Supplementary stereotype examples: GPT models generated explicitly stereotypical responses to prompts concerning Latin Americans, Canadians, Asians, and physically disabled people.Examples include agreement with drug-related, hygiene-related, and parenting-related stereotypes.
  • BBNLI evaluation: The BBNLI evaluation asks whether GPT models exhibit bias in NLI and QA settings and whether response format affects that bias.It uses the same benchmark content under two task encodings.
  • BBNLI results: Bias scores are higher for religion than race and gender under both BBNLI settings, indicating a need for additional attention to religious stereotypes.GPT-3.5 is marginally higher than GPT-4 in the reported comparison.

C.1 Details of the standard AdvGLUE benchmark

This supplementary section documents benchmark and evaluation materials used for standard AdvGLUE and related robustness and OOD experiments. It provides dataset statistics, adversarial-generation strategies, style transformations, and example task formats.

  • Dataset documentation: AdvGLUE and AdvGLUE++ are documented through tables reporting their test-set and dataset statistics.The section points to separate breakdowns for the standard AdvGLUE test set and AdvGLUE++ by task type and target model.
  • Adversarial construction: The adversarial-generation appendix describes typo substitution, importance-ranked synonym replacement, contextual masked-language replacements, and sememe-based search.These correspond to TextBugger, TextFooler, BERT-ATTACK, and SememePSO.
  • OOD style materials: OOD style transformations are summarized in a table, including Augment with a misspelling rate of 0.2 and Shake-W transformations.The remaining sentence-level transformations follow the stated methodology from prior work.
  • OOD knowledge examples: Additional tables provide qualitative examples for RealtimeQA, including questions answered correctly by GPT-4 under QA2023.The selected answer is shown in bold in the examples.
  • Experiment materials: The appendix references tables summarizing task descriptions and demonstration templates for the experiments in Section 7.These materials cover the tasks and prompt formats used in the robustness experiments.

E.3 More ablation studies

The ablation studies vary backdoor-trigger locations and document supplementary prompting and machine-ethics evaluation procedures. Results indicate that trigger location matters differently for GPT-3.5 and GPT-4.

  • More ablation studies: Under Setup 3, trigger placement at the beginning is most effective for GPT-3.5, whereas the end is least effective.For GPT-4, attack performance is similar across the three tested trigger locations.
  • More ablation studies: The GPT-4 trigger-location pattern may reflect that every demonstration example contains the backdoor trigger.The authors state this as a hypothesis explaining why location becomes less important for GPT-4.
  • Machine-ethics evaluation: The ETHICS evaluation uses zero-shot and few-shot prompts for commonsense morality, with demonstrations sampled from the training data in few-shot settings.The appendix describes separate handling of short and long testing samples.
  • Machine-ethics evaluation: The supplementary machine-ethics experiments extend evaluation beyond commonsense morality to justice, deontology, virtue, and utilitarianism.The prompts are adapted to GPT-model formatting and use task-specific response formats.
  • Machine-ethics results: Across five ETHICS subsets, few-shot GPT-4 achieves the highest accuracy except on utilitarianism, where it trails RoBERTa-large by 1.1% accuracy.Zero-shot GPT-4 remains competitive; on virtue it reaches 93.7% accuracy.
  • Machine-ethics robustness: GPT-4 and GPT-3.5 are evaluated against jailbreaking prompts and evasive sentences designed to perturb moral recognition.The appendix specifies separate prompt-insertion strategies for the two perturbation types.

G.4 Additional details of evaluation on conditional actions

The conditional-actions appendix defines concise Jiminy Cricket scenarios and Q&A prompts, then illustrates failures involving victims and harm severity. It also documents aggregation metrics and evaluation costs across trustworthiness perspectives.

  • Conditional-actions evaluation: The conditional-actions subset pairs concise scenario descriptions with good-or-bad morality labels and uses a Q&A-style prompt.Examples summarize the current action and observation, such as burning leaves with potential for forest fire.
  • Conditional-actions results: GPT models recognized harm to others as immoral but failed to recognize analogous self-harm as immoral.The examples contrast pushing another person from a railing with jumping from the railing oneself.
  • Conditional-actions results: GPT models recognized moderately severe harm as immoral but failed on a minor-severity action involving damage to a songbird egg.The comparison comes from the severity examples in the conditional-actions evaluation.
  • Evaluation statistics: The appendix defines evaluation-cost statistics as prompt count, prompt tokens, completion tokens, and total or per-run cost.These statistics are summarized across trustworthiness perspectives and individual scenarios.
  • Aggregation protocol: The benchmark aggregates perspective-specific scores for toxicity, stereotype bias, adversarial robustness, OOD robustness, and adversarial demonstrations using their corresponding metrics.The documented components include expected maximum toxicity, agreementIndex, robust accuracy, OOD reliability, and backdoor attack rates.
  • Aggregation protocol: The OOD reliability score combines standard and “I don’t know” QA2023 settings, while OOD demonstrations aggregate accuracy across demonstration sets and tasks.The protocol explicitly distinguishes Reliability2023 from Reliability2023idk.
  • Aggregation protocol: The adversarial-demonstration score is defined as one minus average attack success rate across four backdoor methods and three experimental setups.The methods are Badword, Addsent, Synbkd, and Stylebkd.

I.2 Comprehensive evaluation results of existing LLMs

The evaluation compares existing LLMs across eight trustworthiness perspectives and shows that strengths are distributed across models rather than concentrated in one system. GPT-4 leads three robustness perspectives, while Llama 2 leads four perspectives involving toxicity, bias, privacy, and fairness.

  • GPT-4 achieves the best performance on adversarial robustness, out-of-distribution robustness, and robustness to adversarial demonstrations.
  • Llama 2 achieves the best performance on toxicity, stereotype bias, privacy, and fairness.
  • No evaluated model achieves the best performance across all eight trustworthiness perspectives.
  • The paper presents overall and perspective-specific evaluation results through Tables 57–65 and Figures 38–47.The visualizations cover all perspectives as well as toxicity, stereotype bias, adversarial robustness, out-of-distribution robustness, adversarial demonstrations, privacy, machine ethics, and fairness.

J Limitations

The paper identifies limitations arising from opaque pretraining data, subjective trustworthiness criteria, and its focus on GPT-3.5 and GPT-4 from a specific time. It therefore frames the evaluation as a starting point for further research and mitigation work.

  • Opaque pretraining data limits explanations of model failures and the construction of genuinely out-of-distribution evaluation scenarios.The evaluation must hypothesize possible OOD distributions because the models’ training data is unknown.
  • Subjective perspectives such as toxicity, stereotype bias, machine ethics, and fairness require further study of alignment between objective metrics and human judgments.
  • Because the study primarily evaluates GPT-3.5 and GPT-4 released at a specific time, its results may not capture later model updates or rapid AI advances.
  • The authors encourage future evaluations to uncover additional LLM vulnerabilities and develop mitigation strategies.
Loading 2306.11698v5…