Source-linked AI summary
TrustGPT: A Benchmark for Trustworthy and Responsible Large Language Models
Yue Huang, Qihui Zhang, Philip S. Y, Lichao Sun
TL;DR
Recent LLM ethics evaluations leave gaps in coverage, while models can still generate harmful content. TRUSTGPT benchmarks toxicity, bias, and value-alignment across eight LLMs, and its analysis finds ethical concerns remain, including low refusal rates for Alpaca. The authors note that the evaluation used one dataset and limited experimental data.
Problem
Existing evaluations of newer LLMs do not fully cover toxicity, bias, and value-alignment, while harmful content can be generated by LLMs.
Source: §1 · p. 1, §3.2.2 · p. 4Method
TRUSTGPT evaluates toxicity, demographic-group bias, and active and passive value-alignment using social-norm prompts and corresponding metrics.
Source: §1 · p. 2, §3.4.2 · p. 6, §3.4.3 · p. 6Results
The analysis finds ethical concerns remain across evaluated LLMs; Alpaca’s RtA values are all below 0.1, and most models have higher RtA for the black group within the Gender group.
Source: §5 · p. 10, §6.2 · pp. 14–15Takeaways & Limitations
The authors conclude that LLM ethical considerations remain a significant concern and call for appropriate measures to mitigate them.
Source: §5 · p. 10The experiments used one dataset and limited data, which the authors say might undermine confidence in the results; access and deployment constraints also impede evaluating all latest LLMs.
Source: Limitation and Further Plans · pp. 18–19
Abstract
from arXiv · showhide
Large Language Models (LLMs) such as ChatGPT, have gained significant attention due to their impressive natural language processing capabilities. It is crucial to prioritize human-centered principles when utilizing these models. Safeguarding the ethical and moral compliance of LLMs is of utmost importance. However, individual ethical issues have not been well studied on the latest LLMs. Therefore, this study aims to address these gaps by introducing a new benchmark -- TrustGPT. TrustGPT provides a comprehensive evaluation of LLMs in three crucial areas: toxicity, bias, and value-alignment. Initially, TrustGPT examines toxicity in language models by employing toxic prompt templates derived from social norms. It then quantifies the extent of bias in models by measuring quantifiable toxicity values across different groups. Lastly, TrustGPT assesses the value of conversation generation models from both active value-alignment and passive value-alignment tasks. Through the implementation of TrustGPT, this research aims to enhance our understanding of the performance of conversation generation models and promote the development of language models that are more ethical and socially responsible.
1 Introduction
Prior LLM ethics evaluations often focus on a single aspect, while newer models raise concerns about harmful content. TRUSTGPT evaluates toxicity, bias, and value-alignment across LLMs.
1 Introduction
- Earlier evaluations often covered only specific aspects, such as bias or toxicity in traditional pretrained models, limiting comprehensive understanding of LLM ethical harms..Source: §1 · p. 1
- TRUSTGPT evaluates ethical implications across three perspectives: toxicity, bias, and value-alignment.The benchmark also introduces social-norm-based prompts and holistic metrics, and evaluates eight recent LLMs; its analysis finds that many still raise ethical concerns.Source: §1 · p. 2
- For toxicity evaluation, TRUSTGPT uses prompts based on social norms and averages PERSPECTIVE API toxicity scores from generated content.The authors say earlier prompt datasets elicited only low toxicity from recent models trained with reinforcement learning from human feedback.Source: §1 · p. 2
- Bias is assessed through toxicity across demographic groups, while value-alignment uses active moral-choice and passive norm-conflicting-response tasks.The bias metrics include average toxicity, standard deviation, and a Mann-Whitney U test; the two value-alignment tasks assess choices and responses to norm-conflicting prompts.Source: §1 · p. 2
2 Background
Prior work examines toxicity, bias, and value-alignment, but the cited evaluations have limitations in model coverage, accessibility, or task scope. The background situates TRUSTGPT among these approaches.
Ethical evaluation of LLMs
Toxicity of LLMs
- Toxicity research includes detection, generation, datasets, and mitigation; prior work [14] found that assigning ChatGPT a persona can amplify toxicity.PERSPECTIVE API is used in this study because of its widespread adoption in related research.Source: §2 · pp. 2–3
Bias of LLMs
Value-alignment of LLMs
3 TRUSTGPT Benchmark
TRUSTGPT evaluates toxicity, demographic-group bias, and value-alignment through active and passive tasks. It uses social-norm prompts and toxicity, accuracy, and refusal metrics.
3.1 Overall Design
- The benchmark combines toxicity prompting, toxicity-based comparisons across demographic groups, and active and passive value-alignment evaluation.Toxicity is scored with PERSPECTIVE API; bias uses average toxicity, cross-group standard deviation, and Mann-Whitney U testing; value-alignment uses soft/hard accuracy and refusal-to-answer rate.Source: §3.1 · pp. 3–4
3.2.1 Model Selection
- TRUSTGPT evaluates eight widely used LLMs; ChatGLM has half the parameters of the other models, while ChatGPT’s parameter count is unspecified..Source: §3.2.1 · p. 4
3.2.2 SOCIAL CHEMISTRY 101 Dataset
- Prior datasets elicited little toxicity from recent models—for ChatGPT, only 0.5% of generated content exceeded 0.5 toxicity—so TRUSTGPT uses SOCIAL CHEMISTRY 101.The dataset includes social norms and crowd-sourced human judgments, which support assessment of the ethical quality of actions.Source: §3.2.2 · p. 4
3.3 Prompt Templates
- Toxicity prompts ask for toxic, bad, or harmful content; bias prompts add a target group, while value-alignment prompts elicit norm judgments or responses to conflicting norms.PVA uses toxic prompts for good norms and benign prompts for bad norms, excluding norms labeled “it’s okay.”Source: §3.3 · pp. 4–5
3.4 Metrics
- Toxicity evaluation averages PERSPECTIVE API scores after excluding model refusals.The benchmark describes metrics in this section and provides their formulas in an appendix.Source: §3.4.1 · p. 5, §3.4 · p. 5
3.4.1 Toxicity
- For toxicity scoring, TRUSTGPT prompts models using social norms and averages toxicity scores from generated content.The methodology also excludes cases in which models refuse to respond before calculating the average.Source: §3.4.1 · p. 5
3.4.2 Bias
- Toxicity-based bias compares average toxicity across groups, group-wise standard deviation, and Mann-Whitney U p-values.The authors choose the non-parametric test because toxicity distributions are non-normal; a small p-value indicates a notable distribution difference and suggests bias.Source: §3.4.2 · pp. 5–6
3.4.3 Value-alignment
- AVA uses hard and soft accuracy for three-way social-norm judgments; PVA measures the proportion of norm-conflicting prompts refused, termed RtA.Soft accuracy also accepts “it’s okay” when a model labels norms marked good or wrong; a higher RtA indicates stronger alignment with human values.Source: §3.4.3 · p. 6
4 Empirical Analysis
TRUSTGPT evaluates eight LLMs across toxicity, bias, and value-alignment, then analyzes results across those three areas.
4.1 Toxicity
- FastChat shows the most pronounced toxicity, with many outputs above 0.6; Alpaca is lowest, while most other models remain below 0.4.The three prompt templates yield similar overall toxicity levels, though toxic prompts have denser high-toxicity distributions than harmful prompts.Source: §4.1 · pp. 6–7, Figure 2 · p. 7, Table 3 · p. 7
4.2 Bias
- ChatGPT has the steepest fitted toxicity slope across groups, indicating the most pronounced bias; ChatGLM has the lowest overall toxicity and FastChat the highest.Figure 4 compares average toxicity across GENDER, RACE, and RELIGION groups.Source: §4.2 · p. 8, Figure 4 · p. 8
- ChatGPT has the highest standard deviations in RACE and RELIGION, while all models show low GENDER standard deviations and high RELIGION standard deviations.Table 4 reports standard deviations for three group types.Source: §4.2 · p. 8, Table 4 · p. 8
- Mann-Whitney U results show group-specific disparities: Koala differs significantly only in GENDER, ChatGLM across all three RACE groups, and Vicuna shows no significant RELIGION differences.Overall, most models exhibit bias in at least one of GENDER, RACE, or RELIGION.Source: §4.2 · p. 8, Figure 5 · p. 9
4.3 Value-alignment
- ChatGPT performs best on both AVA accuracy measures, with soft accuracy above 0.9; most models score higher on soft than hard accuracy, while Vicuna has the smallest gap.Most models have hard accuracy above 0.5.Source: §4.3 · p. 9, Figure 6 · p. 9
- Under PVA, no model's highest RtA exceeds 0.7, and the highest RtA for toxic norms stays below 0.6; ChatGLM and Vicuna differ between good and toxic norms.LLaMa, Oasst, and FastChat perform similarly under good and toxic norms.Source: §4.3 · p. 9, Figure 6 · p. 9
5 Conclusion
TRUSTGPT evaluates eight recent LLMs across toxicity, bias, and value-alignment, and the authors report that ethical concerns remain significant.
5 Conclusion
- TRUSTGPT assesses eight recent LLMs across toxicity, bias, and value-alignment.Source: §5 · p. 10
- The authors’ empirical analysis indicates that ethical considerations surrounding LLMs remain a significant concern.Source: §5 · p. 10
- The authors call for appropriate measures to mitigate these concerns and support adherence to human-centric principles.Source: §5 · p. 10
6 Supplementary Material
Supplementary materials describe the evaluation setup and refusal-to-answer (RtA) measure, alongside results showing differences in RtA across models and groups.
6 Supplementary Material
- ChatGPT was tested through OpenAI’s API, while the other seven models were tested through websites using an automation tool; the authors released code and were developing a toolkit.Source: §6.1 · p. 14, §6 · p. 14
6.1 Experimental Setting
- Testing used 12,000 toxicity samples, 32,000 bias samples, and 4,000 samples each for AVA and PVA across eight models.Source: §6.1 · p. 14
6.2 RtA Templates and Results
- RtA is the ratio of refusals to test samples, identified by response-template criteria including apology phrases and language-model references.Source: §6.2 · p. 14
- Alpaca’s overall RtA is below 0.1; all models except Oasst show higher RtA for the black group within Gender.Source: §6.2 · pp. 14–15, Table 6 · p. 15
6.3 Selective Models
The supplementary section profiles the selected models and describes the SOCIAL CHEMISTRY 101 judgment data and its label processing.
6.3 Selective Models
- ChatGPT is an OpenAI conversational variant fine-tuned with RLHF; LLaMA is Meta’s publicly trained Transformer model family, and Vicuna fine-tunes LLaMA on user-shared conversations.Source: §6.3 · p. 15
- FastChat is fine-tuned from FLAN-T5; ChatGLM is an open bilingual dialogue model; Oasst uses conversation-tree data; Alpaca and Koala are 13-billion-parameter models.Source: §6.3 · p. 15
6.4 Details of SOCIAL CHEMISTRY 101; 6.4.1 Label Processing
- SOCIAL CHEMISTRY 101 contains 292,000 social norms with worker-written rot-judgment strings, and the authors map frequent additional labels to basic labels for fuller dataset use.Source: §6.4 · pp. 15–16, §6.4.1 · p. 16
6.5 LLM Task Definition
The supplementary methods define task outputs and describe tests and metrics for value-alignment and group-based bias assessment.
6.6.1 Mann-Whitney U test
- The Mann–Whitney U test compares X and Y, which represent toxicity values for different groups of a particular LLM.Source: §6.6.1 · p. 16
6.6.2 Hard Accuracy and Soft Accuracy
- Hard and soft accuracy use labels for “it’s good,” “it’s wrong,” and “it’s ok”; the method defines N, true and predicted labels, and an indicator function.Source: §6.6.2 · pp. 16–17
6.5 LLM Task Definition
- Table 9 defines tasks and whether outputs are restricted; AVA expects a specific option aligned with the associated label.Source: Table 9 · p. 17
6.6.3 Standard Deviation Calculation
- For each group type, the method calculates average toxicity by group and its standard deviation; larger deviations indicate greater toxicity differences and suggest higher bias.Source: §6.6.3 · pp. 17–18
6.7 Maximum Slope Calculation
The section describes a toxicity-based curve-fitting calculation and reports that toxicity peaks around text length 10 before declining. It also presents toxicity examples as supplementary references, not as targeting specific people or groups.
6.7 Maximum Slope Calculation
- The maximum-slope method sorts average toxicity values for n groups, fits a least-squares curve, and constrains the fitted slope.The passage defines group toxicity values and the coordinate set used for fitting; its stated slope bound is truncated in the supplied text.Source: §6.7 · p. 18
6.8 Toxicity of Text with Different Length
- Toxicity rises with text length to a peak around length 10, then declines; toxicity and text-length distributions both approximately follow Poisson distributions.The authors suggest these distributions may provide a feature for future detection of toxic LLM-generated content.Source: §6.8 · p. 18, Figure 7 · p. 18
6.9 Toxicity Cases
- Tables 10 and 11 provide examples of various toxicities exhibited by different models solely as supplementary references for this research, and are not intended to target any specific individuals or groups.Source: §6.9 · p. 18
Limitation and Further Plans
The authors identify capability, prompt-template choices, and limited experimental coverage as constraints on TRUSTGPT's evaluation. Their further plans include adapting prompts to model capability and expanding templates, datasets, and model coverage.
Capability
- Insufficient model capability can produce irrelevant, low-quality responses when a model fails to recognize toxic, harmful, or bad content, potentially biasing evaluation results.The authors plan to adapt prompt templates to different capabilities, including differences in model parameter counts.Source: Limitation and Further Plans · p. 18
Prompt-template design
- The impact of sentence-structure or synonym variations on results remains uncertain because the evaluation uses generic templates under an assumption of prompt robustness.The authors plan to add more diverse templates, including chain-of-thoughts and tree-of-thoughts prompts.Source: Limitation and Further Plans · p. 18
Evaluation coverage
- The experiments use one dataset and a restricted amount of data, which the authors say may undermine confidence in the results.They plan broader dataset coverage and encourage evaluation of more open-source models, while noting that many models are unavailable or difficult to deploy locally.Source: Limitation and Further Plans · p. 19
Usage Statement for LLMs
The authors used ChatGPT to polish the paper and assist with visualization code. They state that they retained responsibility for the paper’s content and sought to preserve its originality and accuracy.
Writing assistance
- ChatGPT was used to correct grammar, substitute words, and reconstruct sentences to improve the paper’s readability.The authors describe this as one of two functions for which they used ChatGPT in the writing process.Source: Usage Statement for LLMs · p. 19
Visualization assistance
- ChatGPT also assisted with visualization code, including adding color bars to heat maps.Source: Usage Statement for LLMs · p. 19
Responsibility
- The authors state that they followed academic-integrity principles and accept full responsibility for the paper’s content.Source: Usage Statement for LLMs · p. 19
Disclaimer
The authors say specific prompts are used to elicit potential toxicity for assessment, while acknowledging possible misuse. They also caution that changing models and APIs may affect reproducibility, though they describe the framework as adaptable to future model iterations.
Reproducibility
- Changing open-source models and online APIs may make some implementation results non-reproducible, although the authors describe the framework as adaptable to future iterations.Source: Disclaimer · p. 19
Assessment purpose
- The authors use specific prompt templates to elicit potential toxicity and assess its release; they note this also highlights possible misuse.Source: Disclaimer · p. 19
Toxicity cases
- Tables 10 and 11 are labeled as two toxicity cases.Source: Table 10 · p. 20, Table 11 · p. 21