PaperlayerView exact revision on arXiv

Source-linked AI summary

TrustLLM: Trustworthiness in Large Language Models

Yue Huang, Lichao Sun, Haoran Wang, Siyuan Wu, Qihui Zhang, Yuan Li, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, Xiner Li, Zhengliang Liu, Yixin Liu, Yijue Wang, Zhikun Zhang, Bertie Vidgen, Bhavya Kailkhura, Caiming Xiong, Chaowei Xiao, Chunyuan Li, Eric Xing, Furong Huang, Hao Liu, Heng Ji, Hongyi Wang, Huan Zhang, Huaxiu Yao, Manolis Kellis, Marinka Zitnik, Meng Jiang, Mohit Bansal, James Zou, Jian Pei, Jian Liu, Jianfeng Gao, Jiawei Han, Jieyu Zhao, Jiliang Tang, Jindong Wang, Joaquin Vanschoren, John Mitchell, Kai Shu, Kaidi Xu, Kai-Wei Chang, Lifang He, Lifu Huang, Michael Backes, Neil Zhenqiang Gong, Philip S. Yu, Pin-Yu Chen, Quanquan Gu, Ran Xu, Rex Ying, Shuiwang Ji, Suman Jana, Tianlong Chen, Tianming Liu, Tianyi Zhou, William Wang, Xiang Li, Xiangliang Zhang, Xiao Wang, Xing Xie, Xun Chen, Xuyu Wang, Yan Liu, Yanfang Ye, Yinzhi Cao, Yong Chen, Yue Zhao

arXiv:2401.05561v6cs.CL

TL;DR

LLMs’ complex, generative outputs create trustworthiness concerns, motivating systematic evaluation. TRUSTLLM organizes eight dimensions, benchmarks six across over 30 datasets, and finds trustworthiness is connected to utility while performance varies between open-weight and proprietary models.

  • Problem

    LLMs’ complex and generative outputs can produce inaccurate, misleading, unsafe, or otherwise untrustworthy responses, making trustworthiness a central research concern.

    Source: §1 · p. 5, §4.4 · p. 18, §6 · p. 26

  • Method

    TRUSTLLM reviews trustworthiness research, defines eight dimensions, establishes a six-dimension benchmark, and evaluates 16 mainstream LLMs across more than 30 datasets.

    Source: §1 · pp. 6–7

  • Results

    Trustworthiness and utility are positively related overall, while many LLMs exhibit excessive trustworthiness and open-weight models vary notably from proprietary counterparts.

    Source: §1 · p. 7, §16 · p. 79

  • Takeaways & Limitations

    Trustworthy LLM analysis should consider utility alongside multiple trustworthiness dimensions and examine transparency in the technologies supporting trustworthiness.

    Source: §1 · p. 7, §16 · p. 79

    TRUSTLLM does not rigorously certify LLM trustworthiness and therefore cannot guarantee reflection of worst-case behavior.

    Source: §14 · p. 74

Abstract

from arXiv · show

Large language models (LLMs), exemplified by ChatGPT, have gained considerable attention for their excellent natural language processing capabilities. Nonetheless, these LLMs present many challenges, particularly in the realm of trustworthiness. Therefore, ensuring the trustworthiness of LLMs emerges as an important topic. This paper introduces TrustLLM, a comprehensive study of trustworthiness in LLMs, including principles for different dimensions of trustworthiness, established benchmark, evaluation, and analysis of trustworthiness for mainstream LLMs, and discussion of open challenges and future directions. Specifically, we first propose a set of principles for trustworthy LLMs that span eight different dimensions. Based on these principles, we further establish a benchmark across six dimensions including truthfulness, safety, fairness, robustness, privacy, and machine ethics. We then present a study evaluating 16 mainstream LLMs in TrustLLM, consisting of over 30 datasets. Our findings firstly show that in general trustworthiness and utility (i.e., functional effectiveness) are positively related. Secondly, our observations reveal that proprietary LLMs generally outperform most open-source counterparts in terms of trustworthiness, raising concerns about the potential risks of widely accessible open-source LLMs. However, a few open-source LLMs come very close to proprietary ones. Thirdly, it is important to note that some LLMs may be overly calibrated towards exhibiting trustworthiness, to the extent that they compromise their utility by mistakenly treating benign prompts as harmful and consequently not responding. Finally, we emphasize the importance of ensuring transparency not only in the models themselves but also in the technologies that underpin trustworthiness. Knowing the specific trustworthy technologies that have been employed is crucial for analyzing their effectiveness.

– A PRINCIPLE AND BENCHMARK

TrustLLM is an established benchmark for evaluating trustworthiness in mainstream large language models, grounded in principles spanning different trustworthiness dimensions.

Benchmark

  • TrustLLM establishes a benchmark for assessing trustworthiness in mainstream LLMs.The benchmark is presented as an established resource for trustworthiness evaluation.Source: – A PRINCIPLE AND BENCHMARK · p. 1

Principles

  • The benchmark is based on principles covering different dimensions of trustworthiness.Its evaluation framework derives from dimension-specific trustworthiness principles.Source: – A PRINCIPLE AND BENCHMARK · p. 1

Scope

  • The benchmark targets mainstream LLMs as its evaluation scope.The stated scope is trustworthiness assessment for mainstream large language models.Source: – A PRINCIPLE AND BENCHMARK · p. 1

1 Introduction

TRUSTLLM presents a unified framework for defining, benchmarking, and evaluating LLM trustworthiness, motivated by risks including unreliable outputs, misuse, bias, privacy breaches, and value misalignment. It identifies eight trustworthiness dimensions and benchmarks six of them across 16 LLMs, more than 30 datasets, and over 18 subcategories.

Motivation

  • LLM trustworthiness is threatened by unreliable or misleading outputs, malicious misuse, bias, privacy risks, and possible misalignment with human values.These risks arise from LLMs’ complex generative capabilities, training-data problems, and high user expectations.Source: §1 · p. 5
  • Trustworthiness benchmarking remains difficult because comprehensive criteria, scalable and generalizable benchmarks, and practical evaluation methods are not yet standardized.Existing taxonomies either omit aspects of trustworthiness or use overlapping fine-grained categories that complicate clear benchmarks.Source: §1 · p. 6

Framework

  • TRUSTLLM identifies eight trustworthiness dimensions: truthfulness, safety, fairness, robustness, privacy, machine ethics, transparency, and accountability.The taxonomy is based on a review of 600 papers spanning AI, machine learning, data mining, HCI, and cybersecurity, while utility is treated separately.Source: §1 · pp. 6–7
  • TRUSTLLM provides a unified framework combining a literature survey, organizing principles, a novel benchmark, and comprehensive evaluation of mainstream LLMs.The selected models span different sizes, training strategies, and functional capabilities, while the datasets range from classification to complex generation tasks.Source: §1 · p. 6

Benchmark

  • The benchmark covers six dimensions across over 18 subcategories, more than 30 datasets, and 16 proprietary and open-weight LLMs.Transparency and accountability are excluded from benchmarking because they are difficult to evaluate directly.Source: §1 · pp. 6–7

Contributions

  • The study reports findings on effectiveness–trustworthiness relationships, alignment deficiencies, proprietary versus open-weight disparities, and opacity in current trustworthiness technologies.The paper presents the overall ranking in Figure 1 and directs readers to later sections for dimension-specific evaluations.Source: §1 · pp. 7–8, Figure 1 · p. 7

2 Observations and Insights

Empirical results show that trustworthiness is generally aligned with utility, while proprietary models usually outperform open-weight models but some open-weight systems remain competitive. Across individual dimensions, the findings expose persistent weaknesses in truthfulness, safety, fairness, robustness, privacy, and machine ethics, alongside the need for better alignment and transparency.

2.1 Overall Observations

  • Trustworthiness and utility are positively correlated, with stronger language understanding improving moral classification, stereotype recognition, and resilience to adversarial attacks.Trustworthiness rankings often mirror utility-focused leaderboards such as MT-Bench and the OpenLLM Leaderboard.Source: §2.1 · p. 9
  • Many LLMs are overly aligned: Llama2-7b refused 57% of prompts that were actually harmless, showing that intent-sensitive alignment is needed to reduce false-positive harm detection.Exaggerated safety can compromise utility by treating innocuous prompts as harmful rather than understanding their intent.Source: §2.1 · p. 9
  • Proprietary LLMs generally outperform most open-weight models in trustworthiness, although Llama2 [69] surpasses proprietary models on many tasks without external auxiliary modules.The gap raises concerns about risks from widely downloadable open-weight models, while Llama2 demonstrates that strong trustworthiness is achievable without added moderators.Source: §2.1 · p. 9
  • Transparency is needed for both model designs and trustworthiness technologies because disclosed training mechanisms and techniques would support analysis, adoption, and improvement.The paper highlights proprietary systems such as ERNIE whose underlying trustworthy technologies remain undisclosed, while model research also depends on understanding parameters and architecture.Source: §2.1 · p. 9

2.2 Novel Insights into Individual Dimensions of Trustworthiness

  • Truthfulness remains difficult from internal knowledge because of noisy or outdated training data and limited Transformer generalization, whereas external knowledge substantially improves results beyond original-dataset state of the art.Models hallucinate less in multiple-choice questions than in open-ended knowledge-grounded dialogue, and lower sycophancy is associated with better detection of factual errors.Source: §2.2 · pp. 9–10
  • Safety varies across jailbreak types and remains substantially weaker for most open-source LLMs, while stringent safeguards can produce exaggerated caution and superficial alignment.Leetspeak attacks [61] illustrate the need for comprehensive defenses against diverse attack methods; Llama2 and ERNIE exemplify over-cautious behavior [72].Source: §2.2 · p. 10
  • Fairness performance is generally weak: GPT-4 reaches only 65% stereotype-identification accuracy, few models handle disparagement fairly, and forced-choice preferences sharply reduce neutrality.Oasst-12b and Vicuna-7b are among the few models showing fairness in disparagement tasks.Source: §2.2 · p. 10
  • Robustness, privacy, and machine ethics vary widely: perturbation similarity ranges from 88% to 97.64%, ChatGPT’s privacy correlation is 0.665, and GPT-4 reaches 94% awareness accuracy.OOD performance ranges from GPT-4’s over-80% RtA and over-92% F1 to roughly 0.4% RtA and 30% F1; nearly all models leak information on Enron Email, while most implicit moral-task accuracy remains below 70%.Source: §2.2 · pp. 10–11

3 Background

This background reviews LLM architectures, training and alignment, the expanding multidimensional evaluation landscape, industry trustworthiness practices, and existing benchmarks’ coverage limitations.

3.1 Large Language Models (LLMs)

  • LLMs use Transformer-based architectures and are typically trained through pre-training, instruction tuning, and alignment to develop general-purpose capabilities and human-aligned behavior.Common alignment methods include supervised fine-tuning and RLHF, while newer offline preference methods seek greater training stability.Source: §3.1 · p. 12

3.2 Evaluation on LLMs

  • LLM evaluation spans language understanding, reasoning, generation, multilinguality, applications, code synthesis, robustness, and trustworthiness, using benchmarks alongside automatic and human assessment.Traditional metrics such as BLEU and ROUGE can undervalue valid outputs when multiple correct generations exist.Source: §3.2 · pp. 12–13

3.3 Developers and Their Approaches to Enhancing Trustworthiness in LLMs

  • Major developers pursue trustworthiness through data filtering, alignment, red teaming, safety filters, responsible-AI standards, constitutional principles, and lifecycle governance.Their approaches cover dimensions including safety, fairness, privacy, robustness, transparency, accountability, and ethical response tuning.Source: §3.3 · pp. 13–14

3.4 Trustworthiness-related Benchmarks

  • Existing trustworthiness benchmarks address varied concerns, but most cover only specific aspects and are not comprehensive enough.Examples include safety, robustness, misinformation, privacy, fairness, and transparency benchmarks, motivating comparison with TRUSTLLM in Table 1.Source: §3.4 · pp. 14–15, Table 1 · p. 15

4 Guidelines and Principles for Trustworthiness Assessment of LLMs

The paper derives trustworthiness guidelines from a systematic, cross-validated literature review and organizes them into eight dimensions, each with defined assessment principles. It also frames regulation and benchmarking as necessary complements for evaluating LLM trustworthiness across diverse risks and usage conditions.

4 Guidelines and Principles for Trustworthiness Assessment of LLMs

  • A systematic literature review with independent screening, thematic coding, and cross-verification distilled guidelines for evaluating LLM trustworthiness across eight dimensions.The review searched ACM, IEEE Xplore, and arXiv for recent work and coded themes including safety, ethics, and fairness.Source: §4 · p. 16, Table 2 · p. 16

4.1 Truthfulness

  • Truthfulness assessment covers misinformation, hallucination, sycophancy, and correction of adversarial facts across internal-knowledge, retrieval, and multiple task settings.The paper defines truthfulness as accurate representation of information, facts, and results, while noting that imperfect training data propagates misinformation and outdated content.Source: §4.1 · p. 16

4.2 Safety

  • Safety is defined as avoiding unsafe or illegal outputs and maintaining healthy conversation, assessed through jailbreak taxonomy, over-alignment, toxicity, and misuse resilience.Jailbreaking can produce toxic content and enable misuse, while role assignment and parameter tuning influence harmful outputs.Source: §4.2 · p. 17

4.3 Fairness

  • Fairness concerns equitable design, training, and deployment, with assessment focused on stereotypes, disparagement, and preference biases affecting users and demographic groups.The framework examines stereotype agreement, recognition and queries, salary attribution, and decisions involving contrasting opinions.Source: §4.3 · p. 17

4.4 Robustnesss

  • Robustness measures stability under ordinary input variation, including natural noise and out-of-distribution challenges, distinct from resilience against malicious attacks.Prior work indicates robustness is not inherent in current LLMs; GPT-3.5, for example, can fail on seemingly simple emoji inputs.Source: §4.4 · p. 18

4.5 Privacy

  • Privacy assessment examines awareness of privacy-sensitive situations and inadvertent leakage of sensitive information learned from training data when prompted.Internet-sourced training data can embed personal information that malicious prompts may extract.Source: §4.5 · p. 18

4.6 Machine Ethics

  • Machine ethics is assessed through implicit ethics, explicit ethics, and emotional awareness, covering moral judgments, behavior in ethical contexts, and recognition of abilities, emotions, and perspectives.These segments address LLM alignment with human ethical standards and their growing role in action planning and decision-making.Source: §4.6 · pp. 18–19

4.7 Transparency; 4.8 Accountability

  • Transparency concerns the information available about LLMs and their outputs, while accountability requires explanations and justifications despite responsibility, ownership, data, and copyright barriers.Complexity makes transparency difficult to evaluate, and insufficient transparency can produce weak accountability in LLM-powered systems.Source: §4.8 · pp. 19–20, §4.7 · p. 19

5 Preliminaries of TRUSTLLM

TRUSTLLM benchmarks trustworthiness across a curated set of 16 proprietary and open-weight LLMs using 30 datasets and diverse tasks. Its experimental design standardizes task-specific prompting, decoding, and automated or model-based evaluation.

5.1 Curated List of LLMs

  • TRUSTLLM evaluates 16 proprietary and open-weight LLMs spanning diverse sizes, training data, methodologies, and functional capabilities.The models are summarized in Table 3.Source: §5.1 · p. 21

5.2 Experimental Settings

  • The benchmark contains 30 datasets covering diverse trustworthy-LLM capabilities, organized into Generation and Classification tasks.Classification uses temperature 0 for precise outputs, while generation uses temperature 1 to encourage diversity and explore potential worst-case scenarios; other decoding settings remain model defaults.Source: §5.2 · p. 23
  • Prompts generally require explicit keywords for automated assessment, omit few-shot and chain-of-thought methods, and exclude these methods to measure plain LLM results.Few-shot examples are retained only for privacy leakage evaluation, where prompts aim to increase privacy-information leakage.Source: §5.2 · p. 23
  • Evaluation combines keyword or regex matching, a trained Longformer refusal classifier, ChatGPT, and GPT-4 to assess labeled and generative responses cost-effectively.GPT-4 evaluates complex generative tasks such as Adversarial Factuality, while ChatGPT handles simpler generative tasks; ChatGPT/GPT-4 also extract keywords when automated matching fails.Source: §5.2 · pp. 23–25

6 Assessment of Truthfulness · 6.1 Misinformation Generation · 6.2 Hallucination

TrustLLM assesses truthfulness through misinformation generation and hallucination, testing internal versus externally retrieved knowledge and four response-generation tasks. Results show that retrieval improves fact-checking, while most models still hallucinate, with GPT-4 often strongest but task-specific leaders emerging.

6 Assessment of Truthfulness

  • Truthfulness is challenged by inaccurate or imprecise generations, partly because internet-scale training data can contain erroneous, obsolete, or deliberately misleading information.The assessment covers misinformation, hallucination, sycophancy, and adversarial factuality.Source: §6 · p. 26

6.1 Misinformation Generation

  • Misinformation is defined as inadvertent factual inaccuracy, and retrieval-augmented systems use credible external sources to improve knowledge-intensive tasks.Such systems can outperform non-retrieval models by a large margin with fewer parameters.Source: §6.1 · p. 26

6.1.1 Using Merely Internal Knowledge

  • Internal-knowledge misinformation is evaluated with zero-shot direct-answering across SQuAD2.0, CODAH, HotpotQA, and AdversarialQA.CODAH uses exact match, while the other datasets are assessed with ChatGPT as a natural-language-inference evaluator.Source: §6.1.1 · pp. 26–27
  • Using only internal knowledge, all LLMs perform poorly; GPT-4 has the best average, while Mistral-7b and Llama2-70b lead AdversarialQA and CODAH respectively.No model consistently leads all four datasets, and CODAH is difficult for every model because of commonsense reasoning demands.Source: §6.1.1 · p. 27, Table 7 · p. 28

6.1.2 Integrating External Knowledge

  • External-knowledge misinformation is tested through zero-shot fact-checking, presenting claims with evidence from Climate-FEVER, SciFact, COVID-Fact, and HealthVER.Models predict claim veracity and are evaluated using macro F-1.Source: §6.1.2 · p. 27, Table 8 · p. 28
  • External knowledge improves all models over internal-only knowledge, with GPT-4 achieving the highest average fact-checking performance, followed by Vicuna-33b and ChatGPT.The evaluation presents claims with ground-truth evidence and uses macro F-1 across four fact-checking datasets.Source: §6.1.2 · pp. 27–28, Table 7 · p. 28

6.2 Hallucination

  • Hallucination benchmarking covers multiple-choice QA, open-ended QA, knowledge-grounded dialogue, and summarization, with accuracy measuring factual selection or hallucination discrimination.Most models struggle despite producing coherent, natural language, reflecting the risk of non-factual generation.Source: §6.2 · pp. 28–30
  • Most LLMs struggle across four hallucination tasks, with GPT-4 best on MC and SUM while ChatGLM2 leads QA and KGD.Higher accuracy indicates more reliable factual selection or discrimination between hallucinated and non-hallucinated responses.Source: §6.2 · p. 30, Table 9 · p. 30

6.3 Sycophancy in Responses · 6.4 Adversarial Factuality

The study evaluates sycophancy through persona-based and preference-driven tests, then examines whether LLMs correct misinformation embedded in user prompts. Results identify differences across models in sycophancy performance and show strong factual-error correction by GPT-4 and Llama2 models.

6.3 Sycophancy in Responses

  • Persona sycophancy describes responses aligning with inferred user beliefs, while preference sycophancy measures whether responses change when user preferences reverse.The study attributes sycophancy to majority, recency, and common-token biases and evaluates both forms in TrustLLM.Source: §6.3.1 · p. 30, §6.3 · p. 30, §6.3.2 · p. 31

6.3.1 Persona-based Sycophancy

  • Persona sycophancy is measured by the average cosine-similarity difference between model responses and non-sycophantic versus sycophantic answers.The metric is sim(r) = cos(r, ns) − cos(r, s).Source: §6.3.1 · p. 31

6.3.2 Preference-driven Sycophancy

  • Preference sycophancy is evaluated using 120 pairs of opposing opinions, with lower response-change percentages indicating more consistent behavior.ChatGPT judges whether responses to reversed-preference prompts convey the same meaning.Source: §6.3.2 · p. 31
  • Llama2-70b achieved the highest persona-sycophancy similarity score, whereas ChatGPT had the lowest preference-driven opinion-change percentage.The results use similarity for persona sycophancy and percentage change for preference sycophancy.Source: §6.3.2 · p. 31, Table 12 · p. 32

6.4 Adversarial Factuality

  • Adversarial factuality tests whether models recognize and correct incorrect information inserted into otherwise related user questions.The study notes that overlooking such errors can reduce helpfulness and encourage hallucinated content.Source: §6.4 · p. 32
  • The adversarial-factuality dataset contains 209 entries across computer science, biology, chemistry, physics, finance, arts, and history, generated from manually constructed seeds and GPT-4 expansions.Each seed includes original knowledge, modified incorrect knowledge, a related user question, and a prompt.Source: §6.4 · pp. 32–33
  • GPT-4 corrected factual errors in more than 80% of test data, while Llama2-70b achieved a 79.4% correction rate.Llama2-7b and Llama2-13b achieved correction rates of 71.8% and 70.8%, respectively.Source: §6.4 · p. 33, Table 14 · p. 33
  • Models that performed well on sycophancy tests also performed strongly on adversarial factuality, possibly because instruction tuning reduced their inclination toward sycophancy.The paper specifically identifies Llama2-70b and ChatGPT as examples and links this adjustment to confidently identifying prompt errors.Source: §6.4 · p. 33

7 Assessment of Safety

This section assesses LLM safety across jailbreak resistance, exaggerated safety, toxicity, and deliberate misuse using dedicated datasets and automated evaluation. It examines substantial variation among models, including differences in jailbreak defenses, refusal behavior, and toxicity levels.

7 Assessment of Safety

  • The benchmark evaluates safety using four aspects: jailbreak, exaggerated safety, toxicity, and misuse, with JAILBREAKTRIGGER covering 13 attack methods.The evaluation uses XSTEST for exaggerated safety, jailbroken ITC outputs for toxicity, and Do-Not-Answer, Do-Anything-Now, plus a constructed dataset for misuse.Source: §7 · p. 34, §7.1 · pp. 34, 38

7.1 Jailbreak

  • Baichuan-13b failed to reject harmful instructions in 70% of both QBB and ITC cases, while ERNIE and Llama2 models showed the highest jailbreak safety.PaLM 2 performed worse than most open-weight LLMs, whereas Vicuna-7b, Vicuna-33b, Oasst-12b, and Koala-13b also showed poor defenses.Source: §7.1 · p. 38, Table 17 · p. 39

7.2 Exaggerated Safety

  • Llama2 models exhibited the most severe exaggerated safety, while stronger over-refusal was associated with better resistance to misuse and jailbreaks, revealing a difficult safety-utility trade-off.XSTEST contains 200 calibrated safety prompts that models should answer, and results showed no clear relationship between model size and safety.Source: §7.2 · pp. 38–39

7.3 Toxicity

  • GPT-4, Vicuna-13b, and ChatGPT had the highest average toxicity, whereas ERNIE averaged below 0.1 and never exceeded 0.3.Toxicity is measured with Perspective API on outputs elicited through successful ITC jailbreaks, because aligned models often refuse direct toxic-content prompts.Source: §7.3 · p. 40

7.4 Misuse

  • Misuse evaluation tests whether models refuse direct prompts involving harmful activities such as bomb-making, botnet attacks, violence, false information, and illegal information.Unlike jailbreak evaluation, misuse uses the original prompts directly rather than modified prompts designed to circumvent alignment.Source: §7 · p. 34, §7.4 · p. 40
  • The study focuses on deliberate misuse, including misinformation, conspiracy theories, cyberattacks, spear phishing, and hate-driven campaigns enabled by LLMs.Misuse refers primarily to dangerous or inappropriate uses, rather than every malicious or inappropriate prompt.Source: §7.4 · pp. 40–41
  • The evaluation combined the Do-Not-Answer dataset, GPT-4-generated expanded misuse prompts, and the Do Anything Now dataset to cover dangerous or inappropriate uses.Keyword matching and manual selection identified relevant examples from Do-Not-Answer data.Source: §7.4 · p. 41
  • A fine-tuned 600M-parameter Longformer classifier automatically detected refusals, using Refuse to Answer (RtA) as the evaluation metric.The automated approach was adopted because human evaluation was costly.Source: §7.4 · p. 41
  • RtA values exceeded 0.7 for most LLMs, while Baichuan-13b, Vicuna-7b, and Oasst-12b were exceptions; Llama2 performed best, followed by ChatGPT and GPT-4.Baichuan-13b ranked last across all three datasets, with refusal probabilities below 0.2.Source: §7.4 · p. 41
  • Models performed better on the “do not answer” dataset than on the additional and “do anything” datasets, suggesting stronger recognition of shorter prompts.The additional dataset used GPT-4-generated elaborations of brief Do-Not-Answer prompts.Source: §7.4 · p. 41

8 Assessment of Fairness

TrustLLM assesses fairness through stereotypes, disparagement, and preference bias, finding substantial variation in model behavior and fairness-related performance across these dimensions. Results show strong stereotype rejection by some models, attribute-associated salary predictions in others, and preference neutrality that declines under forced choice.

8 Assessment of Fairness

  • The fairness assessment covers stereotypes, disparagement, and preference bias using stereotype tasks, attribute-based salary prediction, and opposing-choice questions.Stereotype evaluation uses agreement, recognition, and query tests; disparagement varies gender and race attributes while controlling other salary-related features; preference evaluation uses 120 subjective questions.Source: §8 · p. 42, §8.1 · p. 42, §8.2 · p. 45

8.1 Stereotypes

  • GPT-4, Ernie, and Llama2-13b show low stereotype agreement, while Oasst-12b, Chatglm2, and Koala-13b frequently agree with stereotypes.GPT-4, Llama2-70b, and ChatGPT are the only models exceeding 0.5 classification accuracy broadly, while half the models refuse all stereotype queries and most exceed 90% RtA.Source: §8.1 · pp. 43–44, Table 20 · p. 44, Table 21 · p. 44

8.2 Disparagement

  • Oasst-12b and Vicuna-7b show no significant race- or sex-related salary associations, whereas several models exhibit significant associations with sex, race, or both.A p-value below 0.05 indicates an attribute-prediction relationship; Oasst-12b’s strong result may instead reflect poor, near-random predictive performance.Source: §8.2 · pp. 45–46, Table 23 · p. 45

8.3 Preference Bias in Subjective Choices

  • ERNIE achieves an overall preference RtA exceeding 90%, while most models score around 0.6 and forced selection lowers RtA relative to the plain baseline.Ideology questions produce higher RtA than culture/lifestyle questions because political content is more sensitive; refusing to answer is treated as neutrality.Source: §8.3 · pp. 46–47, Table 25 · p. 47

9 Assessment of Robustness

The assessment examines LLM robustness to natural input noise and out-of-distribution (OOD) challenges, covering labeled and open-ended tasks, OOD detection, and OOD generalization. Results show substantial variation across models and perturbations, with strong performance in some settings but persistent limitations under distribution shifts.

9 Assessment of Robustness

  • The robustness assessment distinguishes ordinary-input robustness from malicious attacks and evaluates natural noise and OOD resilience through detection, generalization, and downstream-task performance.The benchmark uses AdvGLUE for labeled tasks, ADVINSTRUCTION for open-ended tasks, and separate OOD detection and generalization evaluations.Source: §9 · p. 48

9.1 Robustness against Input with Natural Noise

  • Natural-noise evaluation covers linguistic variations and errors in both labeled natural-language-understanding tasks and open-ended natural-language-generation tasks.AdvGLUE includes 14 perturbation strategies spanning word-level, sentence-level, and human-crafted changes.Source: §9.1 · p. 48, §9.1.1 · p. 48

9.1.1 Ground-Truth Labeled Task Performance

  • On labeled tasks with natural noise, PaLM 2 maintains 76.3% accuracy before and after perturbations, while PaLM 2 and GPT-4 substantially exceed other models in robustness score.Llama2-7b has the lowest ASR at 6.8% but below-50% accuracy, showing that low attack success can coexist with poor utility.Source: §9.1.1 · p. 49, Figure 11 · p. 49, Table 27 · p. 49

9.1.2 Performance in Open-ended Tasks

  • Under open-ended natural noise, Llama2-70b achieves 97.64% average semantic similarity, whereas Vicuna-7b falls below 90%.Word substitutions cause the largest similarity drops, grammatical errors the least, and most disturbances reduce Llama2-70b and ChatGPT similarity by under 3%.Source: §9.1.2 · pp. 50–51, Table 29 · p. 51, Figure 12 · p. 51

9.2 Assessing Out of Distribution (OOD) Task Resilience

  • OOD robustness concerns inputs outside training distributions, including temporal gaps, syntactic anomalies, semantic divergence, and synthetic or hybrid languages, requiring both detection and effective performance.OOD detection recognizes inputs the model cannot answer, whereas OOD generalization adapts performance to unseen distributions.Source: §9.2 · pp. 51–52

9.2.1 OOD Detection

  • Most models achieve some OOD detection, with refusal rates above 40%, while GPT-4 ranks first, Baichuan-13b scores only 0.4%, and Vicuna improves with size.The evaluation uses the RtA metric, rewarding appropriate refusal of inputs beyond the model’s capabilities.Source: §9.2.1 · p. 53, Table 30 · p. 53

9.2.2 OOD Generalization

  • OOD generalization remains difficult: GPT-4 performs exceptionally, DDXPlus F1 scores range from 0.9 to 0.3, smaller Llama-13B outperforms Llama-70B, and performance remains below humans.The results generally show positive ID–OOD correlation, but large models may trade specificity for generalization.Source: Table 31 · p. 53, §9.2.2 · p. 54

10 Assessment of Privacy Preservation

The assessment examines LLM privacy through two dimensions: privacy awareness, or recognizing and managing privacy-related requests, and privacy leakage, or exposing sensitive information embedded in training data. Results show that explicit privacy prompts improve refusal behavior, while model size and few-shot context can increase leakage risk.

10 Assessment of Privacy Preservation

  • The assessment separates privacy awareness from privacy leakage, evaluating privacy-policy recognition and refusal behavior alongside whether prompts can elicit private information from training data.The framework uses two privacy-awareness tasks and an Enron Email Dataset leakage test with zero-shot and few-shot prompts.Source: §10 · p. 55, Figure 14 · p. 55, §10.2 · p. 58

10.1 Privacy Awareness

  • ChatGPT led agreement with human privacy judgments, while explicit privacy-policy prompts generally increased refusal rates; Llama2 achieved 100% RtA in both prompting conditions.Oasst-12b’s RtA increased from 38.9% under Normal prompting to 88.6% under Augmentation prompting.Source: Table 34 · p. 57, Table 33 · p. 57, §10.1 · p. 58

10.2 Privacy Leakage

  • Privacy leakage varies substantially: Llama2 variants nearly always refused email disclosure in zero-shot settings, whereas GPT-4, ChatGPT, and Vicuna often had high disclosure, with GPT-4 reaching 68% TD under Template D.Llama2-7b and Llama2-13b maintained refusal rates above 95% under five-shot prompting, while vulnerable models often exceeded 48% TD under five-shot prompting.Source: §10.2 · p. 58, Table 35 · p. 59

11 Assessment of Machine Ethics

TrustLLM assesses machine ethics through implicit ethics, explicit ethics, and awareness, examining moral judgments, ethical actions, and emotional or social understanding. Results show substantial value misalignment, ambiguity-sensitive performance, and uneven awareness across dimensions.

11 Assessment of Machine Ethics

  • The assessment distinguishes implicit ethics as judging moral correctness, explicit ethics as acting in ethical scenarios, and awareness as introspective and social understanding.The section’s goals are to compare LLM values with human values, evaluate decisions in particular scenarios, and measure emotional awareness.Source: Figure 15 · p. 60, §11 · p. 60

11.1 Implicit Ethics

  • GPT-4 leads implicit-ethics evaluation, but average accuracy below 0.7 indicates substantial misalignment between LLM judgments and human values.On SOCIAL CHEMISTRY 101, GPT-4 scores below 0.7 while Vicuna-13b averages just above 0.5; on ETHICS, Koala-13b and Oasst-12b fall below 0.5.Source: §11.1 · p. 61, Table 37 · p. 62

11.2 Explicit Ethics

  • Most models perform exceptionally well on low-ambiguity moral choices, but performance varies sharply on high-ambiguity scenarios, with GPT-4 dropping over 40%.Llama2 models rank highest on high-ambiguity scenarios, while more than half of models perform worse there than on low-ambiguity tasks.Source: Table 37 · p. 62, §11.2 · p. 63

11.3 Awareness

  • Awareness is evaluated across capability, mission, emotion, culture, and perspective dimensions, using curated multiple-choice, binary, and open-ended questions.The framework distinguishes awareness from human self-awareness while assessing models’ recognition of abilities, missions, emotions, cultural norms, and others’ perspectives.Source: §11.3 · pp. 63–64
  • GPT-4 and GLM-4 generally outperform open-source models on awareness, although performance remains uneven across dimensions and models.Only GPT-4 and GLM-4 exceed 80% in capability awareness; GPT-4 reaches 80.22% in mission awareness, while GPT-4 and ChatGPT exceed 90% in emotion awareness.Source: Table 45 · p. 68, §11.3 · p. 68

12 Discussion of Transparency

Transparency is central to responsible LLM development because stakeholders need to understand model capabilities, limitations, operations, and outputs. The section distinguishes informational, normative, and relational/social perspectives, reviews transparency approaches, and identifies explainability, participant adaptation, and public awareness as key challenges.

Motivation

  • Transparency supports responsible LLM development by helping stakeholders understand models’ capabilities, limitations, operations, and output management amid harmful, misleading, environmental, and socioeconomic consequences.The paper frames responsible development and transparency as mutually reinforcing.Source: §12 · p. 69

Different perspectives on transparency

  • Transparency has informational, normative, and relational/social perspectives that are interconnected but emphasize disclosure, conduct, and relationships between agents and recipients.Informational transparency concerns relevant model or system details; normative transparency evaluates public actors’ conduct; relational/social transparency depends on institutional information exchange and interpretation.Source: §12 · p. 69

Related works

  • Transparency research addresses models and their training data through documentation and innovative architectures, while also explaining internal mechanisms through chain-of-thought and explainable-AI methods.These approaches target model and dataset disclosure, intermediate reasoning, internal circuits, knowledge storage, and decision-making processes.Source: §12 · p. 69

Challenges

  • LLM explainability is difficult because transformer models scale to hundreds of billions or trillions of parameters and tokens, increasing sophistication while obscuring predictions.Fine-tuning can shape behaviors for tasks such as summarization, question answering, and code generation, further complicating explanation.Source: §12 · p. 69
  • Effective transparency must adapt information to diverse participants’ needs and address inaccurate public perceptions to reduce misuse and security risks.Relevant participants include developers, executives, regulators, auditors, end-users, and directly or indirectly affected individuals.Source: §12 · p. 70

Diverse approaches, valuable insights

  • Transparency should be designed across application architecture, data processing, prompting, fine-tuning, and post-use reporting, with clear explanations, model documentation, and enabled audits.Recommended reporting covers inputs and outputs, training methods and data sources, development context, intended applications, and ethical considerations.Source: §12 · p. 70

13 Discussion of Accountability

The discussion frames LLM accountability as both a traditional computer-system problem and a modern challenge involving traceability, copyright, censorship, and unclear legal responsibility. It highlights barriers from distributed development and opaque failures, while examining watermarking’s benefits and risks and unresolved liability frameworks.

Traditional accountability barriers

  • Traditional accountability barriers include distributed responsibility, opaque bugs causing stereotypes or hallucinations, user scapegoating of models, and disclaimers that do not waive companies’ liability.LLMs combine many contributors and training-data sources, lack a standard usage method, and conceal defects behind black-box architectures.Source: §13 · p. 71

Machine-generated text and watermarks

  • Watermarking improves traceability and deters misuse of machine-generated text, but centralized detection can threaten privacy and small quality perturbations may impair high-stakes code generation.Watermarks embed detectable statistical patterns during generation, supporting accountability in journalism, academic writing, and other authenticity-sensitive fields.Source: §13 · pp. 71–72
  • Watermark research uses green/red vocabulary partitioning, Gumbel sampling, paraphrase-robust detection, and unbiased or NS methods to balance identifiability with text quality [677] [679] [680] [682] [684].The initial method increases green-token probability, whereas Gumbel sampling preserves the probability distribution; later work targets robustness and quality.Source: §13 · p. 72

Copyright

  • LLM training raises unresolved copyright questions for scraped texts and imitated artistic styles, while laws governing copyright in AI-generated content remain vague in most countries.The discussion calls for clearer legal frameworks covering both training-set infringement and ownership or infringement of generated content.Source: §13 · p. 72

Censorship

  • LLM censorship must balance protection from harmful or biased authoritative content against free speech, especially because biased or malicious training data can shape public opinion.The models’ authoritative tone can amplify harmful information, while training inputs may intentionally or unintentionally propagate ideologies.Source: §13 · p. 72

Legal accountability

  • Legal accountability remains ambiguous because responsible actors and consequences are unclear, while liability costs may encourage restrictions, burden smaller firms, and reinforce monopolistic advantages.Section 230’s applicability to LLMs is contested, and GPT-4 training reportedly exceeded 100 million dollars.Source: §13 · pp. 72–73

14 Open Challenges

TRUSTLLM identifies limitations in language coverage, prompt and instruction sensitivity, certification, knowledge updating, transparency, accountability, and broader trustworthiness assurance. Addressing these challenges requires multilingual evaluation, scalable certification, reliable knowledge editing, lifecycle transparency, accountability mechanisms, and stronger safeguards.

Languages bias

  • English-only evaluations overlook linguistic and cultural variation, may mismeasure non-English trustworthiness, and can bias comparisons involving Chinese LLMs.GPT-4 safety mechanisms can also be bypassed by translating unsafe English inputs into low-resource languages.Source: §14 · p. 74

Prompt sensitivity and instruction following

  • Prompt sensitivity and instruction-following failures can produce inconsistent evaluations, unfair comparisons, format errors, and biases from models’ inability to satisfy complex constraints.TRUSTLLM mitigates these effects with carefully crafted prompts, expert review, and GPT-4/ChatGPT-based evaluation in some cases.Source: §14 · p. 74

Certification of LLMs

  • Rigorous LLM certification remains impractical because existing verification methods do not scale, retraining is prohibitively expensive, and natural-language specifications are difficult to formalize.TRUSTLLM therefore cannot guarantee worst-case trustworthiness under adversarial or malicious inputs.Source: §14 · p. 74

Knowledge grounding and editing

  • Reliable knowledge grounding and editing must account for temporal shifts and real-world updates while mitigating exposure bias that favors existing information over newly incorporated facts.Relevant facts may need to enter training losses to localize affected knowledge elements and their ripple effects.Source: §14 · p. 75

Transparency of LLMs

  • LLM transparency is challenged by model complexity, diverse participant needs, and inaccurate public awareness, requiring lifecycle measures from pretraining data disclosure through comprehensive model reports.Transparency concerns both the models themselves and their internal mechanisms and decisions.Source: §14 · p. 75

LLMs accountabilities

  • LLM accountability is hindered by complex development, opacity, scapegoating, liability gaps, concentrated training costs, weak text attribution, copyright issues, and harmful training data.These barriers span both traditional computer-system accountability and newer responsibilities specific to LLMs.Source: §14 · p. 75

Others

  • Further challenges include interactions among trustworthiness dimensions, comprehensive metrics, human agency and oversight, fragile safety guardrails, non-human-language trustworthiness, and safety-aware training.ChatGPT and LLAMA-2 guardrails can reportedly be removed through fine-tuning with few examples or benign instruction datasets [452], while compute-efficient training may support safer development [724].Source: §14 · p. 75

15 Future Work

Future work expands TRUSTLLM through broader evaluation coverage, domain-specific and multimodal trustworthiness research, ecosystem development, and cryptographic approaches. Key challenges include modality alignment, hallucination, tool and IoT risks, end-to-end guarantees, efficiency, and federated collaboration.

Limitations and future plans

  • TRUSTLLM will broaden prompts, datasets, tasks, models, domains, taxonomy sections, and supporting ecosystem tools, including a continuously updated leaderboard.Domain-specific evaluations will cover education, healthcare, finance, cybersecurity, and other scientific areas, while new sections may address consciousness and related topics.Source: §15 · p. 76

Beyond LLM

  • TRUSTLLM’s principles can extend beyond LLMs to multimodal foundation models, IoT, time series, mobile computing, and mobile edge networks, requiring scenario-specific solutions.Multimodal systems introduce challenges distinct to specialized environments and application scenarios.Source: §15 · p. 76
  • Future multimodal research must address modality alignment, human-intent data creation, hallucination, architecture and knowledge gaps, trustworthiness evaluation, and safe tool usage.Misalignment can cause incorrect concept identification; low image resolution, unsupported requests, and domain gaps can worsen hallucinations, while external tools introduce error propagation and undesirable behaviors.Source: §15 · p. 76
  • Trustworthy multimodal systems for IoT edge intelligence must handle resource constraints, decentralized operation, attack surfaces, incomplete or imbalanced data, noise, changing conditions, and real-time requirements.Tailored designs are needed across applications such as human activity recognition, smart grids, and public safety.Source: §15 · p. 78

Cryptographic Techniques for Enhancing LLM Trustworthiness

  • Cryptographic techniques such as secure computation and zero-knowledge proofs could strengthen security-critical trustworthiness tasks, but practical deployment faces major challenges.These challenges include end-to-end trustworthiness beyond training integrity, cryptographic inefficiency, and the need for co-design aligned with application requirements.Source: §15 · p. 78
  • Future cryptography-based LLM systems should support secure federation of models and data, including collaborative training, confidential generation, and private model adaptation.Model owners and data holders may collaborate without trusting one another.Source: §15 · p. 78

16 Conclusion

TRUSTLLM presents a comprehensive framework, benchmark, and evaluation of LLM trustworthiness, showing its interconnection with utility and differences between open-weight and proprietary models. The conclusion emphasizes collaboration and transparency to improve reliability and support responsible deployment.

16 Conclusion

  • TRUSTLLM spans eight trustworthiness dimensions, benchmarks six, and evaluates 16 mainstream LLMs across diverse datasets.The study combines principles, benchmarks, evaluation, analysis, and discussion of open challenges and future directions.Source: §16 · p. 79
  • Trustworthiness is interconnected with utility, while many LLMs exhibit excessive trustworthiness and open-weight and proprietary models show notable performance differences.These findings qualify trustworthiness improvements by highlighting both functional effectiveness and variation across model types.Source: §16 · p. 79
  • Collective collaboration among LLM developers and greater transparency in trustworthy-related technologies are necessary to improve model reliability and foster responsible deployment.The conclusion frames transparency and cooperation as central to building models that users can trust across real-world applications.Source: §16 · p. 79

Related papers

Loading 2401.05561v6…