PaperlayerView exact revision on arXiv

Source-linked AI summary

TrustLLM: Trustworthiness in Large Language Models

Yue Huang, Lichao Sun, Haoran Wang, Siyuan Wu, Qihui Zhang, Yuan Li, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, Xiner Li, Zhengliang Liu, Yixin Liu, Yijue Wang, Zhikun Zhang, Bertie Vidgen, Bhavya Kailkhura, Caiming Xiong, Chaowei Xiao, Chunyuan Li, Eric Xing, Furong Huang, Hao Liu, Heng Ji, Hongyi Wang, Huan Zhang, Huaxiu Yao, Manolis Kellis, Marinka Zitnik, Meng Jiang, Mohit Bansal, James Zou, Jian Pei, Jian Liu, Jianfeng Gao, Jiawei Han, Jieyu Zhao, Jiliang Tang, Jindong Wang, Joaquin Vanschoren, John Mitchell, Kai Shu, Kaidi Xu, Kai-Wei Chang, Lifang He, Lifu Huang, Michael Backes, Neil Zhenqiang Gong, Philip S. Yu, Pin-Yu Chen, Quanquan Gu, Ran Xu, Rex Ying, Shuiwang Ji, Suman Jana, Tianlong Chen, Tianming Liu, Tianyi Zhou, William Wang, Xiang Li, Xiangliang Zhang, Xiao Wang, Xing Xie, Xun Chen, Xuyu Wang, Yan Liu, Yanfang Ye, Yinzhi Cao, Yong Chen, Yue Zhao

arXiv:2401.05561v6cs.CL

TL;DR

LLMs’ expanding capabilities create substantial trustworthiness concerns, motivating systematic assessment. TRUSTLLM develops principles across eight dimensions, benchmarks six of them, and evaluates 16 mainstream LLMs across diverse datasets, finding positive trustworthiness–utility relationships, proprietary-model advantages, and frequent over-alignment.

  • Problem

    LLMs’ broad generative capabilities raise concerns about inaccurate outputs, misuse, and attacks, while existing trustworthiness taxonomies do not fully or cleanly cover the relevant aspects.

  • Method

    TRUSTLLM proposes principles across eight dimensions, establishes a benchmark across six dimensions, and evaluates 16 mainstream LLMs on more than 30 datasets.

  • Results

    Trustworthiness and utility are positively related overall; proprietary LLMs generally outperform most open-weight models, while many LLMs exhibit over-alignment that harms utility.

  • Takeaways & Limitations

    Trustworthiness assessment should consider both model behavior and the technologies supporting trustworthiness, with transparency needed for analyzing their effectiveness.

  • Takeaways & Limitations

    The evaluations do not provide rigorous certification and cannot guarantee reflection of worst-case LLM behavior.

Abstract

from arXiv · show

Large language models (LLMs), exemplified by ChatGPT, have gained considerable attention for their excellent natural language processing capabilities. Nonetheless, these LLMs present many challenges, particularly in the realm of trustworthiness. Therefore, ensuring the trustworthiness of LLMs emerges as an important topic. This paper introduces TrustLLM, a comprehensive study of trustworthiness in LLMs, including principles for different dimensions of trustworthiness, established benchmark, evaluation, and analysis of trustworthiness for mainstream LLMs, and discussion of open challenges and future directions. Specifically, we first propose a set of principles for trustworthy LLMs that span eight different dimensions. Based on these principles, we further establish a benchmark across six dimensions including truthfulness, safety, fairness, robustness, privacy, and machine ethics. We then present a study evaluating 16 mainstream LLMs in TrustLLM, consisting of over 30 datasets. Our findings firstly show that in general trustworthiness and utility (i.e., functional effectiveness) are positively related. Secondly, our observations reveal that proprietary LLMs generally outperform most open-source counterparts in terms of trustworthiness, raising concerns about the potential risks of widely accessible open-source LLMs. However, a few open-source LLMs come very close to proprietary ones. Thirdly, it is important to note that some LLMs may be overly calibrated towards exhibiting trustworthiness, to the extent that they compromise their utility by mistakenly treating benign prompts as harmful and consequently not responding. Finally, we emphasize the importance of ensuring transparency not only in the models themselves but also in the technologies that underpin trustworthiness. Knowing the specific trustworthy technologies that have been employed is crucial for analyzing their effectiveness.

– A PRINCIPLE AND BENCHMARK

TRUSTLLM establishes a benchmark for evaluating LLM trustworthiness from principles spanning multiple dimensions. Its benchmark covers six dimensions and includes mainstream proprietary and open-weight models across more than 30 datasets.

  • TRUSTLLM defines trustworthiness across eight aspects, including truthfulness, safety, fairness, robustness, privacy, machine ethics, transparency, and accountability.
  • TRUSTLLM is presented as a comprehensive study combining organizing principles, benchmarking, evaluation, and analysis of mainstream LLM trustworthiness.

1 Introduction

LLMs’ broad capabilities and deployment create trustworthiness concerns, while existing taxonomies do not fully cover or cleanly organize these concerns. TRUSTLLM responds with an eight-aspect taxonomy and an integrated benchmark and evaluation framework.

  • LLMs’ complex outputs and generative capabilities can produce inaccurate information and enable misuse, including misinformation, cyberattacks, and jailbreaking threats.
  • Benchmarking trustworthiness requires defining comprehensive elements and practical evaluation methods across multiple perspectives.
  • Earlier taxonomies either omitted aspects of trustworthiness or used overlapping fine-grained categories that complicated clear benchmark construction.
  • TRUSTLLM unifies a survey, organizing principles, a benchmark, and evaluation of trustworthiness in mainstream LLMs.
  • The framework identifies eight trustworthiness aspects after reviewing 600 papers and separates utility from these dimensions.
  • The benchmark evaluates 16 LLMs across more than 30 datasets and summarizes their trustworthiness rankings in Figure 1.

2 Observations and Insights

TRUSTLLM finds that trustworthiness is often associated with utility, but models differ substantially across dimensions and can become overly cautious. Proprietary models generally outperform most open-weight models, while individual results reveal important weaknesses in truthfulness, safety, fairness, robustness, privacy, and machine ethics.

  • Overall Observations: Trustworthiness and utility are positively correlated particularly in tasks involving moral behavior classification, stereotype recognition, and resilience against adversarial attacks.
  • Overall Observations: 57% of Llama2-7b responses refused prompts that were not harmful, illustrating how over-alignment can compromise utility.
  • Overall Observations: Proprietary LLMs generally outperform most open-weight models in trustworthiness, although the Llama2 series surpasses proprietary models in many tasks.
  • Truthfulness: External knowledge substantially improves truthfulness, while hallucinations differ between multiple-choice and open-ended tasks.
  • Safety: Most open-source LLMs lag proprietary models in jailbreak, toxicity, and misuse safety, and models do not uniformly resist different jailbreak attacks.
  • Fairness and Robustness: GPT-4 achieves 65% overall accuracy on stereotype identification, while robustness varies from 88% to 97.64% semantic similarity after perturbation.
  • Robustness: GPT-4 exceeds 80% RtA in OOD detection and 92% average F1 in OOD generalization, whereas weaker models reach 0.4% RtA and around 30% F1.
  • Privacy and Machine Ethics: Most LLMs show privacy awareness, while implicit ethical-task accuracy is below 70% for most models and GPT-4 averages 94% across four awareness datasets.

3 Background

This background reviews LLM capabilities, evaluation practices, and emerging efforts to improve trustworthiness. It motivates comprehensive trustworthiness benchmarks by noting that existing benchmarks often cover only specific sections.

  • LLM capabilities: LLMs exhibit emergent abilities for complex tasks through scaling, Transformer architectures, and instruction-following or reasoning capabilities.Pre-training, scaling laws, and subsequent training methods contribute to these abilities.
  • Evaluation landscape: LLM evaluation spans understanding, reasoning, generation, multilingual, and other tasks across diverse datasets and benchmarks.Evaluation uses both automatic and human methods across multiple domains.
  • Evaluation landscape: Traditional text-generation metrics such as BLEU and ROUGE can score accurate outputs poorly when multiple valid generations exist.This limitation arises because n-gram overlap may not capture latent content planning or selection.
  • Trustworthiness efforts: Developers use data filtering, human evaluation, red teaming, safety fine-tuning, and content moderation to improve LLM trustworthiness.The reviewed efforts span OpenAI, Meta, Anthropic, Microsoft, Google, and Baichuan.
  • Benchmarking gap: Existing trustworthiness benchmarks are often not comprehensive enough, motivating comparison and development of broader evaluations.The cited benchmarks commonly address specific sections of trustworthiness rather than the full construct.

4 Guidelines and Principles for Trustworthiness Assessment of LLMs

This section develops guidelines and principles for assessing LLM trustworthiness across multiple dimensions. It combines literature analysis with operational benchmark definitions for selected dimensions and tasks.

  • Assessment principles: A literature review and qualitative coding process distill themes and concepts defining trustworthiness in LLMs.Researchers searched multiple databases, independently screened papers, and cross-verified the coding.
  • Assessment principles: The framework identifies eight trustworthiness dimensions: truthfulness, safety, fairness, robustness, privacy, machine ethics, transparency, and accountability.The dimensions are presented with definitions and keywords in the framework’s tables.
  • Truthfulness: Truthfulness concerns accurate representation of information, facts, and results, including misinformation and hallucination across multiple task settings.The assessment covers internal and retrieved knowledge, question answering, dialogue, and summarization.
  • Safety: Safety is defined as avoiding unsafe or illegal outputs while maintaining healthy conversations, including resistance to jailbreaks and misuse.TRUSTLLM also evaluates over-alignment and toxicity after compromised outputs.
  • Robustness: Robustness concerns performance under varying conditions and handling of exceptions or unexpected inputs, but is not inherent in current LLMs.The section cites failures from seemingly simple inputs, including emojis.
  • Privacy: Privacy safeguards human and data autonomy, identity, and dignity, with evaluation focused on privacy awareness and potential leakage.The guideline addresses recognition and handling of privacy-sensitive scenarios.
  • Machine ethics: Machine ethics is divided into implicit ethics, explicit ethics, and emotional awareness because comprehensive ethical theory remains insufficient.Implicit ethics concerns internal values and moral judgments; the framework evaluates alignment with human ethical standards.
  • Accountability: Accountability requires explanations and justifications for LLM behavior, while transparency and distributed responsibility create practical barriers.The section applies Nissenbaum’s four barriers to LLMs and discusses training-data responsibility.

5 Preliminaries of TRUSTLLM

TRUSTLLM evaluates a curated set of proprietary and open-weight LLMs using diverse tasks, datasets, prompts, and metrics. Its experimental design separates classification from generation and aims to measure plain model performance.

  • Benchmark design: The benchmark evaluates proprietary and open-weight LLMs using selected models, datasets, tasks, prompt templates, and evaluation methods.The experimental setup and model selection are introduced as the core preliminaries.
  • Model selection: The study curates 16 LLMs spanning model sizes, training data, methodologies, and functional capabilities.The collection is intended to represent a broad evaluation landscape.
  • Model selection: The evaluated models include ChatGPT, GPT-4, Vicuna, ChatGLM2, Koala, Llama2, WizardLM, Oasst, Baichuan, ERNIE, Mistral 7B, and PaLM 2.The set includes models developed by OpenAI, LMSYS, Tsinghua and Zhipu, BAIR, Meta, Microsoft, LAION, Baichuan, Baidu, Mistral AI, and Google.
  • Experimental settings: Tasks are grouped into Generation and Classification, using temperature 0 for classification and temperature 1 for generation.The generation setting supports more diverse outputs and exploration of potential worst-case scenarios.
  • Experimental settings: The benchmark introduces 30 datasets selected to evaluate diverse trustworthy capabilities across dimensions.Dataset descriptions and specifications are listed in Table 4.
  • Experimental settings: Prompts generally require specific keywords for efficient automated assessment, omit few-shot examples, and exclude advanced prompting methods.Privacy leakage evaluation is the exception, where prompts aim to increase the probability of privacy leakage.

6 Assessment of Truthfulness

The truthfulness assessment examines misinformation, hallucination, and sycophancy, comparing internal-knowledge responses with fact-checking supported by external evidence. Results show that internal knowledge is unreliable, retrieval improves performance, and hallucination remains common across generation tasks.

  • Assessment scope: The assessment covers misinformation from internal knowledge or retrieved evidence, hallucination across four tasks, and two forms of sycophancy.The hallucination tasks are multiple-choice QA, open-ended QA, knowledge-grounded dialogue, and summarization.
  • Integrating External Knowledge: Retrieval-augmented LLMs can outperform models without retrieval by a large margin with fewer parameters on knowledge-intensive tasks.The paper evaluates zero-shot fact-checking by presenting claims together with ground-truth evidence.
  • Using Merely Internal Knowledge: Zero-shot QA using only internal knowledge is challenging for all evaluated LLMs, which can therefore produce untruthful responses.No model consistently leads across all four datasets; GPT-4 has the strongest average performance, while CODAH remains difficult for commonsense reasoning.
  • Integrating External Knowledge: External evidence improves fact-checking performance for all LLMs compared with relying solely on internal knowledge.GPT-4 achieves the highest average performance across the four fact-checking datasets, followed closely by Vicuna-33b and ChatGPT.
  • Hallucination: Most LLMs struggle across the four hallucination tasks despite producing coherent and natural language.GPT-4 performs best on multiple-choice and summarization tasks, whereas ChatGLM2 performs best on open-ended QA and knowledge-grounded dialogue.

6.3 Sycophancy in Responses

The section evaluates persona-based and preference-driven sycophancy, then examines whether models correct factual errors embedded in user prompts. Results identify different leaders across sycophancy tests and show strong factual-error correction by GPT-4 and Llama2-70b.

  • Sycophancy definitions: Sycophancy is response alignment with a user’s perspective even when that perspective is not objectively correct.The evaluation distinguishes persona-based sycophancy from preference sycophancy, which tests response consistency under changed preferences.
  • Persona-based sycophancy: Persona sycophancy is evaluated by comparing generated responses with paired sycophantic and non-sycophantic answers.The metric is sim(r) = cos(r, ns) − cos(r, s), averaged across the dataset.
  • Preference-driven sycophancy: Preference sycophancy is measured as the percentage of opinion changes across reversed-preference prompt pairs, with lower values indicating less sycophancy.The dataset contains 120 pairs of opposing opinions, and ChatGPT assesses whether response pairs convey the same meaning.
  • Results: Llama2-70b achieves the highest persona-sycophancy performance, while ChatGPT achieves the best preference-sycophancy performance.The persona result is based on the largest similarity score, whereas the preference result is based on the smallest percentage change.
  • Adversarial factuality: The adversarial-factuality dataset contains injected misinformation across multiple subjects so models can be tested on correcting errors in user prompts.The authors produced 209 entries spanning fields including computer science, biology, chemistry, physics, finance, arts, and history.

7 Assessment of Safety

TRUSTLLM evaluates safety through jailbreak resistance, exaggerated safety, toxicity, and misuse resilience. Results show substantial variation across models and attacks, with trade-offs between refusing harmful requests and answering benign ones.

  • Evaluation scope: The safety evaluation covers jailbreak attacks, exaggerated safety, toxicity after compromise, and misuse scenarios.The benchmark uses a taxonomy of five jailbreak classes and 13 subclasses, alongside XSTEST and misuse datasets.
  • Jailbreak: 13 jailbreak attack methods are included in the extensible JAILBREAK TRIGGER dataset.The dataset transforms malicious prompts into modified prompts intended to elicit harmful responses.
  • Jailbreak: Leetspeak and multitask attacks exhibit relatively high success rates, motivating stronger red-team testing against these methods.The evaluation also finds substantial variation in safety across LLMs; Baichuan-13b fails to reject harmful instructions in 70% of QBB and ITC cases.
  • Exaggerated Safety: Llama2 models show the most severe exaggerated-safety problem, while greater over-safety coincides with better resistance to misuse and jailbreaks.The findings indicate that current LLMs struggle to balance refusing harmful requests with answering harmless ones.
  • Toxicity: GPT-4, Vicuna-13b, and ChatGPT have the highest average toxicity, whereas ERNIE remains below 0.1 on average and never exceeds 0.3.For most data points across models, toxicity remains below 0.2.
  • Misuse: Most LLMs reject misuse requests with RtA values above 0.7, but Baichuan-13b remains below 0.2 across all three misuse datasets.The Llama2 series performs best overall, followed by ChatGPT and GPT-4.

8 Assessment of Fairness

TRUSTLLM assesses fairness through stereotype agreement, stereotype recognition, disparagement, and preference neutrality. Results vary substantially across models, with strong performance on some tasks but persistent classification and attribute-association disparities.

  • Evaluation scope: Fairness evaluation covers stereotype agreement, stereotype recognition, stereotype queries, salary-based disparagement, and preference choices.The stereotype tasks use CrowS-Pair, StereoSet, and Do-Not-Answer data.
  • Stereotype agreement: GPT-4, Ernie, Llama2-13b, and Vicuna-13b generally show lower agreement with stereotypical sentences than several other models.Oasst-12b, Chatglm2, and Koala-13b show high agreement rates in most categories.
  • Stereotype recognition: GPT-4 achieves accuracy above 0.6 for stereotype classification, while most other models barely reach or fall below 0.5.Oasst-12b, Koala-13b, Baichuan-13b, and ChatGLM2 have overall accuracy below 0.4.
  • Stereotype query test: Half of the evaluated models refuse 100% of stereotype-related queries, and most models have RtA above 90%.These results indicate effective refusal behavior on the stereotype query task.
  • Disparagement: Sex or race is significantly associated with salary classification for multiple models, including Ernie, Llama2-13b, Chatglm-2, and Llama2-7b.The study uses p < 0.05 as the threshold for rejecting the null hypothesis and indicating a stronger-than-random relationship.
  • Preference bias: The preference evaluation compares a plain baseline with force selection to test neutrality under ordinary and mandatory-choice prompts.The force-selection template evaluates performance under the stated worst-case scenario.
  • Preference bias: ERNIE achieves overall preference-task RtA above 90%, while Ideology prompts receive more refusals than Culture/Lifestyle prompts.The difference is attributed to the greater sensitivity of political content.

9 Assessment of Robustness

TrustLLM assesses robustness under natural noise and out-of-distribution conditions using labeled and open-ended tasks. Results show substantial variation across models, with robustness not determined solely by model size.

  • Scope: Robustness measures stability and performance under diverse inputs, including noise, interference, adversarial attacks, and distribution shifts.
  • Evaluation design: The evaluation covers natural-noise robustness in labeled downstream tasks and open-ended tasks, plus out-of-distribution generalization.AdvGLUE is used for labeled tasks, while ADVINSTRUCTION targets open-ended evaluation without ground-truth labels.
  • Evaluation design: RS combines adversarial accuracy and attack success rate as RS = Acc(adv) − ASR.ASR measures benignly correct samples that become misclassified after perturbation.
  • Labeled tasks: 76.3% accuracy: PaLM 2 maintains this rate before and after perturbations, while Llama2-7b has the lowest disturbance ASR at 6.8% but accuracy below 50%.
  • Labeled tasks: PaLM 2 and GPT-4 achieve the highest RS, while ChatGLM2-6b and Llama2-7b outperform Vicuna-33b, indicating model size alone may not determine robustness.
  • Open-ended tasks: 97.64%: Llama2-70b has the highest average semantic similarity, whereas Vicuna-7b falls below 90% and word substitutions cause the largest similarity drops.Llama2-70b and ChatGPT generally lose less than 3% under disturbances, while Vicuna-7b often loses more than 10%.
  • OOD robustness: GPT-4 ranks first for OOD detection, while Baichuan-13b reaches only 0.4% RtA; OOD generalization F1 scores range from 0.9 to 0.3 on DDXPlus.Smaller models such as Llama-13B outperform Llama-70B on both OOD datasets.

10 Assessment of Privacy Preservation

TrustLLM evaluates privacy awareness and privacy leakage separately. Models often respond more appropriately when privacy rules are made explicit, but leakage susceptibility varies substantially across models and prompting conditions.

  • Scope: Privacy evaluation covers awareness of privacy-related requests and accidental disclosure of private information from training data.
  • Privacy awareness: The awareness evaluation compares normal prompts with augmentation prompts that explicitly require compliance with privacy policies.The benchmark includes agreement with human privacy judgments and scenario-based refusal behavior.
  • Privacy awareness: Augmentation raises refusal rates for many models; Oasst-12b increases from 38.9% to 88.6%, while Llama2 achieves 100% RtA under both conditions.
  • Privacy leakage: Privacy leakage differs sharply across models: Llama2 variants nearly always refuse Enron email requests, whereas GPT-4, ChatGPT, and Vicuna show high disclosure rates.
  • Privacy leakage: Larger models may face greater leakage risks within similar architectures, and few-shot prompting generally increases TD and conditional disclosure.

11 Assessment of Machine Ethics

TrustLLM examines machine ethics through implicit ethics, explicit ethics, and awareness. Results show uneven alignment with human judgments, especially in ambiguous scenarios and across social or emotional dimensions.

  • Scope: Machine ethics is divided into implicit ethics, explicit ethics, and awareness to examine moral judgments, ethical actions, and emotional understanding.
  • Implicit ethics: Implicit ethics evaluates whether embedded model values align with human ethical standards using moral-action judgment tasks.The evaluation uses ETHICS and SOCIAL CHEMISTRY 101 scenarios labeled by moral correctness or social norms.
  • Implicit ethics: GPT-4 leads SOCIAL CHEMISTRY 101 but remains below 0.7 accuracy, while models commonly perform worse on neutral social norms.
  • Implicit ethics: Below 0.7 average accuracy: all LLMs show substantial misalignment with human value judgments on the ETHICS task.GPT-4 performs best, while Koala-13b and Oasst-12b approach random-guess accuracy.
  • Explicit ethics: Explicit ethics evaluates whether LLMs choose morally correct reactions in scenarios, with MoralChoice separating low-ambiguity and high-ambiguity cases.High-ambiguity cases are designed so models should avoid direct choices when neither action is clearly preferable.
  • Explicit ethics: GPT-4 drops by over 40% from low-ambiguity to high-ambiguity scenarios, and more than half of models perform worse in the ambiguous setting.
  • Awareness: Awareness is generally incomplete: only GPT-4 and GLM-4 exceed 80% accuracy on capability awareness, while Llama and Vicuna remain below 50%.

12 Discussion of Transparency

The paper frames transparency as central to responsible LLM development, covering both disclosed information and internal decision-making mechanisms. It identifies complexity, participant needs, and public awareness as major challenges.

  • Motivation: Transparency is central to responsible LLM development because users must understand model capabilities, limitations, operation, and output management.The paper links transparency to harmful content, misinformation, and broader environmental and socioeconomic consequences.
  • Conceptualization: Transparency has informational, normative, relational, and social dimensions rather than one universally accepted definition.The paper operationalizes transparency through three perspectives, beginning with informational disclosure.
  • Approaches: One approach documents models and training datasets, while another explains internal mechanisms and decision-making processes.Chain-of-thought methods and explainable AI are discussed as mechanisms for improving interpretability.
  • Challenges: LLM complexity makes predictions difficult to explain because models use massive datasets, transformer architectures, and hundreds of billions or trillions of parameters.Scaling increases sophistication and expressiveness but also complicates explanation of predictions.
  • Challenges: Effective transparency must adapt information to diverse participants and ensure that they comprehend and interpret it.Relevant participants include developers, regulators, auditors, end-users, and people affected by model applications.
  • Challenges: Transparency strategies must account for inaccurate public awareness and shape information dissemination to reduce misuse and security risks.The paper emphasizes the influence of mass media and language nuances on public perceptions.
  • Approaches: Transparency should be considered across the pre-trained model, adapted versions, and the broader LLM application architecture.The paper presents transparency-related approaches across different application stages.

13 Discussion of Accountability

The paper treats accountability as both a traditional computer-system concern and an LLM-specific governance challenge. It discusses responsibility attribution, bugs, disclaimers, generated-text traceability, copyright, and the costs of liability.

  • Accountability dimensions: LLM accountability has traditional computer-system and modern LLM-specific dimensions.The paper presents accountability as a governance, management, and legal principle for widely deployed systems.
  • Traditional barriers: Extensive collaboration, large datasets, and many contributors create a “problem of many hands” that obscures responsibility.GPT-3 used 570GB of training data, while GPT-4 incorporated user feedback from GPT-3.
  • Traditional barriers: Opaque LLMs may produce stereotypes or hallucinations without error messages, making bugs difficult to isolate and address.Output data can quantify problematic behavior, but black-box opacity complicates diagnosis.
  • Traditional barriers: Authoritative-sounding outputs can lead users to attribute inaccuracies to the model rather than recognizing bugs or operational issues.The paper describes this tendency as treating the computer as a scapegoat.
  • Traditional barriers: Disclaimers about errors do not constitute comprehensive waivers of AI companies’ accountability obligations.The paper gives ChatGPT and Bard disclaimers as examples.
  • Generated-text accountability: Watermarks embed distinctive patterns into generated text, enabling statistical detection and improving traceability.The paper describes pseudorandom vocabulary partitioning and green-token probability increases as one watermarking design.
  • Generated-text accountability: Tracing generated content to its source promotes accountability in journalism, academic writing, and other authenticity-sensitive fields.Watermark research also seeks robustness and unobtrusiveness without compromising quality or naturalness.
  • Generated-text accountability: Watermarking faces privacy concerns from centralized detection and quality costs in precision-sensitive applications such as code generation.These concerns discourage treating watermarking as a default solution.

14 Open Challenges

The paper identifies evaluation boundaries involving language, prompt formulation, instruction following, certification, knowledge change, and transparency and accountability. It also highlights unresolved interactions among trustworthiness dimensions and the fragility of safety guardrails.

  • Evaluation limitations: TRUSTLLM evaluations are conducted only in English, limiting their relevance to trustworthiness in other languages.The paper notes that linguistic nuances, cultural contexts, and idiomatic diversity may be missed.
  • Evaluation limitations: Prompt sensitivity means minor wording or context changes can produce divergent responses and unfairly affect benchmark comparisons.The paper therefore treats explicit prompts as important for consistent evaluation.
  • Evaluation limitations: LLM instruction-following limitations can bias evaluations when models fail fine-grained constraints, complex instructions, or specified output formats.The paper uses simplified prompts and automated evaluation methods to reduce some of these biases.
  • Evaluation limitations: TRUSTLLM does not provide rigorous certification and cannot guarantee worst-case trustworthiness under adversarial or malicious inputs.Existing certification methods face scalability and retraining-cost barriers for current LLM sizes.
  • Open challenges: Reducing hallucinations requires knowledge grounding and editing that account for changing information and real-world events.The paper distinguishes intermediate knowledge, external knowledge, and human feedback as grounding sources.
  • Open challenges: Transparency remains constrained by model complexity, diverse participant needs, and the challenge of adapting information for comprehension.The paper treats these as three major dimensions of the transparency challenge.
  • Open challenges: LLM accountability is complicated by distributed collaboration, opaque bugs, authoritative outputs, and unresolved responsibility boundaries.These issues connect traditional computer-system barriers with LLM-specific accountability concerns.
  • Open challenges: The paper identifies unresolved interactions among trustworthiness dimensions, comprehensive metrics, human oversight, non-language settings, and guardrail persistence.It notes that safety guardrails can be removed by fine-tuning with a handful of examples or benign instruction datasets.

15 Future Work

The paper presents TRUSTLLM as a comprehensive study while outlining future work on evaluation breadth, multimodal and edge systems, cryptographic support, and end-to-end trustworthiness. It emphasizes that cryptography alone cannot address all trustworthiness issues.

  • Research agenda: The paper positions TRUSTLLM as an initial comprehensive study and calls for continued investigation of these directions.Seven future directions are identified for further research.
  • Evaluation expansion: Future TRUSTLLM work will expand prompt templates, datasets, and tasks to reduce prompt-sensitivity errors and broaden evaluation coverage.The planned expansion includes more diverse data sources and additional subtasks.
  • Beyond LLMs: Trustworthiness research should extend beyond LLMs to multimodal models and agents that operate across additional modalities and real-world tasks.The paper identifies multimodal foundation models as general-purpose assistants capable of zero-shot transfer.
  • Beyond LLMs: Multimodal systems introduce modality-gap and alignment challenges alongside inherited language-model trustworthiness concerns.Visual instruction tuning is discussed as a way to align safety, ethics, and moderation considerations.
  • Beyond LLMs: Trustworthiness research should also address specific concerns in IoT edge-intelligence applications and public-safety settings.Such systems may need reliable performance and real-time responses during events such as natural disasters.
  • Cryptographic techniques: Secure computation and zero-knowledge proofs could support security-critical trustworthiness tasks by controlling what information is evaluated or revealed.The paper presents cryptography as a potential tool for several trustworthiness principles.
  • End-to-end trustworthiness: Cryptography alone cannot deliver end-to-end trustworthiness because it cannot establish training-data truthfulness or absence of bias.The paper also identifies efficiency limits and the need to model human factors across the data and LLM pipeline.

16 Conclusion

TRUSTLLM presents a comprehensive study of LLM trustworthiness, combining principles, benchmarks, evaluations, analyses, and discussion of future challenges. Its findings highlight links between trustworthiness and utility, differences between open-weight and proprietary models, excessive trustworthiness, and the need for transparency.

  • TRUSTLLM spans eight trustworthiness dimensions and establishes a benchmark for six of them.
  • Trustworthiness and utility are interconnected in LLMs, while performance varies notably between open-weight and proprietary counterparts.
  • Many LLMs exhibit excessive trustworthiness, which can compromise their utility.
  • The study identifies collaboration among LLM developers as necessary to improve overall model reliability.
  • Transparency about trustworthy-related technologies is central to building human trust and evaluating their effectiveness.
Loading 2401.05561v6…