PaperlayerView exact revision on arXiv

Source-linked AI summary

Trustworthy AI: A Computational Perspective

Haochen Liu, Yiqi Wang, Wenqi Fan, Xiaorui Liu, Yaxin Li, Shaili Jain, Yunhao Liu, Anil K. Jain, Jiliang Tang

arXiv:2107.06641v3cs.AI

TL;DR

AI’s broad benefits coexist with risks including unreliable safety-critical decisions, discrimination, and privacy harms. This survey synthesizes computational approaches to trustworthy AI across six dimensions, their interactions, and future research needs, while noting that important additional dimensions remain underdeveloped.

  • Problem

    AI applications can produce unreliable decisions, discriminatory outcomes, and other harms, motivating a computational understanding of trustworthy AI.

  • Method

    The survey defines trustworthy AI, organizes computational technologies across six dimensions, reviews representative algorithms and applications, and examines cross-dimensional interactions.

  • Results

    The survey provides a comprehensive computational overview of trustworthy AI technologies and discusses accordance, conflicts, and potential research directions across dimensions.

  • Takeaways & Limitations

    Trustworthy AI must be considered across multiple dimensions because these dimensions can both reinforce and conflict with one another.

  • Takeaways & Limitations

    Additional dimensions, including human agency and oversight and credibility, remain important but have limited computational literature and require dedicated research.

Abstract

from arXiv · show

In the past few decades, artificial intelligence (AI) technology has experienced swift developments, changing everyone's daily life and profoundly altering the course of human society. The intention of developing AI is to benefit humans, by reducing human labor, bringing everyday convenience to human lives, and promoting social good. However, recent research and AI applications show that AI can cause unintentional harm to humans, such as making unreliable decisions in safety-critical scenarios or undermining fairness by inadvertently discriminating against one group. Thus, trustworthy AI has attracted immense attention recently, which requires careful consideration to avoid the adverse effects that AI may bring to humans, so that humans can fully trust and live in harmony with AI technologies. Recent years have witnessed a tremendous amount of research on trustworthy AI. In this survey, we present a comprehensive survey of trustworthy AI from a computational perspective, to help readers understand the latest technologies for achieving trustworthy AI. Trustworthy AI is a large and complex area, involving various dimensions. In this work, we focus on six of the most crucial dimensions in achieving trustworthy AI: (i) Safety & Robustness, (ii) Non-discrimination & Fairness, (iii) Explainability, (iv) Privacy, (v) Accountability & Auditability, and (vi) Environmental Well-Being. For each dimension, we review the recent related technologies according to a taxonomy and summarize their applications in real-world systems. We also discuss the accordant and conflicting interactions among different dimensions and discuss potential aspects for trustworthy AI to investigate in the future.

1 INTRODUCTION

Trustworthy AI is presented as a computational response to AI’s societal benefits and potential harms. The survey organizes relevant technologies around six dimensions and examines their interactions and future directions.

  • The survey reviews computational solutions for trustworthy AI, covering concepts, taxonomies, representative algorithms, applications, interactions, and underexplored issues.
  • The six focal dimensions are Safety & Robustness, Non-discrimination & Fairness, Explainability, Privacy, Auditability & Accountability, and Environmental Well-Being.
  • Safety & Robustness addresses unreliable decisions caused by input perturbations, including attacks that can fool autonomous vehicles and spam detectors.
  • Non-discrimination & Fairness addresses biased decisions, including unequal performance in face recognition and voice dictation.
  • Explainability is needed when opaque decisions affect critical applications such as disease diagnosis, while Privacy concerns models exposing sensitive personal information.
  • The dimensions can reinforce or conflict with one another, and additional dimensions remain less developed computationally and are reserved for future research.

2 CONCEPTS AND DEFINITIONS

The survey defines trustworthy AI as AI that benefits people without threatening or harming them, and characterizes it through technical, user, and social perspectives. It distinguishes this concept from related AI terminologies while emphasizing overlapping principles.

  • The survey uses related terminology and a principles table to situate trustworthy AI across disciplines and perspectives.
  • Trustworthy AI is defined as programs and systems that solve problems like humans, benefit people, and present no threat or risk of harm.
  • The definition is developed from technical, user, and social perspectives, whose properties are described as complementary rather than independent.
  • From the technical perspective, trustworthy AI should provide accurate, robust, explainable, nondiscriminatory, fair, accountable, and environmentally friendly behavior.
  • Trustworthy AI overlaps with ethical, beneficial, responsible, fair, and explainable AI, but its primary focus is the AI technology itself.

3 SAFETY & ROBUSTNESS

Safety and robustness require AI systems to maintain accurate, secure decisions despite noisy or intentionally designed input perturbations. The survey organizes threats, victim models, and defensive strategies for this dimension.

  • Robust trustworthy AI should maintain stable, sustained accuracy under different circumstances and resist small perturbations in real-world data.
  • Adversarial attacks use fake training or test examples to degrade machine-learning performance and create security concerns across applications.
  • Threat Models: Threat models are classified by whether attackers modify training or test samples, what information they possess, and whether they seek targeted predictions.
  • Victim Models: Studied victim models range from traditional classifiers such as Naive Bayes and SVMs to CNNs, GNNs, and RNNs.
  • Defense Strategies: Defenses include adversarial training, certified defense, and inference-time detection of adversarial examples.

3.3 Representative Attack Methods

The survey reviews representative evasion and poisoning attacks, including perturbation-constrained test-time attacks and training-time manipulations. It also notes that real-world perturbations may require measures beyond L_p norms.

  • Evasion Attacks: Evasion attacks occur at test time and constrain adversarial perturbations by pixel-based L_p bounds or other distance measures.
  • Evasion Attacks: Projected Gradient Descent seeks an adversarial example that maximizes loss while keeping perturbations within an L_p norm bound.
  • Evasion Attacks: AutoAttack combines four attacks, including white-box and black-box methods, to provide more reliable adversarial-robustness evaluation.
  • Beyond L_p Bounds: L_p norm perturbation is not sufficient for all real-world noise or perceptual similarity, motivating alternatives such as Wasserstein distance.
  • Poisoning Attacks: Poisoning attacks modify training data, whereas backdoor attacks insert triggers during training that produce target predictions when repeated at test time.

3.4 Representative Defense Methods

The survey reviews adversarial defense methods spanning training, certification, detection, and applications, showing that small input or data modifications can compromise systems across domains.

  • Training-based defenses: Adversarial training minimizes worst-case loss around training samples to improve resistance, but provides only partial robustness and incurs longer training.It can also face natural-versus-robust trade-offs and robust overfitting.
  • Certified defenses: Certified defense seeks provable robustness against all perturbations within a specified norm bound, unlike empirical defenses that achieve robustness only to some extent.Randomized smoothing is one approach for constructing a certified smooth classifier.
  • Detection defenses: Detection defenses classify inputs as adversarial or natural, then can reject harmful examples or build models using recognized natural samples.Detection can also exploit statistical differences between adversarial and natural sample distributions.
  • Real-world applications: Small modifications can cause autonomous vehicles to misread 35 mph as 85 mph or fail to detect stop signs, illustrating risks in safety-critical deployment.The survey also describes adversarial glasses and T-shirts that mislead face-recognition and person-detection systems.
  • Cross-domain threats: Adversarial examples affect language, dialogue, audio, and graph systems through paraphrases, trigger inputs, perturbations, fingerprint manipulation, or graph changes.Dialogue attacks may target semantically identical responses, while graph attacks modify node connections or features.

3.6 Surveys and Tools

The survey points readers to adversarial-robustness surveys, domain-specific reviews, software libraries, and benchmarks, while identifying unresolved evaluation, generalization, and efficiency challenges.

  • Surveys: Existing surveys cover adversarial attacks and defenses across image, graph, and natural-language domains, with additional reviews focused on individual domains.These resources are intended to provide broad or domain-specific introductions to adversarial robustness techniques.
  • Tools: Advertorch and DeepRobust provide PyTorch-based access to attack, defense, and evaluation methods for image models, graph models, or graph properties.DeepRobust includes both image- and graph-domain algorithms and evaluation functions.
  • Tools: RobustBench is presented as a resource for evaluating adversarial robustness, alongside libraries that support practical experimentation.The supplied passage identifies RobustBench but does not provide further details about its evaluation protocol.
  • Open challenges: Future work seeks fewer-query black-box attacks, better generalization beyond robust overfitting, broader threat coverage, and more efficient certified defenses.Certified methods currently require substantial computational resources.

4 NON-DISCRIMINATION & FAIRNESS

The fairness section surveys how bias arises, how fairness is defined and measured, and how mitigation methods operate across the AI pipeline and application domains. It also highlights unresolved trade-offs and conceptual challenges.

  • Motivation: AI systems can reproduce discriminatory patterns, including poorer face recognition for African Americans, male-favoring voice recognition, and biased conversational behavior.The survey frames non-discrimination and fairness as requirements for avoiding unfair treatment of individuals or groups.
  • Sources of bias: Bias may originate in data generation, algorithms, or evaluation, with annotation and collection practices providing multiple pathways for data-related bias.The passage lists non-representative, inexperienced, or stereotyped annotators as examples of annotation sources.
  • Types of bias: Explicit bias directly uses a sensitive attribute, whereas implicit bias arises through seemingly neutral attributes associated with sensitive characteristics.Residential address and language style are examples of attributes that can encode demographic information indirectly.
  • Types of bias: Acceptable bias is explainable by relevant factors, while unacceptable bias lacks an appropriate explanation and should be avoided in practice.The survey illustrates acceptable bias with salary predictions linked to differences in weekly working time.
  • Fairness definitions: Group fairness compares statistical treatment or outcomes across groups, while individual fairness requires similar predictions for similar individuals.The individual-fairness formulation constrains prediction differences to remain below a small constant for individuals close under a similarity metric.
  • Mitigation methods: Debiasing methods intervene before training, during model optimization, or after prediction through pre-processing, in-processing, and post-processing strategies.Post-processing can select group-specific thresholds using measures such as equalized odds, while in-processing often balances performance and fairness.
  • Applications: The survey organizes real-world fairness studies by tabular data, images, texts, audios, and graphs, covering representative tasks and detection or mitigation work.Tabular examples include credit and recidivism prediction, while the overview directs readers to representative summaries in Table 3.
  • Future directions: Fairness research faces a trade-off between fairness and performance and lacks sufficiently precise conceptualizations for every task and setting.The survey notes that improving fairness typically comes with performance degradation and that different tasks can exhibit different forms of bias.

5 EXPLAINABILITY

Explainability concerns making AI decisions understandable to humans, especially as increased model complexity has produced opaque deep-learning systems. The survey introduces explainability concepts, taxonomies, and distinctions among explanation techniques.

  • Increased model complexity has improved predictive performance but made many deep-learning systems difficult for humans to understand.These black-box models are difficult to fully trust in critical applications such as healthcare and autonomous driving.
  • Explainability is commonly understood as presenting an AI decision or its cause in terms understandable to humans.
  • Explainable AI and interpretable AI are closely related terms, although some studies discuss subtle differences between them.
  • An interpretable model allows humans to understand its prediction procedure from the model itself, including how changes may affect outcomes.
  • Explanation techniques can be classified by model usage, explanation scope, and algorithmic approach.The survey distinguishes model-intrinsic from model-agnostic methods and local from global explanations.

• Differences in the methodology:

The survey organizes explanation methods by how they use gradients or input modifications, by model dependence, and by explanation scope. It reviews representative intrinsic, model-agnostic, local, global, gradient-based, perturbation-based, and counterfactual approaches.

  • Explanation via Other Approaches: Counterfactual Explanations.: Counterfactual explanations describe how changing an instance’s feature values would alter its prediction toward a predefined output.Their objective balances prediction distance against distance between the original and counterfactual inputs.
  • 5.2.1 Model usage.: Model-intrinsic explanations are tied to a specific architecture, whereas model-agnostic methods can be applied across algorithms.Linear or logistic regression and decision trees are representative intrinsic models; LIME is a representative model-agnostic method.
  • 5.2.1 Model usage.: Linear regression explains predictions through learned feature weights and bias, while decision trees explain labels through paths from root nodes to leaves.
  • 5.2.1 Model usage.: LIME identifies important local image patches for predicted classes, while GNNExplainer learns masks over graph edges and node features.The cited examples include image classification and graph classification on molecule graphs.
  • Scope of Explanation:: Local explanations attribute features for one input, whereas global explanations summarize model behavior across inputs or labels.Saliency maps use gradients to identify influential input pixels for an individual prediction.
  • Gradient-based Explanations: CAM and Grad-CAM produce image-region explanations, with Grad-CAM locating regions important for image caption generation.
  • Differences in the methodology:: Gradient-based methods use partial derivatives during back-propagation to generate attributions, while perturbation-based methods focus on modifying input data.
  • The survey identifies real-world applications and presents a summary of published research in explainability.

6 PRIVACY

This section surveys privacy risks in AI systems and organizes privacy-preserving approaches around confidential computing, federated learning, and differential privacy.

  • Motivation: AI data and models may expose sensitive information, motivating privacy protection throughout the machine-learning system.Examples include credit-card and medical data, as well as private information memorized by dialogue models.
  • Privacy Attacks: Representative privacy attacks target training membership, input information, dataset properties, and confidential model functionality.Membership inference identifies training records; model inversion infers sensitive inputs; property inference extracts population attributes; model extraction reconstructs or approximates a target model.
  • Privacy-Preserving Techniques: Privacy-preserving countermeasures comprise confidential computing, federated learning, and differential privacy.These directions address data safety during computation, decentralized learning without raw-data transmission, and privacy protection through controlled information release.
  • Confidential Computing: Confidential computing uses trusted execution environments, homomorphic encryption, and secure multi-party computation to protect data during computation.TEE isolates data and programs from privileged processes; MPC enables joint computation without disclosing participants’ private inputs; HE computes on ciphertexts.
  • Federated Learning: Federated learning coordinates collaborative training between clients and a server while keeping client data decentralized and local.Models move between server and clients instead of transmitting raw private data, reducing the risk of privacy breaches.

7 ACCOUNTABILITY & AUDITABILITY

This section frames accountability as responsibility for AI impacts and auditability as a methodology for assessing systems, while distinguishing external and internal audits.

  • Accountability: AI accountability concerns both trust in system performance and responsibility for failures or impacts.Determining responsibility is difficult because AI systems can be black boxes and real-world systems involve many contributors.
  • Auditability: Auditability consists of principled assessments of an AI system’s conformance to applicable requirements.The survey presents auditability as one of the main methodologies for ensuring accountability.
  • External Audits: External audits are conducted by independent third parties and can provide comprehensive, objective perspectives.They may lack access to internal training data and implementation details, limiting audit difficulty and coverage.
  • External Audits: External audits occur after deployment, which can make system adjustments costly and may follow harm that has already occurred.This timing creates a practical boundary for preventive intervention.
  • Internal Audits: Internal audits are conducted within designer or deployer organizations and can access substantial internal system data.SMACTR organizes internal auditing into scoping, mapping, artifact collection, testing, and reflection.
  • Future Directions: The survey recommends considering both external and internal audits for a more comprehensive and objective overview of AI systems.Explainability is also identified as important for understanding mechanisms and distributing responsibility.

8 ENVIRONMENTAL WELL-BEING

This section surveys environmental impacts of AI and techniques for reducing energy use through algorithmic, architectural, hardware, and measurement approaches.

  • Motivation: AI’s environmental burden is primarily associated with energy consumption from large-scale development and deployment.The survey reports that training a common NLP pipeline can match seven years of human carbon emissions, while training and fine-tuning a large Transformer can use five times a car’s energy consumption.
  • Energy-Efficient AI: Energy-efficiency research includes model compression, adaptive design, and hardware-level solutions.These approaches target algorithm design, model architecture, and specialized computing platforms.
  • Algorithm-Level Approaches: Model compression reduces model size, storage, and training or deployment energy with an acceptable performance sacrifice.Methods include pruning, quantization, low-rank factorization, compact filters, and knowledge distillation.
  • Algorithm-Level Approaches: Adaptive design optimizes model architectures for energy efficiency by pruning or designing networks under energy-consumption restrictions.One approach prunes the most energy-consuming layer first, while another designs CNNs subject to energy constraints.
  • Hardware-Level Approaches: Hardware research improves efficiency through DNN accelerators and devices specialized for neural-network computations.Examples include NPUs, RENO, sparse-network accelerators, speech-recognition engines, and GAN accelerators.
  • Evaluation and Tools: Energy estimation tools measure or predict AI energy use and carbon emissions before or during deployment.Examples include NeuralPower, SyNERGY, the Machine Learning Emissions Calculator, Accelergy, and Timeloop.
  • Future Directions: Future work should prioritize energy-aware AutoML and customized hardware for model training, which current research comparatively overlooks.Existing AutoML emphasizes performance, while DNN accelerator research focuses more on inference than training.

9 INTERACTIONS AMONG DIFFERENT DIMENSIONS

The six trustworthy AI dimensions can reinforce or conflict with one another, so satisfying them simultaneously may be impossible in some scenarios. The survey illustrates these interactions through examples involving robustness, explainability, fairness, privacy, and environmental well-being.

  • Interactions Among Different Dimensions: The six dimensions are interdependent: satisfying one can promote another, but realizing one may also violate another.Some scenarios therefore make simultaneous satisfaction of two or more dimensions impossible.
  • Accordance: Robustness and explainability can reinforce each other through robust training and explainable gradients.Lipschitz regularization aligns gradients with inputs, while Interpretation Regularization improves robustness to adversarial attacks.
  • Accordance: Reducing the cost of training large AI models can support both environmental well-being and fairness.Large models consume more computing resources, harming the environment and aggravating inequality among researchers.
  • Conflicting Interactions: Robust training can increase privacy risks by making models more likely to expose sensitive training information through membership inference attacks.This illustrates a conflict between robustness and privacy requirements.
  • Conflicting Interactions: Adversarial training can create disparities in performance and robustness across groups, even with balanced datasets.The training algorithm may improve robustness for one group while reducing it for another.
  • Conflicting Interactions: Differential privacy and exact equal opportunity are theoretically unlikely to be achieved simultaneously, although approximate fairness can coexist with differential privacy.Relaxing the fairness condition makes joint satisfaction possible.

10 FUTURE DIRECTIONS

Future trustworthy AI research should extend beyond the six surveyed dimensions to address human autonomy and the integrity of AI-produced content. The survey also identifies interactions among dimensions as an early-stage area requiring further investigation.

  • Future Directions: Potential trustworthy AI dimensions beyond the six surveyed dimensions remain worth exploring.The survey notes that some additional dimensions have not yet received extensive attention.
  • Human Autonomy: Human autonomy requires human-centered AI systems that preserve self-determination and guarantee human agency and oversight.AI agents should not subordinate, coerce, or manipulate humans.
  • Content Integrity: AI-produced or AI-screened content requires mechanisms that ensure its credibility because deployed systems can expose people to misinformation.Examples include answers from question-answering agents and news delivered by recommender systems.
  • Interactions Among Dimensions: Research on interactions among trustworthy AI dimensions remains at an early stage, with additional dimension pairs still requiring investigation.The survey specifically identifies fairness–interpretability relationships and other interactions as open topics.

11 CONCLUSION

The survey presents a computational overview of trustworthy AI, defining the field, organizing six crucial dimensions, and reviewing technologies, applications, tools, surveys, and future directions. It also analyzes how these dimensions interact and acknowledges additional issues beyond its six-dimensional focus.

  • Conclusion: The survey clarifies trustworthy AI from multiple perspectives and distinguishes it from related concepts.It addresses perspectives including computer science, sociology, law, and business.
  • Conclusion: It organizes six crucial dimensions—Safety & Robustness, Nondiscrimination & Fairness, Explainability, Accountability & Auditability, Privacy, and Environmental Well-being—using concepts and taxonomies.The organization is intended to help readers understand how each dimension is studied.
  • Conclusion: For each dimension, the survey summarizes representative technologies, real-world applications, related surveys, tools, and future research directions.These materials support readers in following research progress across the dimensions.
  • Conclusion: The survey analyzes accordance and conflicting interactions among dimensions while recognizing potential trustworthy AI issues outside its six-dimensional coverage.Its scope therefore includes both cross-dimensional relationships and directions beyond the dimensions elaborated in the survey.
Loading 2107.06641v3…