Source-linked AI summary
Trustworthy AI
Richa Singh, Mayank Vatsa, Nalini Ratha
TL;DR
Modern AI systems are increasingly deployed despite weaknesses in robustness, explainability, fairness, privacy, security, and transparency. This tutorial brings these concerns together with decent AI and attribution, surveying their connections and open challenges. It frames trustworthy AI as requiring systems that are fair, explainable, robust, secure, transparent, attributable, and decent.
Problem
Increasingly deployed AI systems have limitations in robustness, explainability, fairness, privacy, security, transparency, lineage disclosure, and decent behavior.
Method
The paper proposes a tutorial that unifies trustworthy-AI challenges and surveys concepts, research contributions, interlinkages, and open challenges.
Results
The tutorial identifies six trust-enhancing issues: bias and fairness, explainability, adversarial robustness, privacy and security, decency, and model attribution with lineage transparency.
Takeaways & Limitations
Trustworthy AI is presented as requiring complementary attention to technical reliability, social fairness, transparency, attribution, security, and decent interaction behavior.
Abstract
from arXiv · showhide
Modern AI systems are reaping the advantage of novel learning methods. With their increasing usage, we are realizing the limitations and shortfalls of these systems. Brittleness to minor adversarial changes in the input data, ability to explain the decisions, address the bias in their training data, high opacity in terms of revealing the lineage of the system, how they were trained and tested, and under which parameters and conditions they can reliably guarantee a certain level of performance, are some of the most prominent limitations. Ensuring the privacy and security of the data, assigning appropriate credits to data sources, and delivering decent outputs are also required features of an AI system. We propose the tutorial on Trustworthy AI to address six critical issues in enhancing user and public trust in AI systems, namely: (i) bias and fairness, (ii) explainability, (iii) robust mitigation of adversarial attacks, (iv) improved privacy and security in model building, (v) being decent, and (vi) model attribution, including the right level of credit assignment to the data sources, model architectures, and transparency in lineage.
1 INTRODUCTION
The paper frames trustworthy AI as necessary because AI increasingly influences consequential decisions while remaining vulnerable to adversarial attacks, bias, opacity, and indecent behavior. It proposes a tutorial that brings these challenges together and examines their interlinkages.
- AI is increasingly used for both routine recommendations and consequential decisions such as disease diagnosis, fraud detection, hiring, autonomous driving, loans, and cancer treatment.
- Modern AI systems exhibit brittleness to adversarial changes, biased performance across groups, limited explainability, and opaque model lineage and testing conditions.
- Trustworthy AI also requires privacy and security, appropriate attribution of data and models, and decent responses in human–AI interactions.
- The tutorial unifies bias and fairness, adversarial robustness, explainability, blockchain-based security, privacy preservation, attribution and transparency, and decent AI.
2 BUILDING TRUSTED/TRUSTWORTHY AI SYSTEMS
Trustworthy AI spans technical, social, ethical, and governance concerns, including robustness, privacy, transparency, fairness, and accountability. The paper surveys research directions addressing these components and related reliability and verifiability.
- Trustworthy AI encompasses human oversight, robustness and safety, privacy and data governance, transparency, fairness, societal well-being, and accountability.
- Research on bias and fairness studies bias estimation, mitigation, and accounting across AI systems.
- Adversarial-robustness research develops attack models, detects adversarial perturbations, and mitigates attacks.
- The paper connects trustworthy AI with robust, ethical, fair, and safe AI concepts while emphasizing reliability and verifiability.
3 DECENT AI
The paper introduces decent AI as a complementary dimension of trustworthy AI focused on machines behaving like decent humans. It proposes the Decent AI Turing Test to evaluate this behavior.
- The Decent AI Turing Test asks a human evaluator to judge whether an AI response is correct and decent without knowing whether a human or AI produced it.
- An AI passes when the evaluator cannot reliably distinguish its response from that of a decent human agent.
- Decent AI is conceptually distinct from, and complementary to, trustworthy AI properties such as fairness, dependability, and trust.
4 RESEARCH QUESTIONS
The paper identifies open questions about evaluating trustworthiness and decency, integrating them with performance, understanding their relationships, and democratizing such systems.
- Researchers need methods to automatically measure or evaluate an AI system’s trustworthiness and decency.
- Open questions include how to integrate trustworthiness and decency with system performance and accuracy.
- Further questions concern relationships among decency, trustworthy and dependable AI factors, and democratization of trustworthy and decent systems.
5 BRIEF BIOGRAPHIES
The authors include Mayank Vatsa, Richa Singh, and Nalini K. Ratha, whose biographies highlight academic leadership, awards, and contributions to biometrics.
- Richa Singh is a Professor at IIT Jodhpur and serves as Vice President (Publications) of the IEEE Biometrics Council.
- Mayank Vatsa is a Professor at IIT Jodhpur and has held leadership and editorial roles in computer vision, information fusion, and pattern recognition.
- Nalini K. Ratha is an Empire Innovation Professor at the University at Buffalo and has authored more than 100 biometrics research papers.