Source-linked AI summary
One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques
Vijay Arya, Rachel K. E. Bellamy, Pin-Yu Chen, Amit Dhurandhar, Michael Hind, Samuel C. Hoffman, Stephanie Houde, Q. Vera Liao, Ronny Luss, Aleksandra Mojsilović, Sami Mourad, Pablo Pedemonte, Ramya Raghavendra, John Richards, Prasanna Sattigeri, Karthikeyan Shanmugam, Moninder Singh, Kush R. Varshney, Dennis Wei, Yunfeng Zhang
TL;DR
High-stakes AI deployment has created demand for explanations, but stakeholders need different kinds of explanations and the field lacks a precise organizing definition. The paper introduces the AIX360 open-source toolkit, a taxonomy, and an extensible pipeline-based architecture, finding that these resources structure the explainability space, broaden practical access, and expose areas needing further methods.
Problem
Explainability research and societal needs remain misaligned because explanations lack a precise definition and different stakeholders require different forms.
Method
The paper develops the AIX360 open-source toolkit with eight explainability algorithms, two metrics, a stakeholder-oriented taxonomy, an extensible software architecture, and educational resources.
Results
The toolkit and taxonomy organize diverse explainability methods for practitioners, support accessible use through demonstrations and tutorials, and identify taxonomy categories not yet represented in the toolkit.
Takeaways & Limitations
A structured explanation space and shared toolkit can guide practitioners, reveal algorithmic gaps, and support technical, educational, and operational use.
Takeaways & Limitations
The lightweight BRCG variant replaces integer programming with heuristic beam search and therefore lacks a certificate when the search finds no improving solution.
Abstract
from arXiv · showhide
As artificial intelligence and machine learning algorithms make further inroads into society, calls are increasing from multiple stakeholders for these algorithms to explain their outputs. At the same time, these stakeholders, whether they be affected citizens, government regulators, domain experts, or system developers, present different requirements for explanations. Toward addressing these needs, we introduce AI Explainability 360 (http://aix360.mybluemix.net/), an open-source software toolkit featuring eight diverse and state-of-the-art explainability methods and two evaluation metrics. Equally important, we provide a taxonomy to help entities requiring explanations to navigate the space of explanation methods, not only those in the toolkit but also in the broader literature on explainability. For data scientists and other users of the toolkit, we have implemented an extensible software architecture that organizes methods according to their place in the AI modeling pipeline. We also discuss enhancements to bring research innovations closer to consumers of explanations, ranging from simplified, more accessible versions of algorithms, to tutorials and an interactive web demo to introduce AI explainability to different audiences and application domains. Together, our toolkit and taxonomy can help identify gaps where more explainability methods are needed and provide a platform to incorporate them as they are developed.
1 INTRODUCTION
AI explainability is increasingly demanded in high-stakes settings, but different consumers need different kinds of explanations. The paper addresses this gap with a taxonomy, an open-source toolkit, and supporting resources.
- Different consumers require different explanations, including similar cases for doctors, actionable reasons for applicants, global behavior for regulators, and confidence information for developers.
- AIX360 provides a simple, comprehensive taxonomy that considers varied perspectives and helps users choose explainability approaches for particular applications.
- The open-source toolkit implements the taxonomy through an extensible architecture spanning diverse explainability methods and software categories.
- AIX360 includes two explainability metrics, Faithfulness and Monotonicity, to evaluate the quality of explanations.
- Demonstrations, tutorials, notebooks, and guidance material extend the toolkit's accessibility to practitioners and non-experts.
2 TAXONOMY FOR AI EXPLAINABILITY
The taxonomy organizes explainability methods by what is explained, how explanations are produced, and their local or global scope. It helps stakeholders select methods for distinct explanation needs while acknowledging that the taxonomy is not complete.
- Static explanations do not change with consumer feedback, whereas interactive explanations support drilling down or requesting different explanation types.AIX360 focuses on static explanations.
- Local explanations address single predictions, while global explanations describe the behavior of an entire model.
- The taxonomy is a decision tree that structures explainability methods by target, production mode, and explanation level.Its dimensions include data versus model, direct versus post-hoc, static versus interactive, and local versus global explanations.
- The authors present the taxonomy as useful and comprehensive enough for varied users but explicitly do not claim it is perfect or complete.
- Navigating the Taxonomy: The taxonomy maps consumer needs to methods, such as instance-based explanations for loan officers, feature-based counterfactuals for applicants, and surrogate models for executives.The loan example illustrates different branches for validating an individual decision, changing an applicant profile, and understanding overall model behavior.
3 IMPLEMENTATION OF TAXONOMY
AIX360 implements explainers through a unified, extensible Python architecture aligned with stages of the AI modeling pipeline. Its class organization corresponds to, but is not identical with, the paper's taxonomy.
- AIX360 offers a flexible programming interface and Python class hierarchy for explainers of data, models, and predictions.The interface is designed for data scientists and algorithm developers and resembles popular Python model-development tools.
- The software architecture organizes explainers by their use at different stages of the AI modeling pipeline rather than reproducing the taxonomy exactly.The implementation retains clear correspondences with the taxonomy while prioritizing users' modeling workflows.
- Data explainers use DIExplainer for unsupervised dataset explanations, while directly interpretable explainers use DISExplainer to train interpretable models from labeled data.
- Example workflows demonstrate training BRCG rules and obtaining local post-hoc CEM explanations for images.
- The toolkit includes dataset classes for loading and processing commonly used datasets, enabling users to experiment with implemented algorithms.
4 ENHANCEMENTS TO BRING RESEARCH INNOVATIONS TO CONSUMERS
AIX360 enhances explainability methods and supporting materials to make research techniques more accessible to broader consumers. These enhancements span algorithmic simplification, synthetic data, interpretability layers, evaluation metrics, and educational interfaces.
- Enhancement strategy: AIX360 adapts research methods for broader consumers through algorithmic, data, interpretability, metric, visualization, and educational enhancements.The toolkit targets barriers affecting machine learning researchers, expert data scientists, non-experts, and domain-specific users.
- Algorithmic accessibility: BRCG-light replaces CPLEX-based integer programming with heuristic beam search while retaining a DNF rule classifier.The modification reduces access barriers but changes the optimization procedure.
- Algorithmic accessibility: BRCG-light lies between BRCG-IP and earlier rule-learning methods in accuracy–complexity trade-offs, with beam width 5 outperforming smaller widths on most datasets.Larger beam widths may improve results but increase computational complexity.
- Data synthesis: AIX360 synthesizes explanation-labeled training data for TED by generating random feature vectors and applying domain rules to assign predictions and explanations.Its employee-retention dataset contains 10,000 entries, and the generation script supports changing feature distributions, rules, and dataset size.
- Scope boundary: GLRM models with many rules remain interpretable but may require a non-trivial investment of effort to understand.This limits how readily rule-based explanations can be consumed as model size grows.
- Interpretability layers: The toolkit adds feature importances to ProtoDash and CEM outputs, helping users identify which factors are most critical for maintaining or changing a classification.Higher θj values indicate greater feature importance for the corresponding prototype or contrastive explanation.
- Evaluation and education: AIX360 includes Faithfulness and other quantitative evaluation support, alongside tutorials and a web demo designed to make explainability usable across audiences and application domains.The authors describe the included metrics as quantitative measures of explanation goodness, although the metrics themselves are not novel.
5 RELATED WORK
Prior taxonomies classify explainability methods but do not consistently guide practitioners across diverse explanation needs. AIX360 addresses this gap with a broader taxonomy and toolkit spanning methods, data explanations, metrics, persona-specific explanations, and educational resources.
- Existing surveys classify explainability methods by criteria such as model interpretability, but these classifications do not necessarily guide method selection.
- Some taxonomies offer more actionable mappings of methods to model, outcome, and inspection explanations, but focus primarily on post-hoc black-box models.
- The paper motivates mapping methods to explanation problems because diverse consumers require different kinds of explanations.
- AIX360 differs from other toolkits by offering a wider spectrum that includes data explanations, metrics, and persona-specific explanations.
- AIX360 also provides educational materials for less technical and industry-specific users, including a web demo, guidance, glossary, and tutorials.
6 DISCUSSION
The discussion presents AIX360 as an open-source toolkit combining diverse explainability algorithms, metrics, tutorials, and an extensible interface with a taxonomy for organizing the field. It also identifies missing explanation categories and framework interoperability as directions for extension.
- AIX360 contains eight state-of-the-art explainability algorithms that explain AI models or complex datasets in different ways for diverse users.
- The toolkit supplements algorithms with two metrics, a credit approval demonstration, five tutorials, and 13 Jupyter notebooks for practitioners and non-experts.
- Its taxonomy guides practitioners, reveals algorithmic gaps, and informs explainability software design for data scientists and developers.
- The toolkit lacks algorithms in three taxonomy categories—interactive, static data distributions, and static model global post-hoc visualization—and has limited coverage for some modalities such as text.
- Future work should implement framework-specific model classes exposing a common API so algorithms can explain models built with different deep learning frameworks.
A ACCURACY-COMPLEXITY TRADE-OFFS FOR BRCG-LIGHT
BRCG-light is evaluated against alternative rule-learning methods through classification-accuracy versus DNF-rule-complexity trade-offs. Its performance is generally below BRCG-IP but above the other methods, with wider beam search usually helping.
- Accuracy-complexity comparisons: BRCG-light offers accuracy-complexity trade-offs worse on average than BRCG-IP but better than BRS, AM, and BCD.The comparison covers 15 datasets and measures complexity by clauses plus conditions across all clauses.
- Beam-search settings: B = 5 generally produces better results than B = 1 or 3, especially on larger datasets.The evaluated beam-search widths are B = 1, 3, and 5.
- Figure interpretation: The figures connect Pareto-efficient points to show rule-complexity and test-accuracy trade-offs.Horizontal and vertical bars represent standard errors in the means.
(a) three types of explanation consumers in the FICO use case
The FICO web demo illustrates that different explanation consumers require different explainability methods. The scenario involves a loan applicant, a loan officer, and a data scientist.
- Consumer groups: The demo uses a loan-repayment-risk prediction scenario to distinguish explanation needs across three consumer groups.The groups are data scientists, loan officers, and bank customers.
- Method selection: Different explainability methods are matched to the differing needs of data scientists, loan officers, and bank customers.The loan officer can choose either customer to see an explanation.