Source-linked AI summary
Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyond
Anna Hedström, Leander Weber, Dilyara Bareeva, Daniel Krakowczyk, Franz Motzkus, Wojciech Samek, Sebastian Lapuschkin, Marina M. -C. Höhne
TL;DR
XAI explanation evaluation lacks ground truth, shared correctness criteria, and standardized procedures, making results difficult to interpret and compare. Quantus addresses this gap with a comprehensive, customizable Python toolkit that organizes metrics, guidance, tutorials, and automated evaluation. The toolkit is intended to improve reproducibility, transparency, and comparability in XAI evaluation.
Problem
XAI evaluation lacks ground-truth data, a universally accepted definition of correct explanations, and standardized procedures.
Method
Quantus is a comprehensive, customizable Python toolkit that organizes evaluation metrics, guidance, tutorials, and automated procedures for explanation methods.
Results
Quantus provides 30+ reference metrics, documentation, tutorials, and evaluation capabilities spanning multiple categories and use cases.
Takeaways & Limitations
Quantus aims to support more reproducible, transparent, simple, holistic, and comparable evaluation of XAI explanations.
Abstract
from arXiv · showhide
The evaluation of explanation methods is a research topic that has not yet been explored deeply, however, since explainability is supposed to strengthen trust in artificial intelligence, it is necessary to systematically review and compare explanation methods in order to confirm their correctness. Until now, no tool with focus on XAI evaluation exists that exhaustively and speedily allows researchers to evaluate the performance of explanations of neural network predictions. To increase transparency and reproducibility in the field, we therefore built Quantus -- a comprehensive, evaluation toolkit in Python that includes a growing, well-organised collection of evaluation metrics and tutorials for evaluating explainable methods. The toolkit has been thoroughly tested and is available under an open-source license on PyPi (or on https://github.com/understandable-machine-intelligence-lab/Quantus/).
1. Introduction
XAI explanation evaluation remains unresolved because there is no universally accepted definition of a correct explanation or standardized procedure. Different evaluation choices can produce difficult-to-interpret or contrasting results, motivating a comprehensive evaluation toolkit.
- XAI evaluation remains an unsolved problem.
- The absence of ground-truth data and agreement on correct explanations makes evaluation difficult.
- Different parameterisations, preprocessing, and normalisations can produce contrasting evaluation outcomes.
- One-sided or methodologically questionable evaluations may hinder access to XAI state-of-the-art and hurt the field’s perceived credibility.
- Quantus was developed to collect, organise, explain, and automate quantitative evaluation of explanation methods.
2. Toolkit Overview
Quantus is an evaluation-focused XAI library offering a broad and expanding collection of metrics, guidance, tutorials, and comparative analyses. Its initial focus is attribution-based explanations, with planned extensions beyond image classification and toward additional checks and sensitivity analyses.
- Quantus provides 30+ reference metrics for evaluating explanations of machine-learning predictions.
- The toolkit includes guidance on metric use and potential application pitfalls.
- Its documentation and tutorials cover comparative XAI analyses, multiple domains and tasks, and dependence on metric parameterisations.
- Quantus focuses primarily on evaluation and spans multiple evaluation categories.
- Initial releases focus on attribution-based explanation techniques, which assign importance values to model features.
- Planned releases aim to extend applicability beyond image classification with additional metrics, checks, verifications, and sensitivity analyses.
3. Library Design
Quantus provides a structured, fast, and customizable interface for evaluating precomputed explanations while accounting for task-specific evaluation context and metric sensitivity.
- A single line of code can produce quantitative insights into explanation behaviour under varied criteria.
- The toolkit demonstrates evaluation of precomputed attributions through a PixelFlipping experiment.
- Quantus supports custom evaluation components, including user-specified perturbation baselines for masking input features.
- Because no single metric fits every task, explanation evaluation must be calibrated to the application, data, model, and stakeholders.
- Warnings, checks, and guidelines encourage users to reflect on metric parameterisation and confounding factors.
4. Broader Impact
Quantus aims to replace ad hoc XAI evaluation with procedures emphasizing reproducibility, simplicity, transparency, and multidimensional assessment. The authors present these properties as supporting faster development, reduced ambiguity, and greater comparability.
- Quantus is intended to replace ad hoc and sometimes ineffective evaluation with reproducibility, simplicity, and transparency.
- The toolkit is presented as helping researchers speed development and application of explanation methods, dissolve ambiguities, and enable comparability.
- The authors argue that objective, reproducible, holistic, and multidimensional XAI quantification benefits the broader community.
Appendix
Because ground-truth explanations are usually unavailable, Quantus organises explanation-evaluation metrics into six logically similar categories. The toolkit documents category meanings and cautions users to inspect individual metric specifications before interpretation.
- Quantus groups explanation-evaluation metrics into six categories: faithfulness, robustness, localisation, complexity, randomisation, and axiomatic properties.The categories are organised by logical similarity in the source code.
- Faithfulness (↑) assesses whether important features more strongly affect model decisions, while robustness (↓) assesses explanation stability under slight input perturbations with nearly unchanged outputs.
- Localisation (↑) tests whether explanatory evidence centres on a defined region of interest, such as a bounding box, segmentation mask, or grid cell.
- Complexity (↓) measures explanation concision by assessing how few features are used to explain a model prediction.
- Randomisation (↑) tests whether explanations deteriorate as data labels or model parameters become increasingly randomised.
- Axiomatic (↑) measures whether explanations satisfy specified axiomatic properties, while category arrows generally indicate whether higher or lower values are better.Exceptions exist within categories, so individual metric docstrings should be consulted before usage or interpretation.