Source-linked AI summary
Measuring the Quality of Explanations: The System Causability Scale (SCS). Comparing Human and Machine Explanations
Andreas Holzinger, André Carrington, Heimo Müller
TL;DR
Opaque AI and ML systems, particularly in medicine, create a need for explanations that domain experts can understand and evaluate. The paper introduces the System Causability Scale by combining a causability concept with usability-scale ideas, and illustrates it with the Framingham Risk Tool. Its main scope boundary is that ground truth and causal mechanisms may not always be well defined, while current models often rely on correlations or similarity.
Problem
Opaque AI and ML systems make it difficult for developers and human experts, especially in medicine, to understand decision processes and evaluate explanation quality.
Method
The paper introduces the System Causability Scale, combining causability with concepts adapted from a widely accepted usability scale and applying it to an explanation interface and process.
Results
The paper presents SCS as a simple, rapid tool for evaluating explanation quality and illustrates its use with the Framingham Risk Tool.
Takeaways & Limitations
SCS directly measures users’ perceived utility of explanations, including causal aspects, rather than relying only on qualitative ratings or proxy measures.
Takeaways & Limitations
Ground truth is not always well defined for causal mechanisms, while successful machine models typically rely on correlation, similarity, or distance.
Abstract
from arXiv · showhide
Recent success in Artificial Intelligence (AI) and Machine Learning (ML) allow problem solving automatically without any human intervention. Autonomous approaches can be very convenient. However, in certain domains, e.g., in the medical domain, it is necessary to enable a domain expert to understand, why an algorithm came up with a certain result. Consequently, the field of Explainable AI (xAI) rapidly gained interest worldwide in various domains, particularly in medicine. Explainable AI studies transparency and traceability of opaque AI/ML and there are already a huge variety of methods. For example with layer-wise relevance propagation relevant parts of inputs to, and representations in, a neural network which caused a result, can be highlighted. This is a first important step to ensure that end users, e.g., medical professionals, assume responsibility for decision making with AI/ML and of interest to professionals and regulators. Interactive ML adds the component of human expertise to AI/ML processes by enabling them to re-enact and retrace AI/ML results, e.g. let them check it for plausibility. This requires new human-AI interfaces for explainable AI. In order to build effective and efficient interactive human-AI interfaces we have to deal with the question of how to evaluate the quality of explanations given by an explainable AI system. In this paper we introduce our System Causability Scale (SCS) to measure the quality of explanations. It is based on our notion of Causability (Holzinger et al., 2019) combined with concepts adapted from a widely accepted usability scale.
1 Introduction
AI and ML can solve tasks autonomously, but opaque systems—especially in medicine—make transparency, responsibility, and human understanding difficult. The paper motivates evaluating explanation quality and introduces a direct user-perception approach for doing so.
- Motivation: AI and ML increasingly automate prediction and problem solving, including speech recognition, game playing, and medical image analysis.Deep learning has achieved strong results in several applications, including medical image classification, nodule detection, and segmentation.
- Motivation: Opaque, resource-intensive systems require substantial engineering and training data while limiting trust, acceptance, responsibility, and transparency.Their lack of explicit decision processes creates difficulties for developers and human experts, particularly in medicine.
- Human involvement: Medical decision processes therefore require human experts who can understand, evaluate, and potentially oversee algorithmic results.Interactive learning frameworks allow users to interact with and correct explanations, combining human expertise with machine learning.
- Human involvement: Interactive machine learning supports re-traceability, transparency, and explainability, especially when data are scarce or complex.These mechanisms are presented as important characteristics for future information systems.
- Evaluating explanations: Existing studies assess explanation effectiveness through qualitative ratings or proxies such as simplicity and response time, leaving direct utility measurement underdeveloped.The paper addresses this gap by adapting a usability approach to measure users’ perceptions of explanation utility, including causal aspects.
2 Causability and Explainability
The explanation process links human and machine statements to representations, knowledge, context, models, and ground truth. Its central requirement is that statements align with ground truth and explanations expose the model parts used to produce them.
- Definitions: A statement s may be produced by either a human, s_h, or a machine, s_m.The paper treats human and machine statements as components of the same explanation process.
- Definitions: Statements are defined as s = f(r, k, c), combining representations, pre-existing knowledge, and context.For machines, context is the technical runtime environment; for humans, it is the physical environment in which the decision was made.
- Ground truth: An unknown or unobserved fact, u_e, represents ground truth that human and machine models attempt to model.The paper relates this framing to hidden or latent variables in several probabilistic model families.
- Explanation process: The overall goal is for statements to be congruent with ground truth and explanations to highlight the applied parts of the model.Figure 1 presents this relationship among representations, knowledge, statements, models, explanations, and ground truth.
3 Process of Explanation and the importance of a Ground Truth
The paper distinguishes technical explainability from causability and frames explanation quality around causal understanding, effectiveness, efficiency, and satisfaction. It also highlights that ground truth and mappings between machine and human models are often difficult to define.
- Ideal explanations align human and machine statements with a shared ground truth.
- Medical diagnosis may lack a well-defined ground truth, while successful machine models commonly rely on correlation, similarity, or distance rather than causal mechanisms.
- Explainability highlights decision-relevant parts of machine representations and models, whereas causability concerns users’ causal understanding.
- Causability measures whether an explanation achieves causal understanding with effectiveness, efficiency, and satisfaction in a specified context.
- Measuring machine explanations requires either a causal machine model or a mapping between machine and human-understandable models.
4 Background
The System Usability Scale provides the conceptual and practical background for the proposed approach. Its appeal comes from a short questionnaire, while prior research also identifies interpretive and dimensionality limitations.
- The System Usability Scale measures user-interface usability, while SCS targets explanation usefulness and explanation-interface usability.
- SUS was introduced by John Brooke in 1986 and later published as a book chapter.
- SUS uses a 10-item questionnaire with five response options to provide a rapid usability assessment.
- SUS scores range from 0 to 100 after response conversion, but they are not percentages and are best interpreted through percentile rankings.
- Factor analysis indicates that SUS contains Usable and Learnable factors rather than being strictly unidimensional.
5 The System Causability Scale
The paper proposes SCS as a Likert-scale instrument for rapidly evaluating whether explanations and their interfaces suit their intended purpose. It illustrates the scale with a medical prediction model and lists causal-understanding criteria for users.
- SCS adapts the Likert-scale approach used by SUS to evaluate explanation interfaces, explanations, or explanation processes.
- The questionnaire asks whether explanations include relevant causal factors, fit the user’s work context, and support adjustable detail.
- Additional items assess support needs, causal understanding, compatibility with the user’s knowledge base, consistency, learning speed, and reference needs.
- SCS was illustrated by applying it to the Framingham Risk Tool, a 10-year coronary-artery-disease prediction model.
- The Framingham application used inputs including sex, age, cholesterol, smoking, HDL cholesterol, systolic blood pressure, and hypertension treatment.
- Table 1 reports SCS ratings on a five-point scale from strongly disagree to strongly agree.
6 Conclusions
The conclusion presents SCS as a simple, rapid tool for evaluating explanation quality while cautioning that Likert responses are ordinal rather than equal-interval measurements. The paper therefore warns that inappropriate statistical treatment can lead to incorrect conclusions.
- SCS is intended to provide simple and rapid evaluation of explanation-interface or explanation-process quality.
- Likert response categories have an order, but the intervals between categories cannot be presumed equal.
- Using statistical techniques suited to interval data on Likert-type responses can increase the chance of incorrect conclusions.
Abbreviations
The paper defines abbreviations for artificial intelligence, machine learning, and related systems and tools.
- AI means Artificial Intelligence, while ML means Machine Learning.
- aML, DL, and iML denote automatic Machine Learning, Deep Learning, and interactive Machine Learning.
- FRT, SCS, and SUS denote the Framingham Risk Tool, System Causability Scale, and System Usability Scale.