Source-linked AI summary
One Explanation Does Not Fit All: The Promise of Interactive Explanations for Machine Learning Transparency
Kacper Sokol, Peter Flach
TL;DR
Black-box ML predictions require explanations that address diverse stakeholder expectations, yet one-size-fits-all methods remain limited. The paper develops and analyses interactive, personalised explanations using contrastive counterfactuals, showing through Glass-Box that users can adjust explanation content and ask follow-up questions. It also identifies engineering overhead, domain-specific generalisation issues, and other desiderata for interactive explainers.
Problem
Existing explainability methods often provide noninteractive, one-size-fits-all explanations that do not accommodate diverse user expectations or customised questions.
Method
The paper proposes and examines an interactive XAI architecture using personalised contrastive counterfactual explanations, natural-language interaction, and follow-up “What if?” questions.
Results
Building and demonstrating Glass-Box yielded desired system properties, lessons for interactive explainers, and evidence that personalising explanation content is more important than customising only the interaction medium.
Takeaways & Limitations
Interactive explanation should let explainees modify the explanation itself and guide its breadth, scope, context, purpose, and target.
Takeaways & Limitations
Interactive explainers incur substantial engineering overhead, and adapting them to new datasets may require annotations and newly trained natural-language-processing models.
Abstract
from arXiv · showhide
The need for transparency of predictive systems based on Machine Learning algorithms arises as a consequence of their ever-increasing proliferation in the industry. Whenever black-box algorithmic predictions influence human affairs, the inner workings of these algorithms should be scrutinised and their decisions explained to the relevant stakeholders, including the system engineers, the system's operators and the individuals whose case is being decided. While a variety of interpretability and explainability methods is available, none of them is a panacea that can satisfy all diverse expectations and competing objectives that might be required by the parties involved. We address this challenge in this paper by discussing the promises of Interactive Machine Learning for improved transparency of black-box systems using the example of contrastive explanations -- a state-of-the-art approach to Interpretable Machine Learning. Specifically, we show how to personalise counterfactual explanations by interactively adjusting their conditional statements and extract additional explanations by asking follow-up "What if?" questions. Our experience in building, deploying and presenting this type of system allowed us to list desired properties as well as potential limitations, which can be used to guide the development of interactive explainers. While customising the medium of interaction, i.e., the user interface comprising of various communication channels, may give an impression of personalisation, we argue that adjusting the explanation itself and its content is more important. To this end, properties such as breadth, scope, context, purpose and target of the explanation have to be considered, in addition to explicitly informing the explainee about its limitations and caveats...
1 Introduction
The paper argues that one-shot explanations cannot satisfy diverse stakeholder needs, and proposes personalising explanation content through interaction. It illustrates this approach with editable representations and interactive counterfactual dialogue.
- Motivation: Black-box ML systems need explanations for stakeholders, but available interpretability methods cannot satisfy all competing expectations.The paper contrasts ante-hoc transparency with post-hoc tools and frames interactive explanation as a response to this diversity.
- Motivation: Most explainability methods provide a single explanation optimised by a predefined metric, limiting end-user customisation.The paper distinguishes methods that cannot be customised from those accepting user input at different expertise levels.
- Interactive personalisation: Interactive explanation personalises content and process by allowing users to constrain counterfactual features and ask user-specific questions.For counterfactuals, users can specify which features may or may not appear in the conditional statement.
- Examples: Users can modify image segmentation, feature representations, or word groupings so explanations reflect meaningful concepts in images, tabular data, and text.The image example allows users to merge or split super-pixels; analogous adjustments apply to tabular and text representations.
- Challenges: Interactive explainers require coordinated user-interface, language-processing, dialogue, and XAI components, many of which are domain-specific.The paper identifies this component diversity as an engineering and generalisation challenge.
- Interactive personalisation: The proposed system focuses interaction on personalising explanations through natural-language dialogue rather than merely presenting multiple explanation techniques.This distinction separates interactive explanations from explainability systems whose interface is interactive but whose explanation content is fixed.
2 Background and Related Work
Related work spans interactive ML, interactive explainability interfaces, and theories of explanatory dialogue. The paper argues that presenting explanations interactively is insufficient unless users can modify their content and pursue targeted questions.
- Research strands: The literature includes interactive ML, multi-modal explainability tools, and theories that model explanatory communication as dialogue between intelligent agents.These strands approach interaction from model refinement, interface design, and conversational perspectives.
- Interactive tools: Interactive multi-modal tools offer diverse explanations through a unified interface, but combining methods can produce contradictory evidence.Coherence is difficult when different explanation techniques support competing accounts of the same outcome.
- Interactive tools: The paper distinguishes interactive presentation from truly interactive explanations that let users tune and personalise explanation content.Choosing an explanation technique or viewing interactive plots changes the medium, not necessarily the explanation itself.
- Human-centred XAI: Human-centred XAI research emphasises dialogue-like, contrastive explanations, yet one-off explanations remain the dominant operationalisation.The authors identify limited uptake of bidirectional conversations that let explainees customise explanations.
- Open challenges: Interactive personalisation requires feedback-capable interfaces, interdisciplinary collaboration, and more elaborate evaluation involving potentially time-consuming user studies.The literature identifies both implementation and validation burdens as barriers to adoption.
- Prior systems: Existing personalised-explanation work includes dialogue-driven systems, offline feature selection, and Google’s interactive What-If tool.The paper notes that some approaches require application-specific ontologies, while others personalise explanations offline.
- This work: The paper presents a voice-driven contrastive explainer for loan applications and derives design guidelines from building, deploying, and demonstrating it.The system lets users interrogate predictions with counterfactual questions and supports the paper’s analysis of desiderata and lessons learned.
3 Interactively Customisable Explanations
Glass-Box demonstrates interactive customisation of counterfactual explanations through natural-language “Why?” and “What if?” dialogue. The section also identifies design goals, user feedback, and limitations of this approach.
- Glass-Box Design: Glass-Box provides personalised counterfactual explanations through natural-language dialogue, allowing users to challenge and customise model decisions.The system supports voice- or chat-based interaction and was demonstrated to domain experts and lay audiences.
- Glass-Box Design: The system was implemented with a decision-tree classifier trained on a subset of the UCI German credit dataset and supports binary class-contrastive explanations.Its natural-language input was processed using a Rasa-based module, while predefined fictional data points simplified demonstrations.
- Interactive Queries: Users can request plain, exclusion-constrained, or feature-conditioned counterfactuals, combine these constraints, and ask “What if?” questions.Repeated questions enumerate explanations under increasingly restrictive feature conditions.
- Explanation Scope: Interactive explanations can shift from individual predictions toward cohort or whole-model questions, while ante-hoc explanations remain truthful to the underlying model.Contradictory explanations therefore signal possible flaws in the predictive model rather than inconsistency between separate explanation mechanisms.
- Explanation Desiderata: Design goals include concise explanations, user-appropriate complexity, mental-model awareness, and explicit communication of context and limitations.Glass-Box attempted to avoid repeated answers and adapt explanations to interests inferred from prior questions, but sometimes produced lengthy monologues.
- Reception and Feedback: Participants generally valued personalised answers, but some criticised the voice interface and worried that the system could not capture the complexity of individual cases.Several participants switched from speech synthesis to text chat because responses sounded slow, unnatural, or clunky.
4 Discussion
The discussion distills lessons from building Glass-Box, emphasizing substantial engineering demands, constrained generalisability, and the benefits and limits of interactive personalisation. It also outlines domain-specific explanation controls and security, usability, and mental-model challenges.
- Lessons Learnt: Building, deploying, and refining interactive explainers requires substantial engineering effort despite adapting off-the-shelf components.The overhead includes voice-enabled hardware, natural-language processing, and system integration.
- Lessons Learnt: Glass-Box’s usability was limited by lengthy personal-data entry, which the authors addressed with predefined individuals and attribute editing.They retrospectively recommend a dedicated input form to give users fuller control and reduce interrogation time.
- Lessons Learnt: Interactive personalisation can assess individual fairness and tailor explanations, but only some explainability algorithms support such customisation.Incorporating feedback into the predictive model further restricts applicable techniques because the model must support post-training refinement.
- Lessons Learnt: Fully simulating an explainee’s mental model remains beneficial but currently out of reach across the board.The authors regard this as a central unresolved challenge for natural-feeling explanations.
- Lessons Learnt: Glass-Box explanations cannot be safely generalised from one data point to other similar instances.Users may generalise counterfactuals, but the underlying decision-tree model does not encode causal relationships; contextualising limitations can partially mitigate trust risks.
- Lessons Learnt: Interactive explainers introduce security risks because repeated arbitrary queries may expose model internals and enable reverse engineering or gaming.Possible mitigations include query budgets and obfuscating numerical thresholds, with a resulting transparency-security trade-off.
- Next Steps: Explanation personalisation is implemented differently across tabular, text, and image data domains.Text users adjust multiword concepts, image users modify super-pixel segments, and tabular users indirectly control tree-based feature splits.
- Next Steps: The system supports multiple explanation views, including whole-tree or focused-tree visualisations and feature-importance queries.Users can inspect all feature importances or query selected features.
5 Summary and Conclusions
The paper presents Glass-Box as an interactive, voice-driven counterfactual explainer that personalises explanations through dialogue. Its experience-based conclusions identify engineering overhead, mental-model limitations, and the need to distinguish interactive interfaces from interactive explanations.
- Summary and Conclusions: Glass-Box is a class-contrastive counterfactual system that personalises explanations through natural-language dialogue.The authors report it as the first XAI system tested in the wild, to their knowledge, supporting explanation customisation via interaction.
- Summary and Conclusions: The most important implementation lesson is the substantial engineering overhead required to build an interactive explainer.This remains significant even when many off-the-shelf components are adapted.
- Summary and Conclusions: Wizard of Oz studies are recommended instead when the primary goal is testing explainability techniques, unless the platform will later be deployed.Fully satisfying an explainee’s mental model is also currently out of reach.
- Summary and Conclusions: The literature review finds that many explainability tools do not support interaction, although contrastive approaches can often accommodate it.The paper positions its work as connecting related research strands around personalised explanations.