Source-linked AI summary
XAI Evaluation Cards: A Practical Method for Designing Human-Centred XAI Evaluations
Kristýna Sirka Kacafírková, Ivania Donoso-Guzmán, Denis Parra, Katrien Verbert, An Jacobs
TL;DR
XAI evaluation is often fragmented, and practical guidance for choosing and prioritising human-centred evaluation properties remains limited. The paper introduces evidence-informed evaluation cards and a three-step planning process. Initial testing suggests teams found the exercise helpful for structuring evaluation discussions.
Problem
Researchers have limited practical guidance for selecting and prioritising human-centred evaluation properties, while XAI evaluation remains fragmented and often ad hoc.
Source: §1 · p. 2Method
The XAI Evaluation Cards turn an evidence-informed framework into a structured process for identifying relevant properties, prioritising them, and exploring their operationalisation.
Source: §1 · pp. 2–3Results
Initial testing suggests research teams found the exercise helpful for structuring evaluation discussions.
Source: §8 · p. 20Takeaways & Limitations
The cards can provide teams with a shared and structured space for evaluation discussions.
Source: §6 · p. 17The authors describe this as initial testing and call for less abstract terminology and context-tailored examples.
Source: §7 · pp. 19–20
Abstract
from arXiv · showhide
Evaluating explainable AI (XAI) systems from a human-centred approach requires researchers to select from numerous evaluation dimensions and measures, often in an ad hoc and fragmented manner. This paper introduces a method to help HCI, computer science, designers and social science researchers systematically evaluate XAI systems. The approach is based on an updated XAI-specific evaluation framework derived from an analysis of 82 studies. Using this framework, we developed a card-sorting method with 36 cards to help researchers prioritise relevant evaluation aspects. The process was tested with two research groups (n = 13) across five projects. The XAI Evaluation Cards are available as a printable appendix, along with an online repository of methods from previous XAI studies. Although not exhaustive, our findings indicate that the card-sorting approach can organise and streamline the design of the evaluation process, encouraging a more comprehensive and multidisciplinary assessment of XAI systems in research and development.
1 Introduction
XAI evaluation remains fragmented, and researchers lack practical guidance for selecting and prioritising human-centred evaluation properties. The paper introduces evidence-informed cards to structure evaluation planning and help teams connect selected properties to measures.
1 Introduction
- Technical evaluation can assess model performance, fidelity, or robustness, while human-centred evaluation examines how explanations are perceived, interpreted, and used in practice.The paper presents objective and subjective approaches as complementary forms of evidence, particularly in interdisciplinary teams.Source: §1 · p. 2
- Existing resources provide limited practical guidance for choosing, prioritising, and operationalising human-centred evaluation properties for a specific use case.Reviews and frameworks often identify what could be evaluated without explaining how teams should select and apply properties.Source: §1 · p. 2
- XAI evaluation remains fragmented and often ad hoc, while empirical evaluations involving users remain comparatively limited [39].Selecting suitable measures can be lengthy and uncertain, and researchers may default to familiar or readily available measures rather than those suited to their system and users.Source: §1 · p. 2
- Interdisciplinary teams bring different conceptual and methodological traditions, making shared understanding of evaluation priorities and operationalisation important.Planning must account for project stage and treat evaluation as iterative across design stages.Source: §1 · p. 2
- The XAI Evaluation Cards translate a framework derived from 82 studies into a structured conversation and prioritisation tool, accompanied by a repository of previously used measures.The process identifies relevant properties, prioritises them for a use case and project stage, and explores how to operationalise them.Source: §1 · p. 3
- The pilot study provides an initial evaluation of how the cards support identifying and prioritising properties needed for evaluation operationalisation.The introduction also frames systematic evaluation as supporting more transparent, comparable, and comprehensive XAI research.Source: §1 · pp. 2–3
2 Related work
This section presents a context-dependent card-based method for helping teams collaboratively prioritize evaluation properties, make the broader evaluation space visible, and turn abstract metrics into a concrete measurement plan.
2.1 The Landscape of XAI Evaluation Frameworks
- XAI evaluation remains fragmented, with frameworks using inconsistent or undefined concepts and sometimes using the same behavioral data to support different constructs.These inconsistencies can undermine scientific rigor and make findings appear contradictory when studies may measure different phenomena under the same label.Source: §2.1 · p. 3
- Existing literature offers a menu of evaluation properties but little guidance for selecting and operationalising them in specific contexts.Conceptual inconsistencies can make it difficult to identify effects of explanations on users.Source: §2.1 · p. 4
2.2 Navigating interdisciplinary collaboration in XAI evaluation
- Interdisciplinary XAI work is hindered by divergent role-specific meanings and success criteria, late UX involvement, and a lack of shared language for evaluation decisions.The section argues that teams need translation tools to make priorities visible and negotiate what to evaluate at each development stage.Source: §2.2 · p. 5
2.3 Card-based methods and XAI domain
- Existing XAI card toolkits focus on creating explanations or eliciting user needs, leaving teams without collaborative tools to plan evaluation measurements and testing.The gap is especially consequential given fragmented evaluation frameworks.Source: §2.3 · p. 6
- The proposed card method treats priorities as context-dependent, helping teams select properties for their use case and development stage and negotiate a shared evaluation plan rather than universal rules.It adapts cards for interdisciplinary XAI evaluation planning.Source: §2.3 · p. 6
- The exercise helps teams prioritize evaluation properties they consider most important for their specific project and design.Source: §2.3 · p. 6
- The exercise makes the broader evaluation space visible and helps teams translate abstract metrics into a concrete action plan.Source: §2.3 · p. 6
3 Development of the XAI Evaluation Cards Method
The method combines a review-informed framework, cards, and an online repository to help teams plan XAI evaluations by selecting and prioritising properties for a specific use case and project stage. It distinguishes planning and development evaluations from evaluations of more mature systems in use, and describes ways teams can prepare for a card session.
3.1 Stage 1: Systematic review of evaluation in existing XAI studies
- The review of 82 XAI studies informed updates to an existing evaluation framework, refining its property and dimension definitions while retaining its structure [9].Three HCI researchers coded the studies and refined the framework through consensus.Source: §3.1 · pp. 6–7
3.2 Stage 2: Translation of the framework into cards
- Stage 1 properties were translated into cards to make the framework actionable and help teams discuss and prioritise evaluation properties for their use case.Cards focus on five categories; personal and situational characteristics are omitted as contextual or individual-level control variables, but should still be considered for each use case. Each card includes a definition and prior measurement examples as options, not prescriptions.Source: §3.2 · pp. 7–9
3.3 The Evaluation Measurements Repository
- The accompanying repository lets researchers search reviewed studies by evaluation property and measurement approach to find concrete examples for their own studies.It takes a broad human-centred perspective, including subjective, user-centred, experiential, and interaction-related measures.Source: §3.3 · p. 9
3.4 Intended Usage and Project Design Stages
- The cards guide a three-step process: identify relevant properties, prioritise what to evaluate at the current project stage, and explore how to measure them across study designs and disciplines.Teams first define an XAI use case and consider the project’s current development stage when judging relevance and priority.Source: §3.4 · p. 10
- Projects are grouped into planning, development, and implementation stages, adapted from design-process models.These encompass empathise and define; ideate and prototype; and test and implement, respectively.Source: §3.4 · p. 10
- During planning and development, evaluation can inform design decisions about the type, content, and presentation of explanations.Source: §3.4 · p. 11
- At implementation, evaluation is more likely to assess an existing or more mature system in use.Source: §3.4 · p. 11
- Teams may familiarise themselves with the framework and cards beforehand, or assign a member to guide the session, answer questions, and help it run smoothly.In testing, the moderator was the paper’s first author.Source: §3.4 · p. 11
4 User Evaluation of the Method
The authors tested the XAI Evaluation Cards with research teams working on five projects, using collaborative sorting and prioritisation to explore how the method supports evaluation planning.
4.1 Participants and Study context
- Two one-hour sessions involved 13 researchers in six groups across five XAI projects; two groups independently addressed the same healthcare use case.Participants were colleagues of the authors but were not directly involved in developing the method.Source: §4.1 · p. 11
4.2 Study Protocol and Procedure
- Groups introduced their projects, sorted properties by relevance, then prioritised relevant properties and discussed how to translate selections into an evaluation plan.The protocol included optional category ranking, which could be omitted in future sessions.Source: §4.2 · pp. 12–13
- Most groups completed the protocol within an hour, but Group 6 did not finish relevance sorting, so its card-selection data were excluded from comparisons.Groups generally ranked properties within each of the five categories rather than against all selected properties.Source: §4.2 · p. 13
4.3 Data collection and Analysis
- Researchers recorded selections, priorities, observations, and audio; they descriptively compared selections and explored top-ranked properties across design stages.Transcripts supplied illustrative quotations, but were not separately thematically analysed.Source: §4.3 · pp. 13–14
4.4 Ethical considerations
- Participants consented to recording and research use of their data; only high-level project descriptions were reported to protect confidentiality and intellectual property.Source: §4.4 · p. 14
5 Pilot Results
The pilot found that teams’ evaluation priorities varied by project, while every framework property was considered relevant by at least one group. Participants also described how sorting and prioritisation supported planning, while surfacing needs for measurement guidance and clearer definitions.
5.1 Card selection and prioritisation
- The earliest-stage Group 5 selected 80% of Objective System Aspects, 60% of Subjective System Aspects, and 25% of Interaction properties.Some user-oriented properties remained abstract because the system and interaction context were not yet fully defined.Source: §5.1 · p. 15
- Every framework property was considered relevant by at least one group, although the number and type selected varied considerably across groups.AI Model Performance was the top Objective System Aspect for three of four first-session groups; other selected properties were more project-specific.Source: §5.1 · p. 15
- In the implementation-stage groups, highest-priority properties formed a relatively concentrated set, whereas development-stage groups selected a broader range.Only one planning-stage group was available, and the authors describe stage patterns as exploratory given the small number of groups.Source: §5.1 · p. 15, Figure 9 · p. 16
5.2 How can the card sorting support XAI evaluation
- Participants used prioritisation to distinguish potentially interesting properties from those important enough to include, since evaluating everything was considered unrealistic.Source: §5.2 · p. 16
- Selecting a property did not settle how to measure it; participants sought validated measures, examples of operationalisation, and information about properties commonly evaluated together.Most groups saw the proposed repository as potentially useful, with suggestions to describe study types and sample-size needs.Source: §5.2 · pp. 16–17
- Participants sometimes struggled to distinguish related property definitions, and moderator examples helped them apply properties to their use cases.They suggested clearer distinctions and more recognisable real-life examples for future card versions.Source: §5.2 · p. 17
6 Discussion
The sessions suggest that the cards structure a shared, context-sensitive discussion of XAI evaluation priorities rather than prescribe a universal checklist. They also point to roles for facilitation, multidisciplinary input, revisiting priorities as projects evolve, and pairing property selection with measurement resources.
6 Discussion
- The cards provided a shared, structured space for teams to decide which evaluation properties fit their context and design stage, rather than use a universal checklist.Source: §6 · p. 17
6.1 Facilitation supports the use of the cards
- A moderator familiar with the framework helped resolve unclear or overlapping definitions and keep discussion tied to the use case and project stage.The authors suggest facilitation may be particularly valuable for first-time users, while clearer definitions and examples could reduce reliance on it.Source: §6.1 · p. 18
6.2 Multidisciplinary participation broadens evaluation planning
- Researchers from different disciplines brought contrasting perspectives that prompted discussion of overlooked properties, including user-centred considerations in a technically focused session.The cards may serve as a shared vocabulary for teams to surface assumptions and negotiate priorities.Source: §6.2 · p. 18
6.3 Evaluation priorities change across design stages of development
- Priorities varied with project maturity and stage; in this sample, implementation-stage projects prioritised a narrower set than earlier-stage projects.Earlier projects sometimes emphasised system-level properties before user interaction was defined, while later projects could draw on prior evaluation activities.Source: §6.3 · p. 18
6.4 The interaction format should support collaborative reflection
- The exploratory online session was less successful, but differing facilitation, use-case familiarity, and interaction format prevent attributing this to digital cards alone.The authors suggest a digital version should retain flexibility and collaborative reflection.Source: §6.4 · pp. 18–19
6.5 From identifying properties to selecting measurements
- The cards support deciding what to evaluate, while the accompanying repository provides examples of how properties have been operationalised.Together, they distinguish property selection and prioritisation from choosing how to measure those properties.Source: §6.5 · p. 19
7 Limitations and Future work
The authors describe this work as an initial, formative test of the cards. They call for broader evaluation and further development of the cards and repository, including testing whether the method shapes evaluations in practice.
7 Limitations and Future work
- The study tested the cards with six groups across a limited number of projects; the authors call for evaluation across more domains, project types, team compositions, and design stages.The pilot aimed to observe boundary negotiation rather than provide summative validation of the toolkit’s efficacy.Source: §7 · p. 19
- Some participants struggled with property definitions or recognised concepts under different terms; future cards should offer clearer definitions and cross-disciplinary examples.The authors expect these changes may reduce reliance on moderator support.Source: §7 · p. 19
- The accompanying repository remains under development, with validated measures, joint operationalisations, and guidance for study designs and contexts among the planned additions.The authors also call for user testing to refine its needs, structure, content, and usability.Source: §7 · p. 19
- The study does not yet show whether the method produces more appropriate, comprehensive, or effective XAI evaluations in practice.Future work should track how selected properties become study designs and affect evaluation decisions over time.Source: §7 · pp. 19–20
- Use with end users is being explored, but the authors note that this audience may need less abstract terminology and context-specific examples.Source: §7 · p. 20
8 Conclusion
The paper introduces and initially tests the XAI Evaluation Cards as an evidence-informed method for evaluation planning. Initial testing suggests the cards help structure team discussions and broaden the evaluation properties considered.
8 Conclusion
- The evidence-informed method gives teams a shared space to explore relevant properties, discuss their fit to a use case and design stage, and prioritise them with measurement guidance.Source: §8 · p. 20
- Teams found the exercise helpful for structuring discussions and welcomed a repository to translate prioritised properties into concrete measures and study designs.The conclusion presents these as findings from initial testing.Source: §8 · p. 20
- The findings suggest the cards broaden properties considered beyond familiarity or disciplinary background, bringing objective and subjective properties into a more comprehensive, context-dependent evaluation.Source: §8 · p. 20
- Rather than prescribe fixed criteria, the cards support situated, iterative planning in which priorities can be revisited as the system, use case, and available evidence evolve.Source: §8 · p. 20