Stop ordering machine learning algorithms by their explainability! A user-centered investigation of performance and explainability

Motivation

• The authors frame perceived explainability as consequential because decision-makers may be more willing to act on predictions they can interpret.They note that users may disregard advice from systems they find insufficiently explainable.Source: §1 · p. 2

Research gap

• The paper identifies a lack of empirical evidence about the performance–explainability tradeoff from end users’ perspective.Prior theoretical accounts describe a tradeoff, but its relation to perceived explainability remains insufficiently understood.Source: §1 · pp. 1–2

Research questions

• The first research question compares common machine-learning algorithm classes on accuracy and end users’ perceived explainability.It asks which algorithms users perceive as more explainable and how those perceptions relate to measured accuracy.Source: §1 · p. 2
• The second research question compares end users’ perceived explainability across common types of XAI explanations.The paper notes that external explanations can augment predictions from high-performing algorithms that lack inherent explainability.Source: §1 · p. 2

Study aims

• The study aims to clarify how algorithm classes differ in perceived explainability and how much performance users may be willing to forfeit for it.The authors connect this evidence to assessing algorithms’ suitability for particular situations.Source: §1 · p. 2

2.1. Machine learning algorithms

• Shallow algorithms commonly require feature selection, whereas deep neural networks learn representations from data but produce models that are difficult for humans to retrace.The review describes linear regression, decision trees, and SVMs as common shallow classes and deep networks as data- and computation-intensive.Source: §2.1 · p. 2

2.2. Interpretability and explainability in machine learning

• Interpretability is an inherent model property, while explainability concerns how users perceive an explanation’s quality.The review defines interpretability in terms of associating cause and effect and describes perceived explanation quality as potentially circumstantial.Source: §2.2 · p. 2
• Global explanations address how an algorithm processes information, while local explanations address why it made a particular prediction.The review treats a model’s innate interpretability as its explainability to users when no XAI augmentation is used.Source: §2.2 · pp. 2–3
• It remains unclear whether the assumed continuous decline in explainability as model performance rises holds in user evaluations across different models.The literature also distinguishes users from developers, theorists, and ethicists as a target audience for explanations.Source: §2.2 · p. 3

3.1. Machine learning tradeoffs

• Existing classification schemes generally order algorithms from deep neural networks as high-performance, low-explainability models to linear models as lower-performance, more interpretable models.The synthesis includes five classes: deep neural networks, ensembles, SVMs, decision trees, and linear models.Source: §3.1 · p. 3
• These schemes are theoretical and data-centered, and their ordering has not been validated across real applications with end users.The section also notes that human-centered XAI evaluations often test explanatory performance against a truth value rather than explanation quality for end users.Source: §3.1 · p. 3
• The study proposes assessing inherent explainability separately from post-hoc XAI augmentations, including XAI for the highest-performing algorithm.The authors describe augmentations as intended to make high-performance models more transparent, while noting that prior evaluations often focus on novel algorithms.Source: §3.1 · pp. 3–4

3.2.1. Performance vs. explainability tradeoff

• For the first research question, the design compares algorithm choice on objective performance and perceived explanation quality, with data complexity as a moderator.The hypotheses anticipate little performance difference for low-complexity tabular data and greater differences for high-complexity data.Source: §3.2.1 · p. 4
• The authors expect decision trees to stand out in perceived explainability for low-complexity cases, while hypothesizing no algorithm effect for high-complexity cases.They argue that tree structures may be difficult to interpret for image data with many input variables.Source: §3.2.1 · p. 4

3.2.2. Perception of explanation types

• For the second research question, the study fixes the deep-learning model and high-complexity dataset, then compares post-hoc explanation types.The authors expect more local explanations to be better received by end users than global explanations.Source: §3.2.2 · p. 4

4.1. Performance vs. explainability tradeoff

• The first experiment compares algorithm classes on a moderate-sized, low-dimensional tabular dataset and a large, high-dimensional image dataset.Both datasets are classification tasks, and the study applies preprocessing and grid-search optimization across the algorithms.Source: §4.1 · pp. 4–5
• Performance is measured as classification accuracy using 15-fold cross-validation, with recall and F-score reviewed as checks.The study measures perceived goodness of explanation separately through a survey of novice healthcare professionals enrolled in medical studies.Source: §4.1 · p. 5
• Participants rated each algorithm’s inherent explanation on a seven-point Likert scale after receiving a case introduction and prediction visualization.The study randomized treatment order, withheld model-performance information, and used a validation question as an awareness check.Source: §4.1 · p. 6

4.2. Perception of explanation types

• The second experiment uses the complex BRAIN dataset with a deep neural network and compares XAI explanation treatments through a survey.Treatments pair the original image and model decision with an explanation and short description; How-To and What-If are combined, with What-If scope limited to input adjustments.Source: §4.2 · pp. 6–7

5.1.1. Participant demographics

• Experiment I received 223 responses and retained 100 HEART and 101 BRAIN participants after data-quality preprocessing.Source: §5.1.1 · p. 7

5.1.2. Performance

• In HEART, linear regression significantly differed from all other algorithms (p < 0.01), while the other models showed no significant distinction.Source: §5.1.2 · p. 7
• Performance ordering was broadly as theorized but not deterministic: models were nearly similar in low-complexity HEART, whereas the deep neural network separated above all others in complex BRAIN.In BRAIN, all models except the deep neural network performed notably worse.Source: §5.1.2 · p. 7
• In BRAIN, linear regression performed worst and decision trees also fell behind; random forests and SVMs were similar, while deep neural networks formed a distinct top group.Source: §5.1.2 · p. 7

5.1.3. Explainability

• Across treatments, random forests and decision trees had the highest or second-highest explanation ratings; H2a was supported, but H2b was not.Source: §5.1.3 · p. 7
• In low-complexity cases, decision trees and random forests were rated similarly, as were SVMs and linear regression.Source: §5.1.3 · p. 7
• In complex cases, decision-tree ratings declined but remained distinguishable from random forests; deep neural networks differed from every other algorithm, consistent with their lack of inherent interpretability.Source: §5.1.3 · pp. 7–8

5.2.1. Demographics

• Experiment II’s BRAIN-XAI survey received 109 responses and retained 98 after the same preprocessing approach.Source: §5.2.1 · p. 8

• The user-centered treatments do not support the hypothetical gradual tradeoff curve; perceived explainability instead has a grouped structure.The discussion averages and normalizes first-experiment data across use cases for comparison with the theoretical scheme.Source: §6.1 · p. 8, Figure 3 · p. 9
• Decision trees and random forests are perceived as most explainable among five models, without a substantial perceived advantage for single trees over random forests.The authors suggest participants may not have reviewed all resulting random-forest trees.Source: §6.1 · p. 9
• The tradeoff depends on case complexity and real-world performance and explainability requirements, making algorithm selection a multi-criteria decision.The discussion notes that performance and explainability behave differently across cases.Source: §6.1 · p. 9
• The follow-up survey compares five common explanation types as possible augmentations for high-performing algorithms.The study reports that explanation design matters alongside the content displayed.Source: §6.1 · p. 9
• End users favor local Why and What-Else explanations over global How and How-To; results for contrastive Why-Not explanations are ambiguous.Why explanations score above Why-Not, while How-To is perceived more benevolently than How.Source: §6.1 · p. 9, Figure 5 · p. 10
• As dataset complexity rises, deep learning's performance advantage grows, while tree-based models' absolute perceived explainability wanes and overall explainability distance shrinks.For low-complexity datasets, deep-learning performance gains over shallow ML are negligible.Source: §6.1 · p. 9, Figure 4 · p. 10

6.2.1. Tackling the tradeoff between performance and explainability is non-trivial

• The tradeoff is not always a curve: performance and explainability require weighted, contextual decisions shaped by policy, law, and user perceptions.The authors note that algorithm distances vary, some effects may be latent, and explainability investments may be in vain if they do not translate into firm productivity; they call for further mixed-method research.Source: §6.2.1 · p. 11

6.2.2. Model-inherent interpretability does not entail explainability

• From a user-centered perspective, perceived explainability forms three groups: none for deep neural networks, mediocre for coefficient-based models, and high for tree-based models.The authors contrast this pattern with a theoretical view in which the performance–explainability difference increases constantly.Source: §6.2.2 · p. 11
• Trees offer accessible paths to decisions, coefficient models share interpretable variable weights, and deep neural networks need XAI to explain predictions to end users.The authors suggest explanation goodness may depend more on design and basic explanation type than on the specific content.Source: §6.2.2 · p. 11

6.2.3. Explanations and XAI augmentations that require low cognitive effort fare better

• End users prefer low-effort local Why and Why-Not over complex global How and How-To explanations; What-Else examples are positively received.The experiment measured initial exposure only, so it did not capture learning through teaching or experience that might make other explanations more accessible.Source: §6.2.3 · p. 11

6.3.1. Start with the performance threshold

• Every candidate algorithm must first meet the task's minimum performance threshold; an explainable model below that threshold has no practical value.The threshold is determined by the system's overarching goal, which may include business goals such as cost savings.Source: §6.3.1 · p. 11

6.3.2. Consider organizational or project context beyond performance

• After performance is adequate, cost, inference time, user abilities, and laws can constrain algorithm choice and acceptable explanations.Tree-based models are particularly accessible to ML laypeople, while coefficient-based models may suit a trained workforce.Source: §6.3.2 · p. 11

6.3.3. Consider the degree of explanation that end users need

• Explanation design should fit end users' work: novice users may respond best to local Why and What-Else, while global How and How-To can require more effort.The passage distinguishes end users' need to assess predictions from experts' need to analyze model decision processes.Source: §6.3.3 · p. 11

6.4. Limitations and future research directions

• Participants may have encountered the scenarios and algorithms for the first time, so the study measured initial explainability; both datasets came from healthcare.The authors note that healthcare data may introduce bias and that results could differ in high-velocity treatment settings, where faster inference matters.Source: §6.4 · p. 12
• The first experiment compared inherently interpretable shallow models with deep neural networks without XAI augmentations; the authors call for comprehensive evaluation of augmentations and combinations of explanations.The surveys had no time constraint, which the authors expect could matter in high-velocity treatment settings.Source: §6.4 · p. 12
• The random-forest ensemble may overestimate ensemble explainability because its decision-tree base class is highly explainable.The authors suggest evaluating multiple ensemble models with varied base-class permutations for a more objective overview.Source: §6.4 · p. 12
• The study did not measure whether ML algorithms or XAI augmentations improved end-user task performance.The authors also note possible expectation and halo biases, and that other factors affecting overall impressions were controlled out.Source: §6.4 · p. 12

7. Conclusion

• The study empirically examined the performance–explainability relationship and explored how different explanation types affect end users.The authors note that empirical evidence on this tradeoff is scarce.Source: §7 · p. 12
• End-user explainability formed groups of no, mediocre, and high explainability rather than a continuous tradeoff, with decision trees and random forests consistently dominating other ML models.The authors characterize data-centered theoretical tradeoff propositions as misleading oversimplifications for end-user explainability.Source: §7 · p. 12
• Explanations requiring less cognitive effort, such as local explanations, fared better.The authors report measuring end-user perception rather than understanding, learning effects, or task performance.Source: §7 · p. 12

CRediT authorship contribution statement

• Lukas-Valentin Herm and Kai Heinrich are credited with conceptualization, methodology, validation, formal analysis, investigation, data curation, writing, and visualization.Herm and Heinrich have the same listed contribution areas.Source: CRediT authorship contribution statement · p. 12
• Jonas Wanner is credited with conceptualization, methodology, validation, formal analysis, investigation, data curation, and writing the original draft.Visualization and resources are not listed among Wanner’s contributions in this statement.Source: CRediT authorship contribution statement · p. 12
• Christian Janiesch is credited with conceptualization, methodology, resources, supervision, and writing.The statement lists review and editing, but not validation, formal analysis, investigation, or data curation, for Janiesch.Source: CRediT authorship contribution statement · p. 12