PaperlayerView exact revision on arXiv

Source-linked AI summary

Integrating Feature Understandability into Feature Selection for Human-Centred Tabular Explanations

Nicola Rossberg, Bennett Kleinberg, Barry O'Sullivan, Luca Longo, Andrea Visentin

arXiv:2610.16627v1cs.HCcs.AIcs.CYcs.LG

TL;DR

Post-hoc explanation evaluation offers little guidance on improving accessibility, even though users’ understanding of textual explanations depends on understanding their cited features. This study incorporates user-rated feature understandability into feature selection and evaluates the resulting explanations; comparative evaluation favored the co-optimised explanations, while random-forest performance remained essentially unchanged.

  • Problem

    Post-hoc explanation evaluation does not identify which aspects to change for better accessibility, although users’ understanding of textual explanations depends on comprehending their cited features.

    Source: §1 · p. 2

  • Method

    The study collects feature understandability ratings across two datasets, selects features for understandability and informativeness or informativeness alone, trains random forests, and evaluates SHAP-based explanations using user feedback.

    Source: §1 · p. 2, §3 · p. 5, §3.1 · p. 6

  • Results

    Random-forest performance remained essentially unchanged and mean understandability increased; individual ratings were directionally favorable but not significant, while comparative preferences for co-optimised explanations exceeded chance significantly.

    Source: §5 · pp. 16–17

  • Takeaways & Limitations

    Integrating understandability into feature selection can produce explanations judged more understandable without degrading model performance.

    Source: §5 · p. 17

    The explanation-quality findings need confirmation across domains and larger cohorts, and possible overlap in selected features may have influenced comparative results.

    Source: §6 · p. 17

Abstract

from arXiv · show

Existing human-centred XAI research predominantly evaluates explanations after they have been generated. However, this fails to provide actionable insights into improving accessibility. This study incorporates human-based feature understandability into feature selection to improve the generation of human-centred explanations. Natural language explanations for machine learning models trained on tabular data often cite influential features in the input data to explain a given decision. As a consequence, users' ability to understand the produced explanation hinges on their understanding of the cited features. To quantify this natural-language feature understandability, Rossberg et al. proposed the Feature Understandability Scale, which assigns numerical `understandability scores' to features based on user feedback. To evaluate the degree to which these scores can improve machine learning explanation quality, this study collects understandability scores across two datasets, proposes an understandability-aware feature selection algorithm to select features that are both understandable and informative, and trains a random forest based on these feature subsets. The SHAP-based explanations generated by this model are evaluated using user feedback, and changes in explanation quality are assessed. The individual evaluation of the explanations showed that while the understandability-based explanations scored higher on average, the difference was not statistically significant. In contrast, the comparative evaluation found that participants preferred these explanations significantly more frequently than expected at the chance level. It is concluded that the integration of understandability can yield more understandable explanations at minimal cost to classification performance, although changes to quality require elucidation in larger cohorts.

1 Introduction

XAI explanations are usually evaluated after generation, which offers little guidance on what to change to improve accessibility. This study proposes integrating feature understandability into feature selection, then evaluating explanations generated from the selected features.

1 Introduction

  • XAI techniques can support auditing by helping identify unstable decisions or decisions based on irrelevant data, including in critical fields such as medicine and finance.Auditing is framed as important for addressing inequitable models arising from biased data or poor training.Source: §1 · p. 2
  • Post-hoc explanation evaluation measures accessibility but does not identify which aspects to change, while users’ understanding of textual explanations depends on comprehending their cited features.Feature understanding is therefore presented as an actionable target for improving explanation quality.Source: §1 · p. 2
  • The Feature Understandability Scale (FUS) uses user ratings across dimensions to assign feature understandability scores that can be incorporated into model training.The scores are expected to favour more understandable features and make resulting explanations more accessible.Source: §1 · p. 2
  • The study collects understandability ratings across two datasets, proposes an understandability-aware feature-selection mechanism, and evaluates explanations generated from models trained on the selected subsets.These are the study’s three successive objectives.Source: §1 · p. 2

2 Literature Review

The review covers XAI, interpretable-by-design models, cost-sensitive feature selection, SHAP, and explanation evaluation. It identifies a gap in using self-reported feature understanding within model training and in evaluating the resulting explanations.

2.1 Explainable Artificial Intelligence

  • XAI approaches include feature attribution, counterfactuals, latent-space mechanistic interpretability, and natural-language methods, supporting audits of black-box decisions.The review links these methods to addressing bias and shortcut learning and promoting transparency and trust.Source: §2.1 · p. 3

2.2 Interpretability by Design

  • Interpretability by design integrates transparency into model architecture and training, although the review notes concerns that it could reduce performance.The review also says that the predicted performance loss is seldom observed in practice.Source: §2.2 · p. 3

2.3 Cost-Sensitive Feature Selection

  • Traditional cost-sensitive feature selection uses feature acquisition or misclassification costs; this study instead treats low understandability as a feature cost and evaluates explanation quality alongside model accuracy.Filter methods pre-select features before model training using feature cost as a scoring function, offering a computationally cheap approach.Source: §2.3 · p. 4

2.4 SHapley Additive exPlanations

  • SHAP provides local and global feature explanations through model-agnostic and model-specific modalities.Its unified framework combined earlier approaches, with later applications including feature selection and outcome explanation.Source: §2.4 · p. 4

2.5 Explanation Evaluation

  • Explanation quality is context- and user-dependent, and the review describes scales for evaluating rule-based explanations and human-AI interfaces.It states that final assessment should be based on user feedback.Source: §2.5 · pp. 4–5

2.6 Synthesis and Literature Gaps

  • The study addresses gaps by incorporating self-reported feature understanding into training a priori, introducing a new cost quantification and evaluation approach, and using preference-based and individual scale-based evaluations.The authors state that, to their knowledge, prior studies had not integrated self-reported feature understanding into the training pipeline on an a priori basis.Source: §2.6 · p. 5

3 Method and Design

The study collects user ratings of feature understandability, selects features for either understandability and informativeness or informativeness alone, and compares the resulting explanations using user feedback. Its methods combine feature-rating surveys, random-forest models, SHAP-based textual explanations, and individual and comparative evaluations.

3 Method and Design

  • The study applies the Feature Understandability Scale across two datasets, compares understandability-aware and informativeness-only feature selection, and evaluates the resulting SHAP-based explanations with users.The workflow comprises rating collection, feature selection and model training, then explanation evaluation.Source: §3 · p. 5

3.1 Step 1 - Understandability Score Collection

  • Feature ratings were collected for the Telco Customer Churn and Body Signals of Smoking datasets, with zero-variance and user-ID features removed before rating.The datasets contained 19 features and 7,043 instances, and 24 features and 55,692 instances, respectively; transferability to other domains is not guaranteed.Source: §3.1 · pp. 5–6
  • Participants rated half the features in their assigned dataset, and mean understandability scores were computed after attention-check failures were removed.Each participant rated 10 Telco features or 12 medical-dataset features; presentation was pseudo-randomized and counterbalanced.Source: §3.1 · p. 6

3.2 Step 2 - Feature Selection and Machine Learning Model Implementation

  • The data were split 70:30 across 50 seeds to assess performance variation, and subsequent feature selection and training used training data only.Mean, standard deviation, and normal-approximation confidence intervals were computed across the splits.Source: §3.2 · p. 6
  • Co-optimised selection retained the five most understandable features from an importance-based shortlist, while informative selection used the five most important features directly.Both settings used recursive feature elimination; the five-feature limit was intended to constrain explanation length and cognitive load while retaining sufficient information.Source: §3.2 · p. 7

Machine Learning Model Implementation

  • A tuned random-forest classifier was trained for each feature subset, and TreeSHAP extracted feature importance values for ten test instances.Hyperparameters were selected using five-fold cross-validation on the training set before retraining on the full training set.Source: Machine Learning Model Implementation · p. 7

3.3 Step 3 - Explanation Evaluation

  • Textual explanations stated the prediction and ranked supporting features, adding detracting features as a caveat when present.Numeric feature values were verbalized as above or below their mean.Source: §3.3 · p. 8

Individual Evaluation

  • In the individual evaluation, participants rated explanations using an adapted scale, with explanation type modeled separately by dataset and participant and prototype effects treated as random intercepts.The survey used a repeated-measures paired design; two items were removed because explanations were not rule-based and participants could not judge feature relevance against the full feature set.Source: Individual Evaluation · pp. 8–9

4 Results and Discussion

Feature understandability ratings were collected for two datasets, and repeated Monte Carlo splits were used to check whether split choice affected model performance. The reported rating ranges were limited, while cross-dataset comparisons lacked a shared baseline.

4.1 Feature Rating Collection

  • US ranged from 3.52–4.52 for Telco and 2.94–4.00 for medical, based on an average of 30.32 ratings per feature.Figure 4 shows feature-level mean US with standard errors; higher scores indicate greater understandability.Source: §4.1 · p. 10, Figure 4 · p. 10
  • Cross-dataset US comparisons are unreliable because participants saw only one dataset and had no shared baseline for interpreting the rating scale.The low rating for an apparently accessible feature such as ‘hearing (right)’ may relate to its measurement units; interviews could investigate rating decisions.Source: §4.1 · p. 10

Monte Carlo Resampling

  • Across 50 stratified 70:30 splits, performance varied little across seeds, so Seed 42 was retained for feature selection and modelling.The repeated-split procedure trained and evaluated a random forest on the full feature set; Table 1 reports means, standard deviations, and normal-approximation confidence intervals.Source: Monte Carlo Resampling · pp. 10–11, Table 1 · p. 11

Feature Selection

Co-optimised feature selection increased the understandability of selected features while classifier performance remained stable relative to informative-only selection. The selected sets favored more everyday features, especially in the medical dataset.

Feature Selection

  • Understandability rose by 0.20 and 0.25 US points, while classifier performance remained stable across settings and datasets.These changes represented 19.86% and 22.63% of the respective datasets’ total US ranges; the largest performance difference was approximately 2% in Telco recall, favoring the informative setting.Source: Feature Selection · p. 11, Table 2 · p. 12
  • The co-optimised setting retained nine Telco features and eight medical features, with two and three of the five most understandable features overlapping the informative setting, respectively.Table 3 ranks selected features by SHAP importance and marks features shared by both settings.Source: Feature Selection · pp. 11–12, Table 3 · p. 12
  • Co-optimised selection favored features considered more understandable at face value, including variables encountered in everyday life, especially in the medical dataset.The reported feature sets were selected using the elbow of the importance plot as the total cut-off.Source: Feature Selection · p. 12, Table 3 · p. 12

Explanation Generation

The study generated SHAP-based prototype explanations for two feature-selection settings across two datasets. Participants then rated explanations in either an individual or comparative evaluation.

Explanation Generation

  • Forty prototype explanations were generated across two settings and two datasets, with SHAP-based feature contributions used to explain individual predictions.The example explains a predicted customer switch using influential features, including Contract and Internet Service.Source: Explanation Generation · p. 12

4.3 Evaluation

  • The evaluation recruited 126 participants, who were assigned to individual or comparative explanation-rating tasks.Participants rated either two or five explanation pairs per dataset, respectively.Source: §4.3 · pp. 12–13
  • After exclusions, the individual evaluation comprised 236 data points from 59 participants, while the comparative evaluation comprised 300 data points from 60 participants.Non-English speakers and participants identified as response-time outliers were removed before analysis.Source: §4.3 · p. 13

Individual Scale-based Analysis

Co-optimised explanations received slightly higher average scores, but the individual evaluation found no statistically significant effect of explanation type. Participant differences accounted for substantial unexplained score variation.

Individual Scale-based Analysis

  • Co-optimised explanations scored higher on average by 1.03 points for Telco and 0.08 points for medical, but neither difference was statistically significant.The mixed-effects analysis modeled Total Score (range 10–50) with random intercepts for participants and question prototypes; Figure 5 shows score distributions across datasets and settings.Source: Individual Scale-based Analysis · pp. 13–14, Figure 5 · p. 13, Table 4 · p. 14
  • Participant differences explained 55.4% of unexplained variance in Telco scores and 79.7% in medical scores.The question prototype accounted for 6.7% and 0.9% of variance, respectively.Source: Individual Scale-based Analysis · p. 14

Comparative Analysis

Participants preferred co-optimised explanations significantly more often than chance in both datasets, although item-level preferences were mixed in the medical dataset. The results also show participant-level variation and leave the proposed domain explanation tentative.

Descriptive Statistics

  • Overall, co-optimised explanations had 70 more preferences in the telco dataset and 100 more in the medical dataset.At the question level, only Q6 favored the informative explanation in telco; Q5 and Q7 marginally favored it in medical.Source: Comparative Analysis · p. 15
  • The proposed domain explanation for dataset differences remains conjectural, and example presentation counts varied despite counterbalancing.The authors call for systematic confirmation of the possible role of domain familiarity or preferences for technical terms.Source: Comparative Analysis · p. 15

Statistical Analysis

  • 64.5% and 71.5% were the estimated probabilities of preferring the co-optimised explanation in the telco and medical datasets, respectively.Preference was significantly above chance in both datasets; the mixed-effects logistic regression results are reported in Table 6.Source: Comparative Analysis · p. 15, Table 6 · p. 16
  • Participant-level variability accounted for approximately one quarter of latent-scale variance in both datasets, while prototype variability was small in medical and zero in telco.The prototype random intercept was omitted from the telco model because its estimated variance was zero.Source: Comparative Analysis · pp. 15–16

5 Conclusion

The study combined feature understandability with informativeness during feature selection and evaluated explanations from random forests trained on the selected features. Understandability scores increased while model performance remained essentially unchanged, and comparative evaluations favored co-optimised explanations significantly above chance.

5 Conclusion

  • The study collected feature understandability scores across two datasets, proposed an understandability-aware feature-selection mechanism, and evaluated explanations from models trained on selected features.Source: §5 · p. 16
  • The random forest's performance remained essentially unchanged between settings while the mean understandability score increased.Individual explanation evaluations were directionally favorable to co-optimisation but did not show a statistically significant quality change.Source: §5 · pp. 16–17
  • Comparative evaluation found co-optimised explanations were considered more understandable significantly more often than chance.The authors note that the paired forced-choice comparison may be more sensitive than the evaluation scale, which was unvalidated and potentially noisier.Source: §5 · p. 17

6 Limitations and Future Work

The authors identify evaluation scale and sample limitations and propose broader testing, alternative feature-selection approaches, and richer measures of comprehension and explanation quality.

6 Limitations and Future Work

  • The current co-optimisation uses filter feature selection; wrapper or embedded methods remain to be tested for greater effectiveness.The filter approach maintained high accuracy and promoted understandable features at low computational cost.Source: §6 · p. 17
  • The study treats understandability as a spectrum, leaving discretisation into levels as an open alternative for examining explanation quality.Source: §6 · p. 17
  • Comparative evaluation measured subjective judgments of which explanation participants understood better, not objective comprehension.The authors suggest measuring objective comprehension in future work.Source: §6 · p. 17
  • Only a Random Forest was tested; the authors propose evaluating more complex models, including Gradient Boosting and TabNet.They suggest assessing whether these models can further minimise the accuracy-understandability trade-off.Source: §6 · p. 17
  • The evaluation found no significant individual-quality difference, and the authors recommend replication across domains and larger cohorts.The comparative result may reflect small sample size or overlap in selected features between settings.Source: §6 · p. 17
  • Evaluation used relatively small samples—236–300 observations from 59–60 participants per dataset—and the explanation-quality scale was not formally validated.The authors say this limits generalisability and that rating validity cannot be guaranteed; they propose psychometric validation of an explanation-quality scale.Source: §6 · p. 17
  • Qualitative interviews could identify further improvements to explanation structure and composition and clarify the impact of specialised terms.Source: §6 · p. 18

Appendix

The appendix includes the scale used for individual explanation evaluation and an example prompt for rating a feature's understandability.

Appendix

  • Table 7 presents the scale adapted from Vilone and Longo for individual explanation evaluation.Source: Table 7 · p. 18
  • Table 8 illustrates a feature-rating prompt using Credit Score, its value range, and a loan-approval example.Source: Table 8 · p. 19

Related papers

Loading 2610.16627v1…