PaperlayerView exact revision on arXiv

Source-linked AI summary

Does Explainability Survive Data Drift?

Samuel Ozechi

arXiv:2610.05379v1cs.LGcs.AI

TL;DR

Explanation fidelity is less routinely monitored than detection performance, leaving open whether explanations remain faithful under input drift. The paper evaluates ExIFFI explanations with structural measures and controlled interventions, finding that prior explanations retain decision relevance but are less faithful than newly generated ones. These findings support monitoring fidelity separately and treating explanations as tied to the model that produced them.

  • Problem

    Whether explanations remain faithful when input distributions shift despite a stable feature–target relationship is unclear, although detection performance and explanation quality are distinct.

    Source: §1 · pp. 1–2

  • Method

    The study evaluates ExIFFI explanations’ structure and decision-level fidelity by replacing top-k attributed features with training-period medians and measuring model-score changes.

    Source: §4.3 · p. 4

  • Results

    Prior explanations retain substantial usefulness under data drift but are less faithful to current decisions than freshly generated explanations.

    Source: §6 · p. 8, §8 · p. 10

  • Takeaways & Limitations

    Explanation fidelity warrants its own monitoring because detection metrics do not reveal fidelity loss and structural stability does not guarantee functional fidelity.

    Source: §8 · p. 9

    The evidence comes from one fraud dataset, one Isolation Forest architecture, and ExIFFI, while fidelity is assessed using relative score changes after top-k median substitution.

    Source: §7 · p. 9

Abstract

from arXiv · show

Model performance monitoring is a standard practice in machine learning deployments. Detection performance is tracked continuously, and model decay is expected as the relationship between the feature and target variables degrades, a phenomenon known as concept drift. Explanation fidelity, however, is rarely monitored with the same discipline, even in domains such as financial systems, healthcare, and other regulated environments where explanations are required for governance purposes. This paper investigates whether explanations can decay under data drift, even when the feature-target relationship remains stable, and whether explanations produced before drift occurs remain faithful to the decisions of the model that replaces them. Using the IEEE-CIS Transaction Fraud Detection dataset, we find statistically significant covariate shift but no statistically significant evidence of concept drift under the implemented conditional-drift tests, thereby providing an empirical setting in which input distributional change can be studied separately from detectable changes in the feature-target relationship. Local explanations are generated with ExIFFI and evaluated at three levels: path validity, structural behaviour, and fidelity under controlled intervention. Results show that while prior explanations retain substantial decision relevance to a retrained model, they are consistently less faithful than newly generated explanations, with no evidence of a systematically widening gap across the evaluated windows. The study shows that explanation fidelity requires its own monitoring, that structural stability of explanations does not guarantee functional fidelity, and that explanations should be treated as artifacts tied to the model that produced them.

1. Introduction

Data drift changes the input distribution, while the feature–target relationship may remain stable. The paper asks whether explanation fidelity survives that shift, since detection performance and explanation quality are distinct properties.

1. Introduction

  • Data drift is a change in the input-feature distribution P(X), and the study asks whether explanations remain faithful when that distribution shifts but P(Y|X) is stable.The motivating concern is that explanations faithful under the training distribution may silently lose fidelity under drift.Source: §1 · pp. 1–2
  • Explanation fidelity may degrade even when a detector continues to perform acceptably, so detection and explanation quality require separate measurement.The paper distinguishes explanation stability—the similarity of explanations across windows—from fidelity to the model’s decision behaviour.Source: §1 · pp. 1–2
  • Explanations can help justify why an observation was flagged, making their fidelity relevant when anomaly-detection decisions need explanation.Source: §1 · p. 2

2. Related Work and Contributions

Prior work developed interpretable isolation-based detectors and methods for evaluating explanation fidelity, but whether fidelity persists under input distribution change is less established. This study investigates that gap and distinguishes detection performance, explanation structure, and decision-level fidelity.

2. Related Work and Contributions

  • ExIFFI derives local and global feature importance from isolation-model decision paths, while perturbation-based evaluation measures how model outputs change when important features are altered [2], [8],,.Source: §2 · p. 2
  • Whether explanations faithful under one input distribution remain faithful under another is less directly established in work focused on static explanation settings and drift adaptation,.Source: §2 · p. 2
  • The study’s contributions include a setting with substantial covariate shift but no statistically detectable conditional concept drift, a decision-level fidelity framework, and analysis of ExIFFI explanations and their transferability.It treats model performance, explanation structure, and fidelity as distinct properties requiring independent monitoring.Source: §2 · p. 2

3. Research Questions

The research questions examine detection performance, explanation structure, decision-level fidelity, and whether explanations transfer to an updated model. Together, they separate descriptive changes in explanations from whether those explanations still reflect model decisions.

RQ3 (Decision-Level Fidelity)

  • RQ3 asks whether ExIFFI-selected features account for model decisions and whether their fidelity changes across future windows.The question treats fidelity loss, if present, as distinct from detection performance.Source: §3 · p. 3

RQ4 (Explanation Transferability)

  • RQ4 asks whether explanations from a previous model remain faithful to an updated model’s decision logic.The question concerns whether explanations transfer or need regeneration after updates.Source: §3 · p. 3

RQ1 (Detection Performance)

  • RQ1 tracks how a static Isolation Forest’s detection performance changes across future windows as the input distribution drifts.Source: §3 · p. 3

RQ2 (Explanation Structure)

  • RQ2 tracks changes in local-explanation structure while distinguishing those descriptive changes from evidence of decision-level fidelity.Source: §3 · p. 3

4. Methods

The methods use chronologically ordered fraud data to test drift and compare static with retrained Isolation Forests. ExIFFI explanations are evaluated through structural measures and controlled feature interventions.

4. Methods

  • The earliest 60% of transactions form the training period, and the remaining 40% is split into four chronological evaluation windows; preprocessing is fitted only on training data.Each evaluation window contains 59,054 transactions, with fraud prevalence varying across windows.Source: §4 · p. 3

4.1. Drift Analyses

  • Drift analyses compare training and future-window feature distributions and test conditional drift using classifier discrimination, permutations, and BH-FDR correction.A complementary fraud-classifier test compares training and future-window ROC-AUC against a permutation-based null, also with BH-FDR correction.Source: §4.1 · p. 3

4.2. Model Conditions

  • A fixed Isolation Forest trained on the initial period is compared with four forests trained separately on future windows, isolating adaptation from drift.Both conditions use the same hyperparameters and preprocessing; each retrained model is evaluated across future windows.Source: §4.2 · pp. 3–4

4.3. Explanation Structure and Fidelity

  • ExIFFI derives feature contributions from isolation-tree paths; fidelity is tested by replacing top-k attributed features with training-period medians and measuring the score change.Explanation mass, entropy, and non-zero features describe structure but do not themselves establish fidelity; k=5 is used as a mid-range reported value.Source: §4.3 · p. 4

5. Results

Across the evaluated windows, explanations from earlier models remain decision-relevant but are less faithful than current-model explanations. The study also finds covariate shift without statistically detectable concept drift, alongside changes in detection performance, explanation structure, and fidelity.

5.1. Distributional Changes in the Dataset

  • Across windows, 53.7% to 70.2% of features show significant distributional change, while the implemented tests find no statistically significant evidence of concept drift.The conditional-distribution test and the BH-FDR-corrected performance-drop test both indicate that concept drift is not statistically detectable in this dataset.Source: Table 5.1.2 · p. 4, Table 5.1.1 · p. 4, §5.1 · pp. 4–5

5.2. Detection Benchmark

  • Detection declines across W1–W4, while retraining mitigates but does not halt the decline; F1 falls 24% to 30% and PR-AUC falls 45% to 49%.For the static baseline, F1 falls from 0.27 to 0.20; retrained models retain higher performance across evaluation windows.Source: §5.2 · p. 5, Figure 5.2 · p. 5
  • Parallel-window training does not optimize F1 consistently, while the W4-trained model achieves superior PR-AUC across all windows.Off-diagonal models outperform in F1 in two windows.Source: §5.2 · p. 6

5.3. Explanation Structure

  • The static model’s explanation concentration declines slightly from W1 to W4, while attribution entropy and effective feature count rise.Top-5 mass drops from 0.18 to 0.17, entropy rises from 5.21 to 5.3 nats, and effective feature count expands from 206 to 222.Source: Figure 5.3 · p. 6, §5.3 · p. 6

5.4. Explanation Fidelity

  • ExIFFI-selected feature interventions produce a larger mean relative score change than random interventions at every tested k, with a 23.8× ratio at k=1.The ratio narrows to 12.2× at k=20.Source: Figure 5.4 · p. 6, §5.4 · pp. 6–7

5.5. Temporal Fidelity Trajectories

  • Within-model fidelity is lower at W4 than W1 for every model, although later-window retrained models show non-monotonic intermediate trajectories.The static baseline and W1-retrained model decline monotonically; W2-, W3-, and W4-retrained models show intermediate recoveries before declining.Source: Table 5.5.1 · p. 7, §5.5 · p. 7
  • Prior explanations remain relevant to current-model decisions, but their fidelity is roughly 25% to 30% below the current model’s own explanations.The static explanation’s fidelity remains between 0.0270 and 0.0303 across windows, and its deficit is stable rather than widening with drift.Source: Table 5.5.2 · p. 7, §5.5 · pp. 7–8

6. Discussions

Across the evaluated windows, detection performance and explanation fidelity both decline, while explanation structure remains comparatively stable. Prior explanations retain decision relevance but are less faithful than freshly generated explanations, with no evidence that this gap widens over time.

6. Discussions

  • The discussion’s central finding is that input distributional shift accompanies degraded fidelity in static-model explanations, while explanations generated before drift remain less faithful to current decisions.The results distinguish detectable input-distribution change from detectable change in the feature-target relationship.Source: §6 · p. 8

6.1. Performance Decline Under Data Drift

  • All evaluated models decline monotonically; between W1 and W4, the static baseline’s F1 falls by 24% to 30% and PR-AUC by 45% to 49%.The decline is steepest between W2 and W3; the analysis finds no statistically detectable conditional concept drift, and static and retrained models decline at comparable rates despite different training sample sizes.Source: §6.1 · p. 8

6.2. Stability of Explanation Structure

  • Explanation structure shifts modestly: top-5 mass decreases by about one percentage point, entropy rises by 0.09 nat, and adjacent-window rank correlations remain 0.81–0.83 for the static model.The persistent ranking does not guarantee that attributions remain faithful to model decisions.Source: §6.2 · p. 8

6.3. Temporal Degradation of Fidelity

  • Under top-k median substitution, ExIFFI features produce a 23-times-larger mean relative score change than random features at k=1, while static-model fidelity falls 9.1% from W1 to W4.The intervention advantage holds across k, while the fidelity decline occurs across the evaluated temporal windows.Source: §6.3 · pp. 8–9

6.4. Implication for Explanation Transferability

  • Static explanations remain relevant to current decisions, but their fidelity stays below current-model explanations by roughly 25 to 30 percent, with a stable 0.72–0.77 ratio across windows.The evaluated windows provide no evidence that the transferability gap systematically widens or narrows.Source: §6.4 · p. 9

6.5. Implication for Anomaly Detection Systems

  • Detection monitoring alone cannot establish explanation trustworthiness; explanations are not interchangeable, and regenerating them after model updates is the most defensible practice when accurate current decision descriptions are required.Transferred explanations can remain informative when used diagnostically or as a secondary signal alongside detection metrics.Source: §6.5 · p. 9

7. Limitations

The findings are bounded by the study’s dataset, detector, attribution method, intervention choice, and finite evaluation horizon.

7. Limitations

  • The experiments use one fraud dataset, one Isolation Forest architecture, and ExIFFI, so the observed trajectories are not universal claims across detectors or drift environments.Faithfulness is measured through relative score changes after top-k median substitution, and transferability-gap claims are limited to four evaluation windows.Source: §7 · p. 9

8. Conclusion

Under data drift, explanation fidelity declines even though prior explanations retain substantial usefulness; regenerating explanations after a model update recovers part of the lost fidelity.

8. Conclusion

  • Under data drift, explanation fidelity declines, and regenerating explanations after a model update recovers part of the lost fidelity.Original explanations nevertheless retain substantial usefulness under drift.Source: §8 · pp. 9–10
  • Fidelity warrants monitoring separately because detection metrics do not reveal its loss, and structural stability does not guarantee functional fidelity.Source: §8 · p. 9
  • The intervention-based metric provides a decision-level test validated on IEEE-CIS to produce a stable, interpretable signal across windows.Source: §8 · pp. 9–10
Loading 2610.05379v1…