PaperlayerView exact revision on arXiv

Source-linked AI summary

Why Is SHAP Not a Reliable Standalone Explanation Framework for Malware Detection?

Seyedreza Mohseni, Edward Raff, Manas Gaur

arXiv:2609.04626v1cs.CR

TL;DR

The paper asks whether SHAP can reliably explain static PE malware classifiers when feature groups are dependent. It formalizes interventional and conditional failure modes and tests them with fixed tree-based detectors, finding that SHAP attributions can change because of coalition and redundancy choices rather than model behavior. The paper therefore positions SHAP as a limited diagnostic requiring explicit distributional assumptions and domain validation.

  • Problem

    SHAP's formal guarantees do not by themselves establish that its attributions explain malware behavior rather than a selected coalition game.

  • Method

    The paper combines theoretical lemmas about dependent PE feature groups with controlled experiments using fixed LightGBM and XGBoost detectors across malware datasets.

  • Results

    Conditional SHAP dilutes used-feature credit across redundant proxies and credits unused features, while interventional SHAP evaluates synthetic off-manifold coalitions.

  • Takeaways & Limitations

    SHAP should be treated as a limited diagnostic rather than a standalone explanation of static PE malware classifiers.

Abstract

from arXiv · show

Machine learning is widely used for malware detection, but its decisions must be explained. An analyst needs to know whether a model has learned genuine malicious behavior or only dataset-specific patterns \cite{gaur2021semantics}. SHapley Additive exPlanations (SHAP) is the standard tool for this, backed by formal properties such as local accuracy, missingness, and consistency. We argue that these guarantees are insufficient for reliable malware interpretation. We claim SHAP explains a chosen feature-coalition game, not malware behavior in the data. That game is fixed only after the analyst selects the feature players, the missing feature rule, the background distribution, and the simplified input mapping. In static Portable Executable feature spaces, groups such as byte histograms, byte-entropy, strings, headers, sections, imports, and data-directories are not independent signals but are jointly shaped by file structure, packing, compiler behavior, and family conventions. We prove that this dependence makes conditional SHAP dilute a model's feature credit by a factor of $1/m$ across $m-1$ redundant features, attributes importance to features the model never uses, and even reverses the sign of an unused feature's attribution when the data distribution changes; interventional SHAP, meanwhile, queries off-manifold coalitions that no real executable would exhibit. Experiments on EMBER-2018, EMBER-2024, and BODMAS with fixed LightGBM and XGBoost detectors confirm these effects. We therefore position SHAP as a limited diagnostic that requires an explicitly stated data distribution and domain validation, not a standalone account of malware behavior.

I. INTRODUCTION

SHAP is widely used to explain malware classifiers, but its formal guarantees apply to a selected coalition game rather than necessarily to malware behavior. The paper argues that dependence among static PE feature groups creates distinct interventional and conditional attribution failures, supported by theoretical analysis and controlled experiments.

  • Motivation: SHAP assigns additive feature importance under formal properties, but those properties characterize a selected value function and input mapping.The analyst must choose the feature players, missing-feature rule, background distribution, and simplified input mapping.
  • Motivation: Static PE feature groups are dependent because they are jointly shaped by executable structure, packing, compiler behavior, family conventions, and collection effects.This dependence makes the choice of coalition game consequential for malware interpretation.
  • Contributions: Interventional SHAP evaluates synthetic off-manifold coalitions, while conditional SHAP can produce redundancy dilution, proxy crediting, and sign instability.The paper formalizes these as separate failure modes under dependent PE feature groups.
  • Contributions: Conditional SHAP dilutes a used feature's attribution by 1/m across m−1 redundant proxies and can assign importance to features the classifier never uses.The paper also identifies attribution sign reversal for unused features across valid dataset distributions.
  • Empirical demonstration: Experiments on EMBER-2018, EMBER-2024, and BODMAS measure dependence, while fixed LightGBM and XGBoost detectors show attribution migration after redundant features are injected.For EMBER-2018, the observed migration agrees closely with the 1/m prediction while predictive performance remains unchanged.

II. BACKGROUND OF SHAP

SHAP represents features as players in a cooperative game and assigns additive contributions under properties such as local accuracy, missingness, and consistency. In malware analysis, the central unresolved choices concern the value function, missing-feature mapping, and treatment of feature dependence.

  • SHAP formulation: SHAP explains a model prediction by treating input features as game players and assigning each a contribution to the final prediction.The explanation uses a value function defined over feature coalitions.
  • SHAP formulation: Additive SHAP explanations use a simplified binary input and attribution values combined with a base value.The original SHAP framework defines additive attribution methods as linear functions of binary variables.
  • Formal properties: Missingness gives absent features zero credit, while consistency requires an attribution not to decrease when that feature contributes more after a model change.These properties support uniqueness for a chosen simplified input mapping.
  • Limitations for malware analysis: For malware analysis, the key questions are how to define the coalition value, map missing features back to real inputs, and handle conditional feature relationships.Different choices of conditional expectations, interventional sampling, or independence approximation can change SHAP values.

III. THEORETICAL EXPLANATION

The theoretical analysis shows that interventional SHAP breaks dependence among PE feature groups by constructing product-distributed hybrid samples. These samples can differ from real executable distributions, so the resulting attributions may reflect classifier behavior on artificial coalitions.

  • Interventional SHAP: Interventional SHAP constructs a coalition by combining selected feature groups with independently sampled missing groups.The resulting synthetic sample takes one group block from the explained file and another from an independent background draw.
  • Interventional SHAP: When PE feature groups are dependent, the synthetic coalition distribution is the product of marginals rather than the real joint data distribution.The mismatch follows because positive mutual information implies the joint distribution differs from the product distribution.
  • Value functions: The interventional value function averages predictions over marginal draws of missing groups, whereas the data-consistent alternative conditions those groups on the observed groups.These value functions encode different assumptions about how missing feature groups relate to the explained file.
  • Interventional SHAP: Interventional SHAP therefore evaluates classifiers on hybrid samples that may not match realistic benign or malicious PE files.The paper identifies these as synthetic, off-manifold coalitions produced by breaking dependencies within an executable.

B. Failure of Conditional SHAP

Conditional SHAP respects the data distribution but can fail when PE features are dependent, redundant, or statistically informative about one another. It can dilute credit for used features, attribute importance to unused proxies, and reverse unused-feature signs across dataset distributions.

  • Conditional SHAP setup: Conditional SHAP uses conditional expectations, making the conditional distribution of missing features the core object determining the explanation.This avoids some artificial combinations but makes explanations depend on highly correlated PE variables and conditional-density estimation.
  • Redundancy failure: 1/m: adding m −1 redundant features divides the model-used feature’s attribution by m while giving equal credit to redundant features.The classifier and underlying malware signal remain unchanged, so attribution depends on redundant feature representation.
  • Proxy attribution failure: Conditional SHAP assigns nonzero attribution to a feature the classifier never uses when that feature is statistically informative about a model-used feature.The attribution arises from dataset dependence rather than direct functional dependence on the proxy feature.
  • Proxy attribution failure: The proxy attribution can be weaker under mere statistical dependence than under exact redundancy, while exact redundancy is the worst case within this failure mode.The m = 2 exact-redundancy case yields equal sharing, whereas a merely informative proxy still receives potentially weaker nonzero credit.
  • Sign instability: Conditional SHAP can assign opposite signs to the same unused feature across valid malware dataset distributions when dependence with the model-used feature changes.The sign is determined by whether conditioning on the unused feature increases or decreases the expected model response.
  • Contrast with intervention: Interventional SHAP instead evaluates synthetic group coalitions, so conditional and interventional formulations expose different reliability risks.Interventional sampling is outside the main conditional-failure results here, but the section’s candidates identify its synthetic-coalition contrast.

A. Experimental environments and features

The experiments use three malware datasets and a static PE feature space restricted to common, meaningful executable features. Non-feature metadata such as hashes is excluded, and the implementation uses an NVIDIA A100 GPU with reproducible code.

  • Datasets and features: Experiments use EMBER-2018, EMBER-2024, and BODMAS with static PE features common and meaningful for malware analysis.Hashes and other metadata that do not directly describe executable structure are excluded.
  • Reproducibility: The implementation uses Google Colab with an NVIDIA A100 GPU, and code is available in the Malware-SHAP repository.

B. Experiments

The experiments test whether PE feature dependence and redundancy undermine SHAP explanations while holding baseline detectors fixed. They measure group dependence and add exact feature duplicates to observe whether model performance stays stable as SHAP credit moves.

  • Experimental design: The baseline detectors are fixed so attribution changes can be assessed as effects of explanation settings rather than retraining differences.LightGBM and XGBoost use five seeds and specified tree-size and learning-rate settings.
  • Dependencies: The dependency experiment measures pairwise group-level mean absolute correlation to test whether malware feature groups behave like independent players.It targets links among byte content, entropy, strings, headers, imports, sections, and data-directories arising from shared executable structure.
  • Feature Redundancy: The redundancy experiment adds exact duplicates Cj = Xj and checks whether model performance remains stable while SHAP credit moves from an original feature to its copy.This experiment is designed to verify the redundancy lemma.

C. Baseline

LightGBM and XGBoost are strong across EMBER-2018, EMBER-2024, and BODMAS, establishing accurate fixed detectors for evaluating SHAP attribution behavior. Subsequent attribution changes are therefore attributed to explanation settings rather than poor classifier quality.

  • Baseline performance: All six detectors are uniformly strong across the three datasets, with bootstrap means and 95% confidence intervals over five seeds.The reported baseline uses fixed LightGBM and XGBoost detectors.
  • LightGBM: LightGBM reaches AUC 0.990 on EMBER-2018, 0.964 on EMBER-2024, and 0.996 on BODMAS.The corresponding precisions are 0.991, 0.972, and 0.995, respectively.
  • XGBoost: XGBoost reaches AUC 0.990 on EMBER-2018, 0.953 on EMBER-2024, and 0.996 on BODMAS.The corresponding precisions are 0.943, 0.962, and 0.994, respectively.

D. Dependency

Static PE feature groups are dependent rather than independent players: EMBER-2018 and BODMAS exhibit redundancy, while EMBER-2024 exhibits structural synergy. These opposing dependence patterns undermine conditional and interventional SHAP in different ways.

  • Group-level dependence: EMBER-2018 and BODMAS contain a dominant redundant cluster linking histogram, strings, and byte-entropy groups.The correlation maps also show weaker associations extending to section and header groups.
  • Group-level dependence: EMBER-2024 instead shows synergistic dependence among structural groups, including data-directories, header, authenticode, and imports.Positive interaction information indicates that groups jointly determine predictions beyond their individual contributions.
  • Implications for SHAP: Redundancy violates the independent-player assumption underlying conditional SHAP, while synergy undermines interventional SHAP.Conditional SHAP can dilute credit and transfer it to proxies; interventional SHAP can destroy informative joint configurations through independent background draws.

E. Proxy Redundancy

Injecting redundant proxies shifts SHAP credit away from original model-used features without changing predictions. LightGBM follows the predicted 1/m pattern smoothly, whereas XGBoost shows the same failure with greater instability.

  • LightGBM: 40–50%: LightGBM’s original-feature attribution loss converges near the m = 2 prediction of 1 − 1/m = 0.50.The measured Dg ≈0.495 closely matches the closed-form prediction.
  • LightGBM: Dg ≈0.495: the LightGBM pairwise proxy experiment nearly matches the predicted 50% original-attribution loss.Displaced credit reappears on proxy copies that the classifier does not functionally require.
  • LightGBM: 44.6%: LightGBM’s mean original attribution drop across the first 100 settings, while predictive performance remains unchanged.The result isolates the effect to SHAP allocation rather than detector predictions.
  • XGBoost: 37.9%: XGBoost’s overall mean original attribution drop across the first 100 settings, with a median of 28.9%.Its curves are highly non-monotonic; some groups drop by more than 80–90%, while others briefly show negative drops.
  • XGBoost: XGBoost exhibits the same proxy-credit transfer as LightGBM but in a markedly less stable form.Greedy split selection and tie-breaking among exact duplicates redistribute attribution unevenly.

V. CONCLUSION

The paper concludes that SHAP’s formal guarantees apply to a selected coalition game, not necessarily to malicious behavior in static PE data. Dependence among PE feature groups makes the game choice materially affect attribution, so SHAP should be used as a constrained diagnostic with explicit distributional assumptions and domain validation.

  • Conclusion: SHAP attributions are formally valid for a fixed value function and simplified input mapping, but their guarantees describe the chosen coalition game.They do not by themselves establish that the explanation represents malicious behavior.
  • Conclusion: Conditional SHAP can dilute model-used feature credit by 1/m, credit unused features, and reverse an unused feature’s sign as the data distribution changes.Interventional SHAP can instead query off-manifold coalitions that no real executable would exhibit.
  • Conclusion: SHAP should be treated as a constrained diagnostic requiring an explicitly stated data distribution and domain validation, not a standalone explanation.The experiments support this conclusion while separating explanation instability from model quality.
Loading 2609.04626v1…