PaperlayerView exact revision on arXiv

Source-linked AI summary

Generating Hierarchical Explanations on Text Classification via Feature Interaction Detection

Hanjie Chen, Guangtao Zheng, Yangfeng Ji

arXiv:2004.02015v3cs.CL

TL;DR

Word- and phrase-level explanations often leave unclear how features interact in a text classifier’s prediction. HEDGE builds hierarchical explanations by detecting feature interactions, and the paper reports effective explanations across automatic and human evaluations. Its interaction approximation assumes that words or phrases usually interact strongly with sentence neighbors.

  • Problem

    Existing NLP explanations often select important words or phrases but do not explain how those features interact, making it harder to connect explanations to model predictions.

    Source: §1 · pp. 1–2

  • Method

    HEDGE recursively detects the weakest feature interactions to divide text into hierarchical spans and scores their contributions to the prediction.

    Source: §1 · p. 2

  • Results

    Across sentiment classification with LSTM, CNN, and BERT on SST and IMDB, HEDGE is reported effective in both automatic and human evaluations.

    Source: §1 · p. 2, §5 · p. 8

  • Takeaways & Limitations

    The hierarchy depicts how features combine at different granularities and can identify an important phrase, such as “waste of good.”

    Source: §1 · p. 2

    The interaction-score approximation assumes words or phrases usually interact strongly with sentence neighbors and considers only m neighboring spans.

    Source: §3.2 · p. 4

Abstract

from arXiv · show

Generating explanations for neural networks has become crucial for their applications in real-world with respect to reliability and trustworthiness. In natural language processing, existing methods usually provide important features which are words or phrases selected from an input text as an explanation, but ignore the interactions between them. It poses challenges for humans to interpret an explanation and connect it to model prediction. In this work, we build hierarchical explanations by detecting feature interactions. Such explanations visualize how words and phrases are combined at different levels of the hierarchy, which can help users understand the decision-making of black-box models. The proposed method is evaluated with three neural text classifiers (LSTM, CNN, and BERT) on two benchmark datasets, via both automatic and human evaluations. Experiments show the effectiveness of the proposed method in providing explanations that are both faithful to models and interpretable to humans.

1 Introduction

Neural text classifiers are difficult to trust when their decision-making is not understood, and word- or phrase-level explanations leave feature interactions unaddressed. HEDGE builds hierarchical explanations by detecting those interactions and is evaluated against other methods using automatic and human evaluations.

1 Introduction

  • Existing local explanations quantify individual word or phrase contributions, but do not show how those features interact and combine to produce a prediction.LIME highlights “waste,” while Contextual Decomposition identifies “waste of.”Source: §1 · pp. 1–2
  • HEDGE recursively detects the weakest feature interactions to divide text into a hierarchy, showing how “good” is dominated and identifying “waste of good” as an important phrase.The hierarchy’s span scores also provide phrase-level explanations.Source: §1 · p. 2
  • The work contributes a top-down, model-agnostic hierarchy, a feature-contribution scoring function, and comparisons using automatic and human evaluations.The experiments use LSTM, CNN, and BERT classifiers on SST and IMDB; the reported comparisons find HEDGE more faithful and human-understandable.Source: §1 · p. 2

2 Related Work

Prior model-agnostic explanation methods largely focus on individual words, while hierarchical methods either require exhaustive search or rely on existing structures. HEDGE instead detects interactions to construct a hierarchy from the text.

2 Related Work

  • Architecture-specific interpretation methods may require deep knowledge of neural networks or work only with particular models, whereas model-agnostic methods use predictions and apply to black-box models.The cited examples include contextual decomposition, gradient-based methods, and attention-based methods; model-agnostic examples include LIME and Lundberg and Lee’s method.Source: §2 · p. 2

2.1 Model-Agnostic Explanations

  • Most model-agnostic explanations assess word-level importance; HEDGE’s feature-importance function is reported to outperform Shapley-based methods in selecting important words.Prior examples include Leave-one-out, LIME, and Shapley variants such as SampleShapley, KernelSHAP, and L/C-Shapley.Source: §2.1 · pp. 2–3

2.2 Hierarchical Explanations

  • Prior hierarchical approaches use exhaustive interaction search or aggregate features with contextual-decomposition scores, while HEDGE constructs structure top-down from detected interactions without external structural information.Lundberg et al. (2018) and Chen and Jordan (2019) require a given hierarchy; the paper reports HEDGE’s top-down approach as more effective and efficient in experiments.Source: §2.2 · p. 3

3 Method

HEDGE builds hierarchical explanations by recursively splitting text spans at weak interaction points, then scores the resulting spans’ contributions to the predicted label. It detects interactions using a context-aware Shapley interaction score and measures feature importance by distance from the prediction boundary.

3.1 Generating Hierarchical Explanations

  • HEDGE starts with the full text, repeatedly splits a selected span into smaller spans, and records each partition and span contribution to form a hierarchy.The bottom-up alternative merges spans with the strongest interaction, but performs worse than the top-down algorithm.Source: §3.1 · pp. 3–4
  • For each span, the inner optimization selects its weakest interaction point as the split, while the outer optimization selects which current span to split.The outer optimization enumerates the spans in the current partition.Source: §3.1 · p. 4

3.2 Detecting Feature Interaction

  • HEDGE scores two candidate child spans with a Shapley interaction index that accounts for the other spans in the new partition.The score uses model-output probabilities for the predicted label, with missing words replaced by <pad> in the experiments.Source: §3.2 · p. 4
  • Restricting interaction calculations to m neighboring spans reduces computational complexity to polynomial under the assumption that strong interactions usually occur between neighbors.The experiments set m = 2; increasing m can improve performance at increased computational cost.Source: §3.2 · p. 4

3.3 Quantifying Feature Importance

  • The feature-importance score measures a span’s predicted-label output relative to the highest output for any other class, representing distance from the prediction boundary.The paper reports that this score helps HEDGE outperform competitive baselines in identifying important features.Source: §3.3 · p. 5

4 Experiments

Experiments evaluate HEDGE on SST and IMDB with LSTM, CNN, and BERT using automatic and human evaluations. The reported results favor HEDGE on feature-importance and human-coherence measures, while qualitative examples illustrate how it represents feature interactions.

4.1 Setup

  • The evaluation uses SST-2 and IMDB, and tests LSTM, CNN, and pretrained BERT; BERT achieves higher classification accuracy than CNN and LSTM on both datasets.For IMDB, the training and test sets each contain 25,000 examples; the SST-2 train/dev/test sets contain 6,920/872/1,821 examples.Source: §4.1 · p. 5, Table 1 · p. 5

4.2 Quantitative Evaluation

  • Automatic evaluation uses AOPC and log-odds for local fidelity, plus cohesion-score to assess word interactions within a text span.Cohesion-score compares predicted-label probabilities before and after shuffling words around an important span; higher scores are better.Source: §4.2 · pp. 5–6

4.2.1 Results

  • HEDGE achieves the best AOPC and log-odds performance across the evaluated models and datasets; the IMDB evaluation uses 2,000 randomly selected samples.Table 2 reports results with k = r = 20; additional results vary these parameters.Source: Table 2 · p. 6, §4.2.1 · p. 6
  • HEDGE scores higher than ACD on cohesion with LSTM on both datasets, while BERT’s cohesion scores significantly exceed those of LSTM and CNN.The authors interpret the latter as indicating greater BERT sensitivity to perturbations on important phrases and use of context information.Source: Table 3 · p. 7, §4.2.1 · p. 7
  • HEDGE has O(n) computational complexity, compared with polynomial complexity for Shapley-based baselines such as SampleShapley and L/C-Shapley.SampleShapley performs well with 100 samples, but its reported computational complexity is 200 times HEDGE’s.Source: §4.2.1 · p. 7

4.3 Qualitative Analysis

  • In a negative review misclassified as positive by LSTM, HEDGE captures sentiment interactions that ACD misses, helping explain the prediction.The example describes HEDGE capturing the interaction in “bravura exercise” that flips the polarity of “in emptiness” to positive.Source: §4.3 · p. 7, Figure 2 · p. 7
  • On a positive review, HEDGE depicts BERT capturing “not a bad” while LSTM misses the interaction between “not” and “bad” and predicts negative.The example contrasts BERT’s correct positive prediction with LSTM’s incorrect negative prediction.Source: §4.3 · pp. 7–8, Figure 3 · p. 8

4.4.1 Results

  • Human annotators guess model predictions from explanations, and HEDGE achieves higher coherence than other LSTM baselines on IMDB.Across models, the authors report relatively high coherence for HEDGE, but BERT has lower coherence despite higher prediction accuracy, suggesting a potential accuracy–interpretability tradeoff.Source: §4.4.1 · p. 8, §4.4 · p. 8

5 Conclusion

The paper presents HEDGE as a model-agnostic method for building hierarchical interpretations through feature-interaction detection. It evaluates the method against competitive baselines using automatic and human evaluations.

5 Conclusion

  • HEDGE builds model-agnostic hierarchical interpretations by detecting feature interactions.The paper reports that its superiority was supported by both automatic and human evaluations.Source: §5 · p. 8
  • The experiments focus on sentiment classification with three neural models and two benchmark datasets.The conclusion identifies LSTM, CNN, and BERT as the models evaluated.Source: §5 · p. 8
  • Table 4 reports human evaluation of interpretation methods with LSTM on IMDB, while Table 5 evaluates HEDGE across models on IMDB.Source: Table 4 · p. 9, Table 5 · p. 9

A Comparison between Top-down and Bottom-up Approaches

For the example sentence “a waste of good performance,” the top-down approach captures an important phrase at an intermediate layer, while bottom-up clustering captures an interaction only at the top layer.

A Comparison between Top-down and Bottom-up Approaches

  • Top-down segmentation captures the phrase “waste of good” at an intermediate layer, whereas bottom-up clustering captures the interaction between “waste” and “good” only at the top layer.The paper reports that “waste” flips the polarity of “of good” to negative, matching the model’s negative prediction.Source: A Comparison between Top-down and… · p. 11
  • The paper reports that top-down segmentation performs better at capturing feature interactions than bottom-up clustering.It attributes this to larger interaction-score calculation errors at the bottom layer, which can propagate and accumulate during clustering.Source: A Comparison between Top-down and… · p. 11
  • Figure 4 uses red for negative sentiment and green for positive sentiment in its hierarchical interpretations.Source: Figure 4 · p. 11

B Results of AOPCs and log-odds changing with different k and r

The appendix figures present AOPC and log-odds results across three models and two datasets as k and r vary; the supplied captions identify the measures and settings but do not state comparative outcomes.

B Results of AOPCs and log-odds changing with different k and r

C Visualization of Hierarchical Interpretations

The visualizations contrast BERT’s correct prediction with LSTM’s incorrect prediction on positive SST reviews, and include an ACD visualization for LSTM’s incorrect prediction.

C Visualization of Hierarchical Interpretations

D Human Evaluation Interface

Figure 17 shows Amazon Mechanical Turk interfaces used to ask annotators to guess the model’s prediction from different explanations.

Loading 2004.02015v3…