Source-linked AI summary
Understanding Convolutional Neural Networks for Text Classification
Alon Jacovi, Oren Sar Shalom, Yoav Goldberg
TL;DR
The paper asks how CNNs process and classify text when vision-based interpretations do not directly apply to discrete sequences. It analyzes convolutional filters and global max-pooling, finding heterogeneous and suppressive filter behavior alongside pooling-based thresholding. These findings support concrete explanations of both filters and individual predictions.
Problem
CNNs for discrete text sequences remain difficult to interpret, while existing vision-based techniques do not trivially transfer and neural NLP interpretability remains under-explored.
Method
The paper analyzes filter activations, max-pooling behavior, ngram patterns, and filter class identities to derive model- and prediction-level explanations.
Results
Max-pooling thresholds irrelevant responses, filters capture multiple ngram families and negative cues, and 40% of pooled ngrams can be dropped on average without performance loss.
Takeaways & Limitations
The findings enable filter summaries and prediction explanations that focus on informative ngrams while accounting for negative cues.
Takeaways & Limitations
Filter class purity depends on the model and dataset, and higher purity can reduce coverage and discard more ngrams.
Abstract
from arXiv · showhide
We present an analysis into the inner workings of Convolutional Neural Networks (CNNs) for processing text. CNNs used for computer vision can be interpreted by projecting filters into image space, but for discrete sequence inputs CNNs remain a mystery. We aim to understand the method by which the networks process and classify text. We examine common hypotheses to this problem: that filters, accompanied by global max-pooling, serve as ngram detectors. We show that filters may capture several different semantic classes of ngrams by using different activation patterns, and that global max-pooling induces behavior which separates important ngrams from the rest. Finally, we show practical use cases derived from our findings in the form of model interpretability (explaining a trained model by deriving a concrete identity for each filter, bridging the gap between visualization tools in vision tasks and NLP) and prediction interpretability (explaining predictions). Code implementation is available online at github.com/sayaendo/interpreting-cnn-for-text.
1 Introduction
The paper investigates how CNNs process and classify discrete text, where vision-based interpretability intuitions do not directly transfer. It refines the ngram-detector view by showing thresholding, heterogeneous and suppressive filters, and practical interpretability applications.
- Motivation: CNN interpretability for discrete NLP sequences remains under-explored because computer-vision techniques assume continuous image inputs.The paper also cautions that pooling’s role in text may differ from deformation invariance in vision.
- Common assumptions: The common account treats convolutional filters as detectors for closely related ngram families and max-pooling as selecting relevant ngrams.The paper examines and refines both assumptions.
- Findings: Max-pooling induces thresholding, and an experiment drops 40% of pooled ngrams on average with no performance loss.Values below a threshold are treated as irrelevant to prediction.
- Findings: A single filter can detect multiple distinct ngram families rather than specializing in one homogeneous family.Filters use different activation patterns to give different linguistic patterns similar activation strengths.
- Findings: Filters can assign negative values that suppress related negated ngrams, and selected nonnatural ngrams can produce unusually large activations.These behaviors qualify the simple view of filters as positive ngram detectors.
- Applications: The findings support model-level filter summaries and prediction explanations that focus on informative ngrams and negative cues.The paper uses these findings to improve both forms of interpretability.
2 Background: 1D Text Convolutions
The studied text CNN embeds words, applies sliding-window convolutions, pools each filter’s strongest ngram response, and classifies the resulting representation. Parallel convolution layers can support multiple window sizes.
- Input representation: Each input word becomes a d-dimensional embedding, forming a d×n matrix for the convolutional layer.The model processes the embedded sequence rather than discrete symbols directly.
- Convolution: A sliding window processes each l-word ngram, while every filter computes an inner product with that ngram.The convolution produces one response per filter and ngram position.
- Pooling: Convolution responses form a matrix, and max-pooling across ngram positions produces one value per filter before ReLU.This pooled vector is the representation passed onward for classification.
- Classification: A linear fully connected layer maps the ReLU-activated pooled representation to class scores and outputs the strongest class.The layer uses the pooled filter responses for the final decision.
- Multiple window sizes: Multiple parallel convolution layers can use window sizes ℓ∈L, concatenate their pooled vectors, and also accommodate dilated convolutions.This extends the basic single-window architecture used to describe the pipeline.
3 Datasets and Hyperparameters
The experiments evaluate binary sentiment classification across three review datasets chosen for varied size and domain while retaining a relatively simple, interpretable task. Models use fine-tuned GloVe embeddings and small convolutional configurations.
- Datasets: The experiments cover sentiment classification of positive versus negative user reviews across three datasets.The datasets vary in size and domain while maintaining a binary task.
- Datasets: MR contains 10k evenly split short movie reviews, while Elec contains 200k training and 25k test evenly split electronic-product reviews.These datasets provide different review domains and scales.
- Embeddings: Word representations use pretrained GloVe Wikipedia 2014—Gigaword 5 embeddings that are fine-tuned during training.The embeddings initialize the model’s word vectors.
- Hyperparameters: Models use 50-dimensional embeddings, filter sizes ℓ∈{2, 3, 4}, and m∈{10, 50} filters, with PyTorch and Adam.These settings define the reported empirical configurations.
4 Identifying Important Features
The section refines CNN interpretation by distinguishing informative from accidental max-pooled ngrams and identifying filter-specific class roles and thresholds. Experiments support thresholding activation signals to improve interpretability without sacrificing classification performance.
- Informative vs. Uninformative Ngrams: The classifier observes ngrams through filter-assigned scores in the pooled vector, not through ngram identities directly.Only ngrams contributing to the pooled vector can influence the decision, while their influence is mediated by filter scores.
- Informative vs. Uninformative Ngrams: The analysis separates deliberate ngrams, scored highly because they are likely informative, from accidental ngrams retained only because no higher activation occurred.The proposed threshold treats above-threshold values as informative and lower values as ignorable for classification.
- Filter Class Identities: Each filter receives a class identity based on the output-layer weight contributing most strongly to a classification logit.The identity is then compared with network predictions to construct filter-specific correlation data for threshold selection.
- Threshold Selection: Per-filter thresholds are chosen as the lowest values achieving a sufficiently high purity, with 0.75 working well experimentally.Purity measures the percentage of informative ngrams scored above the threshold.
- Threshold Effectiveness: Replacing ReLU with per-filter thresholding preserves performance and slightly improves it at purity 0.75 on MR, supporting the thresholding approximation.Similar results were observed on the Elec dataset.
5 What is captured by a filter?
The analysis shows that filters cannot be understood simply as homogeneous ngram detectors: their activations reflect slot patterns, multiple semantic classes, and negative cues. Comparing natural and possible ngrams, then thresholding and clustering activations, reveals what each filter captures.
- 5.1 Slot Activation Vectors: A filter’s ngram score decomposes into slot-level word contributions, represented by a vector of activations across the ngram’s positions.Each slot activation measures how much the corresponding word contributes to the filter’s activation.
- 5.2 Naturally occurring vs. possible ngrams: Top-scoring naturally occurring ngrams leave at least one slot weak, unlike independently maximized possible ngrams.Across Elec filters, natural ngrams scored 30% lower than possible ngrams on average, a difference of 2.5.
- 5.2 Naturally occurring vs. possible ngrams: Selected nonnatural ngrams can drive filters beyond their expected activation range and potentially cause model misclassification.The paper reserves systematic exploration of this adversarial potential for future work.
- 5.3 Clustering (Hypothesis (i)): A single filter can capture multiple semantic ngram classes, each associated with a distinct slot-activation pattern.Mean Shift clustering of threshold-passing ngrams identifies these groups without requiring a preset number of clusters.
- 5.4 Negative Ngrams (Hypothesis (ii)): Negative ngrams lower a filter’s score below its threshold by replacing words with low-scoring or strongly negative alternatives.For example, inserting “not” or “n’t” in slot #1 prevents otherwise related ngrams from passing the filter threshold.
6 Interpretability
The paper turns its CNN findings into model- and prediction-level explanations by identifying filter behavior, relevant ngrams, and word-level contributions.
- 6.1 Model Interpretability: Model explanations associate each filter with its contributing class, threshold, purity and coverage, and semantic activation patterns.Patterns are represented through slot-activation clusters, top-k activating ngrams, and corresponding low-scoring negative ngrams.
- 6.1 Model Interpretability: Table 4 pairs top-scoring ngrams from one Elec-dataset filter with lowest-scoring negative ngrams found at a one-word Hamming distance.Bold entries denote Case 2 negative ngrams.
- 6.2 Prediction Interpretability: Prediction explanations retain only ngrams that pass their filter thresholds, producing more concise and relevant explanations than tracing every max-pooled ngram.This removes ngrams selected by max-pooling when no strongly class-associated phrase is present.
- 6.2 Prediction Interpretability: Word-level slot activations expose each word’s contribution and identify negative ngrams whose strongly positive components are counteracted by highly negative slots.These negative ngrams may be excluded by max-pooling or may fail the filter threshold.
7 Conclusion
The conclusion refines common assumptions about text CNNs: max-pooling thresholds relevance, filters capture varied slot-activation patterns, and negative word contributions can suppress otherwise strong ngrams.
- 7 Conclusion: Max-pooling over time induces thresholding that separates convolutional features relevant to final classification from irrelevant features.The thresholding behavior supports identifying which ngrams matter for a classification.
- 7 Conclusion: Filters are not homogeneous: distinct slot-activation patterns allow one filter to assign similar strengths to different linguistic ngram groups.Clustering high-scoring ngrams by these patterns reveals the linguistic groups captured by a filter.
- 7 Conclusion: Filters sometimes assign negative values to particular word activations, lowering ngram scores despite otherwise highly activating words.The paper uses this behavior to identify negative ngrams and improve CNN interpretability.
- 7 Conclusion: These findings motivate improved model-based and prediction-based interpretability for CNNs processing text.The conclusion connects the behavioral analysis to both forms of explanation.