PaperlayerView exact revision on arXiv

Source-linked AI summary

Incorporating Priors with Feature Attribution on Text Classification

Frederick Liu, Besim Avci

arXiv:1906.08286v1cs.CL

TL;DR

Explanations can expose identity-term bias in text classifiers, but existing explanation methods do not themselves offer a remedy. This paper adds attribution priors to the training objective for fairness and scarce-data classification, reporting improved fairness without an original-task performance drop and gains when training data are limited.

  • Problem

    Explanations can reveal identity-term bias in text classifiers, but explanation methods do not suggest remedies, while existing bias-reduction approaches may require more data or trade off task performance.

    Source: §1 · p. 1

  • Method

    The method adds an L2 loss between Path Integrated Gradients attributions for selected tokens and task-specific target values, including zero for identity terms and positive values for known toxic words.

    Source: §1 · p. 2

  • Results

    The method improves fairness metrics on the template-based dataset without an original-task performance drop, and outperforms the baseline by a big margin at 1% and 5% data ratios.

    Source: §1 · p. 2, §4.5 · p. 7, §6 · p. 9

  • Takeaways & Limitations

    Attribution-based fine-tuning can converge to a more debiased classifier in a few epochs, while priors act on human-interpretable tokens.

    Source: §5 · p. 8, §6 · p. 9

    Fairness evaluation relies on a synthetically generated dataset that may contain biases of its creators.

    Source: §5 · p. 8

Abstract

from arXiv · show

Feature attribution methods, proposed recently, help users interpret the predictions of complex models. Our approach integrates feature attributions into the objective function to allow machine learning practitioners to incorporate priors in model building. To demonstrate the effectiveness our technique, we apply it to two tasks: (1) mitigating unintended bias in text classifiers by neutralizing identity terms; (2) improving classifier performance in a scarce data setting by forcing the model to focus on toxic terms. Our approach adds an L2 distance loss between feature attributions and task-specific prior values to the objective. Our experiments show that i) a classifier trained with our technique reduces undesired model biases without a trade off on the original task; ii) incorporating priors helps model performance in scarce data settings.

1 Introduction

The introduction frames model interpretability as important for adoption, while noting that explanations expose unfairness without fixing it. It presents attribution priors as a way to address identity-term bias and scarce-data classification, reporting fairness improvements without an original-task performance trade-off.

1 Introduction

  • The paper motivates model explanations as necessary for wider adoption of complex models, including in regulated domains where explanations are required.It notes that explainability concerns can lead practitioners to use simpler, potentially lower-performing models.Source: §1 · p. 1
  • The introduction says text-model explainability research is modest compared with image explanation research.Source: §1 · p. 1
  • Model explanations can reveal unfairness, such as toxicity predictions driven by identity terms, but explanation methods do not themselves suggest how to remedy the problem.The introduction gives “I am gay” as an example in which Integrated Gradients attributes most of the toxicity prediction to “gay.”Source: §1 · p. 1
  • Prior bias-mitigation approaches include collecting more data, using unbiased word vectors, and optimizing fairness proxies with adversarial training, but may be costly or trade off task performance.The introduction contrasts the proposed approach with these options as alternatives for addressing bias.Source: §1 · p. 2
  • The approach adds an L2 loss between Path Integrated Gradients for selected tokens and target attribution values, using zero for protected-group terms and positive targets for known toxic words.The two applications are reducing identity-term bias and improving classification with little data.Source: §1 · p. 2
  • The method improves fairness metrics on a synthetic unbiased dataset without degrading original-task performance, and also improves toxicity classification when training data are scarce.The introduction also reports lower average attributions to identity terms and better word vectors as by-products.Source: §1 · p. 2

2 Feature Attribution

Feature attribution represents how input features contribute to a model prediction and can help interpret decisions from nonlinear models. Integrated Gradients relates those attributions to a baseline by aggregating gradients along a path to the input.

2 Feature Attribution

  • Feature attribution represents each input feature’s contribution to a prediction and can support interpretation of nonlinear models that lack transparent feature contributions.For linear models, attribution is the product of the feature value and its coefficient.Source: §2 · p. 2
  • Attribution methods seek to quantify feature contributions to predictions; Integrated Gradients and SHAP also satisfy properties such as completeness, where summed attributions equal the difference between uncertainty and model output.Source: §2 · p. 2

Integrated Gradients

  • Integrated Gradients aggregates gradients from an uninformative, high-entropy baseline toward the actual input to produce feature attributions.It uses a straight path; integrating along the path can alleviate gradient saturation around the input.Source: Integrated Gradients · pp. 2–3
  • In NLP, the input is the concatenated embedding sequence, and each token’s attribution is the sum of its embedding attributions.Source: Integrated Gradients · p. 3
  • The paper selects Integrated Gradients because it is theoretically justified, effective in NLP-related tasks, differentiable with respect to model parameters, and lightweight to implement and execute.Source: Integrated Gradients · p. 3

3 Incorporating Priors

The method makes feature-attribution priors actionable by adding a loss that penalizes deviations from target attributions. Users specify which tokens matter and the attribution values they should receive.

3 Incorporating Priors

  • Penalizing attribution deviations on selected features lets practitioners inject priors, with identity terms targeted toward neutrality so predictions rely more on context.For tokens identifying protected groups, the target attribution is zero.Source: §3 · p. 3
  • The prior loss measures squared distances between token attributions and targets, with targets defined per input token and attributions linked to Definition 2.1.The supplied passages state that the loss is for a scalar output.Source: §3 · p. 3
  • For multi-class problems, the joint objective combines class-specific attribution loss with cross-entropy, controlled by hyperparameter λ.The cross-entropy uses ground-truth labels and class posterior probabilities.Source: §3 · p. 3
  • The joint objective is differentiable with attribution calculated through Equation 1 and is described as applicable across problem settings without dataset dependence.Users specify target attribution values for tokens of interest.Source: §3 · p. 3

4 Experiments

The experiments test attribution priors for mitigating identity-term bias in toxic-comment classification and for improving performance when training data are scarce. They use Wikipedia Talk Page comments and a synthetic dataset designed to assess fairness under controlled identity terms.

4 Experiments

  • The experiments target unintended identity bias in toxic-comment classification and use high attribution targets on human-selected toxic terms to address scarce-data performance.The section compares the proposed method with a classifier without prior loss and two other baselines.Source: §4 · p. 4

4.1 Dataset

  • The Wikipedia Talk Pages dataset has 95,692 training, 32,128 development, and 31,866 test samples; 9.7% of training labels are toxic.Human raters defined toxicity as comments likely to make someone leave a discussion.Source: §4.1 · p. 4
  • A template-based synthetic dataset replaces identity placeholders with identity terms, enabling fairness evaluation across terms in otherwise identical contexts.The authors focus on this dataset because the original test sets follow the same biased distribution.Source: §4.1 · p. 4, Table 3 · p. 4

4.2 Experimental Setup

The setup uses a CNN text classifier and specifies attribution targets and training conditions for the fairness and data-scarcity experiments. The reported Table 4 values include results for the fine-tuned model.

4.2 Experimental Setup

  • The text classifier is a CNN with 128 filters for 2-, 3-, and 4-grams, 128-dimensional embeddings, dropout 0.2, and Adam optimization.Sequences are padded or truncated to length 100; tokens appearing fewer than five times are excluded.Source: §4.2 · p. 4

Acc F1 AUC FP FN

  • The reported Fine-tuned and unlabeled rows give distinct Accuracy, F-1, AUC, False Positive, and False Negative values, but the supplied row fragment does not identify the latter candidate.The Fine-tuned values are reported in the first result point.Source: Acc F1 AUC FP FN · p. 5, Table 4 · p. 5
  • The fine-tuned model reports .958 Accuracy, .752 F-1 score, .960 Area Under ROC curve, .009 False Positive, and .032 False Negative.Table 4 reports means over five runs and a maximum variance of .012.Source: Table 4 · p. 5, Acc F1 AUC FP FN · p. 5

4.2 Experimental Setup

  • For fairness, identity terms receive target attribution 0; for data scarcity, toxic terms receive target 1, with λ set to 10^5 across data sizes.The scarcity experiments use five runs; reducing IG integration steps from 50 to 10 cuts reported training time from 30 to 6 minutes for the joint loss.Source: §4.2 · p. 4, Acc F1 AUC FP FN · p. 5

Implementation Decisions

The implementation blocks the prior loss from back-propagating through interpolated embeddings, preserving Integrated Gradients’ axioms. In fairness experiments, the method avoids degrading original-task metrics and is compared with alternative training settings.

Implementation Decisions

  • Treating interpolated embeddings as constants prevents prior-loss back-propagation to embedding parameters, because differentiating the interpolation would break Integrated Gradients’ axioms.The authors also cite the Hessian’s computational cost; embedding parameters still update through cross-entropy during training.Source: Implementation Decisions · p. 5, Acc F1 AUC FP FN · p. 5

4.3.1 Evaluation on Original Data

  • The method does not degrade—and may improve—all reported original-task metrics, unlike approaches discussed by the authors; Table 5 reports performance on samples containing identity terms.The comparison covers accuracy and F-1, and the table reports statistics for identity-term samples.Source: Table 5 · p. 5

4.3 Results on Incorporating Fairness Priors

  • The comparison includes a cross-entropy baseline, identity-sample importance weighting, identity-token replacement, and a separately fine-tuned classifier.Importance weighting uses a coefficient of 10 selected on the development set; fine-tuning applies joint loss to an already-trained classifier.Source: §4.3 · p. 5
  • Fine-tuning tests whether an already-trained classifier can be adjusted with joint loss, including when its original training took a long time.The supplied passages describe this experiment’s aim but do not provide its outcome here.Source: §4.3 · p. 5

AUC FPED FNED

On template-based synthetic data, the method improves AUC and nearly eliminates false-positive and false-negative inequality across identities. Fine-tuning also mitigates bias, while importance weighting does not produce an unbiased model.

AUC FPED FNED

  • After two epochs, fine-tuning reaches performance similar to joint training, though it is slightly poorer.The reported fine-tuning result is after two epochs.Source: AUC FPED FNED · p. 6

4.3.2 Evaluation on Synthetic Data

  • The method substantially improves AUC and almost completely eliminates false-positive and false-negative inequality across identities on the synthetic dataset.FPED and FNED are proxies for Equality of Odds; lower values indicate better bias-mitigation metrics.Source: §4.3.2 · p. 6, Table 7 · p. 6
  • The synthetic dataset is used to measure bias because it is unbiased toward identities, and the method’s fairness evaluation reports AUC, FPED, and FNED.FPED sums differences between overall and identity-specific false-positive rates; FNED does the same for false negatives.Source: §4.3.2 · p. 6
  • The fine-tuned model outperforms the baseline for mitigating bias, while importance weighting fails to produce an unbiased model.Token replacement is described as a useful bias-mitigation baseline because it treats all identities alike.Source: §4.3.2 · p. 6

4.4 Nearest Neighbors of Identity Terms

The attribution-based method changes identity-term embeddings toward more neutral neighbors. The authors describe these embeddings as inherently unbiased, while noting they may not match general-purpose embedding quality.

4.4 Nearest Neighbors of Identity Terms

  • The table compares nearest neighbors for “gay,” “homosexual,” and the <id> token used by token replacement.The selected identity terms are described as having the largest mean attribution differences between the proposed method and baseline.Source: §4.4 · p. 7
  • The embedding of “gay” shifts from swear-word neighbors toward the <pad> token, while “homosexual” moves toward more neutral terms.The baseline embedding space still places mostly swear words near “homosexual”; the authors note the resulting embeddings may not be general-purpose quality because of data size and the model’s objective.Source: §4.4 · p. 7
  • The authors contrast the methods’ embedding effects: importance weighting does not appear to move “gay” toward neutral words, while token replacement places the shared identity token among neutral neighbors.Collapsing identities into one token is associated with better synthetic-dataset results but lower original-dataset classification performance.Source: §4.4 · p. 7

4.5 Results on Incorporating Priors in Different Training Sizes

The attribution prior improves test accuracy most in the smallest training settings. Its advantage diminishes as more data lets the baseline learn to focus on toxic words, and both methods eventually outperform a rule-based approach.

4.5 Results on Incorporating Priors in Different Training Sizes

  • The prior method outperforms the baseline by a big margin at 1% and 5% training-data ratios, with its impact diminishing as more data is added.The authors attribute the diminishing impact to the model learning to focus on toxic words without prior injection.Source: Figure 1 · p. 7, §4.5 · p. 7
  • The method yields higher attributions for toxic words than the baseline across different data ratios, while baseline attributions also rise as training data increases.Table 8 reports mean attribution values for a subset of toxic terms across training sizes.Source: §4.5 · p. 7
  • As training data grows, both the baseline and prior-based classifier eventually outperform the rule-based approach that predicts toxicity when a toxic word appears.The figure reports test accuracy at different training sizes.Source: §4.5 · p. 7, Figure 1 · p. 7

5 Discussion and Related Work

The discussion situates attribution-based priors among explainability and bias-mitigation approaches, emphasizing their actionability and token-level focus. It also identifies practical limitations in the fairness setting and its evaluation.

5 Discussion and Related Work

  • Text explanations face both a shortage of suitable counterfactual methods and critiques of existing methods’ faithfulness, correctness, and usefulness.Sensitivity-based approaches rely on small, local perturbations, while suitable text perturbations are difficult to generate.Source: §5 · pp. 7–8
  • Park et al. (2018) address gender bias but do not scale across race and religion; Dixon et al. (2018) use data-specific augmentation, and both trade accuracy for fairness.The paper contrasts these constraints with its own method, which works with tokens and does not sacrifice original-task performance.Source: §5 · p. 8
  • Unlike adversarial approaches that remove protected attributes from representation layers, this method operates on interpretable tokens and is described as avoiding an original-task performance sacrifice.The discussion characterizes representation-layer removal as unstable.Source: §5 · p. 8
  • Fairness mitigation can suppress identity terms even when they are used as insults, and manually supplied priors resemble blacklists.The authors also note that evaluation on a synthetic dataset may reflect biases of its creators.Source: §5 · p. 8

6 Conclusion and Future Work

The conclusion presents attribution-based priors as a way to address bias in toxic-comment classification and reports fairness gains without an original-task performance drop. It also points to broader uses and future extensions.

6 Conclusion and Future Work

  • The method incorporates Path Integrated Gradients attributions into the objective to penalize classifiers for focusing on identity words.The authors frame this as enabling practitioners to enforce priors through model explanations.Source: §6 · p. 8
  • Joint cross-entropy and prior-loss training avoids an original-task performance drop while improving fairness metrics on the template-based dataset.Fine-tuning a trained classifier with attribution makes it converge to a more debiased classifier in a few epochs, and the method can force attention to predetermined tokens.Source: §6 · p. 9
  • Future directions include penalizing attributions outside class-relevant tokens, trying DeepLRP, and automatically extracting problematic terms.The current approach uses prescribed identity or toxic terms.Source: §6 · p. 9
Loading 1906.08286v1…