CausaLM: Causal Model Explanation Through Counterfactual Language Models

• Existing explanation tools often measure correlations through local perturbations, so they cannot determine whether correlated concepts actually cause a model prediction.This limitation can produce misleading interpretations, including mistaken conclusions about political bias in text classifiers.Source: §1 · pp. 2–3
• Generating text counterfactuals that alter one high-level concept while preserving others is difficult, motivating counterfactual representation generation through adversarial training.The approach manipulates encoded representations instead of natural-language text, avoiding the intractability of controlled text generation.Source: §1 · p. 3
• CausaLM trains an additional language representation model to forget the target concept while retaining important control concepts, reducing confounding from correlated information.The relevant concepts and controls are determined by the assumed causal model of the text-generating process.Source: §1 · pp. 3–4
• CausaLM estimates a concept’s causal effect on classifier predictions by comparing representations from the original and concept-invariant language models.The method takes classifier training data and a concept of interest, then produces test-set causal effects without requiring counterfactual examples.Source: §1 · p. 4
• The framework is evaluated on four datasets and four BERT-based cases involving Adjectives, Topics, Gender, and Race in sentiment and mood-state classification.Three datasets include counterfactual examples, allowing estimated effects to be compared with manually computed ground-truth causal effects.Source: §1 · p. 4
• Because intervention occurs before supervised task training, CausaLM is not tied to BERT’s implementation and can also support unwanted-bias mitigation.The paper additionally proposes ways to assess estimator quality when real-world ground-truth causal effects are unavailable.Source: §1 · pp. 4–5

2. Previous Work

• Research at the intersection of DNN interpretation, causal inference, and NLP remains limited, although work in each area is extensive.The section surveys causal inference in NLP, model interpretation, and debiasing to position the contribution.Source: §2 · p. 5

2.1 Causal Inference and NLP

• Causal inference is difficult to apply in NLP because high-dimensional language does not fit existing methods, though recent work is beginning to address this gap.Prior studies measure confounders through text, use text as confounders, or learn causally driven representations; Veitch, Sridhar, and Blei (2019) fine-tune BERT to estimate treatment effects.Source: §2.1 · pp. 5–6
• CausaLM applies counterfactual analysis to explain how textual concepts affect neural classifier predictions without generating counterfactual examples.It intervenes on textual representations to infer what predictions would have been under different conditions, extending causal inference toward causal model explanation.Source: §2.1 · p. 6
• Unlike Vig et al. (2020), which relies on manually constructed queries and compares query probabilities, CausaLM generalizes causal explanations to textual concepts and model predictions.CausaLM approximates each example’s counterfactual representation, avoiding manual counterfactual construction and enabling comparison with original model predictions.Source: §2.1 · p. 6

2.2 Model Interpretations and Debiasing in NLP

• Existing NLP interpretation methods are structural or behavioral, but generally do not directly measure how text structure or linguistic concepts causally affect model outcomes.They often rely on token-level analysis and do not account for counterfactuals; causal measurement of concepts’ effects had largely been developed in computer vision instead.Source: §2.2 · pp. 6–7
• CausaLM provides datasets for measuring bias by comparing predictions on examples and their counterfactuals, unlike existing observational bias measures.These datasets support testing whether candidate causal explanation methods estimate textual-feature effects, with future plans for richer counterfactual comparisons.Source: §2.2 · p. 7

3.1 Causal Inference and Language Representations

• The framework distinguishes passive associations from causal effects by modeling concepts, texts, representations, and classifier outputs with causal graphs and the do-operator.Confounding occurs when variables such as Political Figure affect both the treated concept and the text, making simple comparisons inaccurate.Source: §3 · p. 7, §3.1 · p. 9

3. Causal Model Explanation

• The section establishes causal model explanation as estimating a treatment’s effect on model predictions and presenting that effect to explain observed behavior.It connects causal-inference concepts with NLP interpretability, where the classification decision is the outcome and an input feature is intervened upon.Source: §3 · p. 7

3.3 Representation-Based Counterfactual Generation

• CausaLM intervenes on language representations instead of generating counterfactual texts, approximating representations that are insensitive to a chosen concept while preserving other concepts.This avoids the need for conditional generative models, which are often intractable for NLP counterfactuals.Source: §3.1 · p. 10, §3.2 · p. 12, §3.3 · p. 13

3.2 The Textual Representation-based Average Treatment Effect (TReATE)

• TReATE explains text-classification predictions by measuring the causal effect of one or more concepts through interventions on textual representations.It is a representation-based special case of the average treatment effect, designed to control potentially confounding concepts.Source: §3.2 · p. 12

3.3 Representation-Based Counterfactual Generation

• Direct text counterfactuals are difficult because manual edits are costly and conditional generators often fail to produce realistic, locally matched NLP examples.Local explanations require preserving all information except the concept of interest, making generation especially demanding.Source: §3.3 · p. 13
• Adversarial fine-tuning adds tasks that forget treated concepts and remember control concepts, producing a counterfactual BERT representation for causal explanation.Gradient reversal maximizes the treated-concept classifier’s loss while minimizing MLM, NSP, and control-concept losses, encouraging concept invariance.Source: §3.3 · pp. 14–15, Figure 3 · p. 14
• Adversarial objectives are added during language-model fine-tuning, before downstream prediction training, so they are not directly opposed to the prediction objective.The auxiliary tasks follow the researcher’s assumed causal graph and can include classification tasks proposed for PF (CC).Source: §3.3 · pp. 15–16
• TRITE averages absolute class-probability differences between original BERT and BERT-CF predictions, while estimated TReATE sums TRITE across all test examples.The comparison uses the original representation model and the counterfactual model for each example.Source: §3.3 · p. 16
• BERT-CF represents an intervention where TC has no effect while CC is preserved, producing a counterfactual language representation model.The counterfactual representation is obtained by optimizing the loss functions in Equation 9.Source: §3.3 · p. 16

4. Data

• Causal effect identification depends on assumptions encoded in a causal graph, unlike non-causal explanations that may recover arbitrary correlations.The proposed evaluation therefore emphasizes datasets with counterfactual examples and, in three cases, comparisons against ground-truth effects.Source: §4 · p. 16

4.1 Product and Movie Reviews

• The combined sentiment dataset contains 10,000 balanced reviews across five domains and labels, enabling controlled variation in Adjectives and Topics.It combines product reviews from Books, DVD, Electronics, and Kitchen Appliances with sampled IMDB movie reviews; adjective counterfactuals delete tagged adjectives, while topics are derived from a 50-topic LDA model.Source: §4.1 · pp. 17–18
• The sentiment data cannot provide ground-truth topic effects because automatically removing topics while preserving grammaticality is infeasible, and adjective deletion may sometimes damage grammaticality.These limitations motivate datasets with greater control over the data-generating process for more accurate ground-truth estimation.Source: §4.1 · p. 18

4.2 The Enriched Equity Evaluation Corpus (EEEC)

• For every EEEC example, gender and race counterfactuals are generated by changing only the relevant person name and, for gender, corresponding pronouns.This enables causal estimation of Gender and Race effects on mood classifiers while supporting research aimed at detecting and reducing biased predictions.Source: §4.2 · pp. 19–20
• EEEC expands EEC into 33,738 longer, more diverse synthetic sentences from 42 templates, adding contextual and non-informative phrases to reduce memorization and create richer training signals.The original EEC contains 8,640 concise sentences for POMS mood classification, while EEEC supports controlled training-set bias and counterfactual evaluation.Source: §4.2 · pp. 19–20

5. Tasks and Experiments

• The study covers sentiment and mood datasets annotated for Adjectives, Topics, Gender, and Race, using Balanced, Gentle, and Aggressive concept-label correlations.Table 3 summarizes the four tasks and associated forgetting and confounder-control objectives, while Table 4 summarizes correlations across experiments.Source: §5 · pp. 20–21, Table 3 · p. 21, Table 4 · p. 21

5.1 The Causal Effect of Adjectives on Sentiment Classification

• Adjective experiments manipulate adjective-label correlation and adversarially predict adjective presence while preserving other part-of-speech information as a control concept.The data use 64%, 16%, and 20% train-development-test splits; the IMA task masks adjectives and matched non-adjectives, while the control task tags non-adjective words.Source: §5.1 · pp. 22–23

5.2 The Causal Effect of Topics on Sentiment Classification

• Topic experiments represent global concepts with LDA probabilities, use treated and control topic objectives, and estimate effects across domains without ground-truth counterfactuals.Topics cannot be counterfactually generated by deleting words and are mutually confounding because their probabilities lie on a simplex; domain variation enables qualitative estimator analysis.Source: §5.2 · pp. 24–25

5.3 The Causal Effect of Gender and Racial Bias

• Gender and Race experiments use templated sentences to switch one concept while holding the rest fixed, enabling direct comparison with true causal effects and bias-mitigation evaluation.Balanced, Gentle, and Aggressive datasets vary concept-label correlations, with the same 64%, 16%, and 20% split used for training, development, and testing.Source: §5.3 · p. 26

5.4 Comparing Causal Estimates to the Ground Truth

• Ground-truth ATE uses manually created counterfactuals, while TReATE compares BERT-O and BERT-CF predictions against correlation-based CONEXP and TPR-GAP baselines.BERT-MLM provides a fine-tuning baseline; TPR-GAP compares accuracy rather than class-probability distributions, preventing direct comparison with ground-truth ATE or TReATE.Source: §5.4 · pp. 26, 28, 29

6.1 Estimating TReATE (The Causal Effect)

• TReATE(O, CF) closely approximates ATEgt(O) across Adjectives, Gender, and Race experiments, with at most 3.9% absolute error and average error of 2.6%.The estimator remains accurate across Balanced, Gentle, and Aggressive bias settings, whereas CONEXP(O) substantially underestimates effects in 7 of 9 experiments and INLP overestimates them.Source: §6.1 · pp. 29–30, Table 5 · p. 30, Table 6 · p. 31
• Without ground truth for Topics, TReATE estimates highest effects in the most correlated domains, while adding a second control concept reduces but does not substantially alter the estimate.For tT C(movies), effects are highest on Movies and DVD; for tT C(books), they are higher on DVD and Movies than on Electronics and Kitchen Appliances.Source: §6.1 · p. 32, Table 8 · p. 32, Table 9 · p. 33

6.2 Analyzing the Counterfactual Model

• BERT-CF makes treated-concept classifiers perform only slightly above chance while preserving control-concept accuracy, which decreases by just 2–10% absolute versus BERT-O.This structural analysis is a sanity check: classifier performance measures usable information, not how information is encoded, and may miss effects on untested confounders.Source: §6.2 · pp. 33–35, Figure 7 · p. 33, Figure 8 · p. 34
• BERT-CF models outperform BERT-O and BERT-MLM on Balanced test sets after Aggressive training, showing improved generalization when training correlations disappear.The experiments support that models learn treated-concept–label associations under aggressive bias, while BERT-CF is less affected by the resulting distribution shift.Source: §6.2 · p. 35, Table 10 · p. 35

6.3 Analyzing the Stage 2 Multi-Task Training Scheme

• Stage 2 adversarial training briefly increases task losses but typically converges within 1–2 epochs without drastically harming MLM loss or language-representation stability.Adding control-concept tasks can delay and increase treated-concept loss while leaving MLM behavior similar; MLM loss is lower for BERT-CF in Adjectives and Topics and only slightly higher in Gender.Source: §6.3 · pp. 35–38

• CausaLM models data-generating processes with causal graphs, generates counterfactual representations via adversarial training, provides four datasets and evaluation tools, and supports model debiasing.The framework avoids direct text generation by implementing the do-operator through adversarially trained language representations.Source: §7 · p. 38
• Explicit causal graphs and carefully chosen control concepts are essential because uncontrolled confounders can make correlated variables appear to cause model predictions.Sanity checks can test whether a given control concept was controlled, but modelers must still identify treatment and control concepts and state their world-model assumptions.Source: §7 · pp. 38–39
• Causal graphs with parent concepts may combine direct effects with effects mediated through caused concepts, while some NLP interventions remain impossible despite proposed sanity checks.Domain experts may help construct graphs in medicine, but identifying confounders and relationships is more challenging in NLP.Source: §7 · p. 39
• Binary concept interventions cannot change a concept’s value, so TReATE compares representations that encode a concept with representations that omit it rather than switching attributes such as gender or race.Although Gender and Race effects were empirically estimated, this mismatch with real-world counterfactuals remains a limitation targeted for future work.Source: §7 · p. 39
• Global concepts such as topics and style remain difficult to analyze: the proposed Topics solution is both global and integrable with counterfactual representations, but their effects remain an open challenge.Unlike local concepts such as Gender and Race, global concepts cannot be adequately measured through token-level analysis.Source: §7 · p. 39
• Validating causal explanations requires a tradeoff between reliable synthetic data and natural real-world data, where manually precise counterfactuals are usually difficult to create.The four datasets address this need, but EEEC has limited language variety and Sentiment relies on potentially inaccurate automatically generated counterfactuals.Source: §7 · pp. 39–40

• Misinterpreting doctor- or hospital-specific textual features in clinical prediction models could have devastating implications.The need for understanding is amplified when clinical notes are used to predict clinically important factors.Source: A. A Clinical Example · p. 41
• A classifier relying on a doctor’s verbose writing style could fail on a concise note containing the same clinical information.The example note describes extensive mental-health information that could be summarized without losing much clinically relevant content.Source: A. A Clinical Example · p. 41
• Changing note style requires substantial intervention because replacing long words can alter structure, content, and clinical concepts, while section-specific word lengths create potential confounders.Word length is used as a proxy for writing style (Sari, Stevenson, and Vlachos 2018), but no ex-ante test ensures that interventions change only the intended concepts.Source: A. A Clinical Example · p. 41
• Doctor-specific writing style can confound clinical predictions, making performance appear promising while risking patients’ health after deployment with another doctor.Without estimating the causal effect of writing style, the model’s reliance on that style cannot be determined.Source: A. A Clinical Example · p. 41
• Clinical notes may encode intertwined treatment suggestions and patient conditions, making counterfactual texts impossible when an upstream concept changes but its generated consequence remains fixed.The example contrasts graphs where treatment suggestions and conditions jointly generate text with one where depression alone causes the suggestion and text follows it.Source: A. A Clinical Example · p. 41

Alternative causal graphs

• The framework’s ability to intervene on a treated concept and estimate its causal effect depends on the assumed causal graph, especially when concepts have inheritance relations.The paper considers graphs in which one or more concepts cause an intermediate concept that generates the text.Source: B. Alternative Causal Graphs and… · pp. 41–43

Limitations

• Clinical notes are not evaluated because they were not publicly available; the authors plan to create synthetic clinical-note data in future work.Figure 13 provides a fake nursing progress note example, but it is not an experimental dataset.Source: B. Alternative Causal Graphs and… · p. 41

Alternative causal graphs

• Intervening on Depression when Anxiety, Depression, and Suicidal jointly cause Clinical Recommendation measures both Depression’s direct and indirect effects, while intervening on Clinical Recommendation may require blank text.Because of the inheritance relation, the method cannot preserve all recommendation information while removing Depression information.Source: B. Alternative Causal Graphs and… · p. 43
• When only Depression causes Clinical Recommendation, intervening on Depression prevents recovery of the doctor’s recommendation, so neither Depression’s nor Clinical Recommendation’s causal effect can be estimated.Sanity checks can assess whether the learned representation forgets the treated concept while retaining control concepts, but some interventions remain limited.Source: B. Alternative Causal Graphs and… · p. 43

C. The EEEC Dataset

• The EEEC dataset generation process is further detailed through templates presented in Table 11.These templates specify how the data were generated.Source: C. The EEEC Dataset · p. 44
• Table 12 compares descriptive statistics for the original EEC and Enriched EEC (EEEC) datasets to illustrate their key modifications.The comparison highlights how EEEC differs from EEC.Source: C. The EEEC Dataset · p. 44, Table 12 · p. 45
• Table 13 reports the hyper-parameters used in the experiments.The table documents the experimental settings.Source: Table 13 · p. 45

Experimental pipeline

• Experiments consistently fine-tune BERT representations, train downstream classifiers, and evaluate causal and baseline estimators on test sets across classification, probing, and dataset-version settings.The pipeline applies to Sentiment, POMS, TC, and CC tasks and to Balanced, Gentle, and Aggressive datasets; Stage 2 produces BERT-CF and BERT-MLM, followed by Stage 3 classifier training and testing.Source: D. Experimental Pipeline and… · p. 45

Model configuration

• All experiments use case-sensitive BERT-base with 12 layers, 768-dimensional hidden vectors, 12 attention heads, and 110M parameters, pretrained on BookCorpus and Wikipedia.BERT-base is fine-tuned across all 12 layers for BERT-CF; downstream classifiers use top-layer token representations and the CLS token.Source: D. Experimental Pipeline and… · p. 46
• The downstream classifier is fully connected, while INLP receives BERT-O’s CLS representation and passes its output to logistic regression for TReATE(O, INLP).This specifies the representation interfaces and classifier used for the INLP comparison.Source: D. Experimental Pipeline and… · p. 46

Hyper-parameters

• Hyper-parameters were not tuned because the pipeline required many experiments across dataset versions and model variations; all models use cross entropy and ADAM with learning rate 1e−3, fuzz factor 1e−8, and no weight decay.The implementations use PyTorch with modified HuggingFace Transformers and PyTorch Lightning code.Source: D. Experimental Pipeline and… · p. 46