PaperlayerView exact revision on arXiv

Source-linked AI summary

LIBERTy: A Causal Framework for Benchmarking Concept-Based Explanations of LLMs with Structural Counterfactuals

Gilat Toker, Nitay Calderon, Ohad Amosy, Roi Reichart

arXiv:2601.10700v2cs.CLcs.AI

TL;DR

Concept-based explanation benchmarks lack reliable causal references, often relying on costly human-written counterfactuals and limited settings. LIBERTy constructs structural counterfactual datasets from explicit SCMs, evaluates methods across tasks and models, and finds substantial room for improvement alongside low demographic sensitivity in some LLMs. The framework also exposes scope boundaries arising from synthetic text and simplified causal structures.

  • Problem

    Reliable evaluation of concept-based explanations is difficult because ground-truth explanations are scarce and existing benchmarks rely on limited, human-written counterfactuals.

  • Method

    LIBERTy explicitly defines SCM-based data-generating processes in which concept interventions propagate through an LLM to generate structural counterfactuals for evaluation.

  • Results

    The benchmark finds substantial room for improving local and global explanations, while some LLMs show low sensitivity to demographic interventions, likely due to post-alignment mitigation.

  • Takeaways & Limitations

    LIBERTy provides a causal framework for benchmarking faithful explainability methods and for analyzing model sensitivity to concept interventions.

  • Takeaways & Limitations

    LIBERTy uses synthetic text and simplified causal structures that may not perfectly match how humans express concepts or how real-world mechanisms operate.

Abstract

from arXiv · show

Concept-based explanations quantify how high-level concepts (e.g., gender or experience) influence model behavior, which is crucial for decision-makers in high-stakes domains. Recent work evaluates the faithfulness of such explanations by comparing them to reference causal effects estimated from counterfactuals. In practice, existing benchmarks rely on costly human-written counterfactuals that serve as an imperfect proxy. To address this, we introduce a framework for constructing datasets containing structural counterfactual pairs: LIBERTy (LLM-based Interventional Benchmark for Explainability with Reference Targets). LIBERTy is grounded in explicitly defined Structured Causal Models (SCMs) of the text generation, interventions on a concept propagate through the SCM until an LLM generates the counterfactual. We introduce three datasets (disease detection, CV screening, and workplace violence prediction) together with a new evaluation metric, order-faithfulness. Using them, we evaluate a wide range of methods across five models and identify substantial headroom for improving concept-based explanations. LIBERTy also enables systematic analysis of model sensitivity to interventions: we find that proprietary LLMs show markedly reduced sensitivity to demographic concepts, likely due to post-training mitigation. Overall, LIBERTy provides a much-needed benchmark for developing faithful explainability methods.

1 Introduction

LIBERTy addresses limitations in causal evaluation of concept-based explanations by generating structural counterfactuals from explicitly defined SCMs. Across datasets and models, it benchmarks explanation quality and reveals both performance gaps and reduced demographic sensitivity in some proprietary LLMs.

  • Framework: LIBERTy generates structural counterfactuals by intervening on concepts and propagating changes through an SCM until an LLM produces the counterfactual text.The framework treats the LLM as part of the text-generation data-generating process, aligning reference targets with that process.
  • Benchmark and metric: LIBERTy avoids costly human counterfactual annotations and aligns evaluation references with the specified data-generating process.Its counterfactuals are gold relative to the DGP but called silver because the DGP and generated texts are synthetic.
  • Benchmark and metric: Three datasets cover disease detection, CV screening, and workplace violence prediction, while order-faithfulness measures whether explanations preserve the relative ordering of intervention effects.Order-faithfulness supports importance scores on arbitrary scales rather than requiring direct causal-effect estimates.
  • Results: Matching methods using representations from dataset-specific fine-tuned models perform best overall, but substantial headroom remains for improving concept-based explanations.The benchmark covers linear erasure, counterfactual generation, matching, and concept-attribution methods.
  • Results: Some LLMs, including GPT-4o, show very low sensitivity to demographic interventions, potentially reflecting dedicated post-training alignment.Fine-tuned models can track the ground-truth effects specified by the data.
  • Conclusion: LIBERTy provides a reliable, scalable, and flexible causal framework for benchmarking more faithful explainability methods.The framework evaluates explanation methods against structural-counterfactual reference effects.

2 Related Work

Prior concept-based explainability benchmarks are limited by scarce ground-truth explanations, simple datasets, and reliance on indirect or human-written proxies. LIBERTy responds with richer causal structures and structural counterfactuals for systematic evaluation.

  • Concept-based Explainability: Concept-based explanations quantify how human-interpretable concepts influence model predictions and are useful for end-users and decision-makers.They can reduce the complexity of long inputs and communicate model behavior in intuitive terms.
  • Concept-based Explainability: Existing concept-based approaches include attribution, concept erasure, counterfactual generation, and matching methods.LIBERTy evaluates representative methods from these families.
  • Explainability Benchmarks: Ground-truth explanations are rarely available in real-world datasets, so evaluations often rely on agreement between methods or simple heuristics.Most prior evaluations also focus on token-level explanations rather than high-level semantic concepts.
  • Explainability Benchmarks: LIBERTy addresses these gaps by simulating diverse text types with richer causal graphs and explicitly defined data-generating processes.Its framework combines concept SCMs, exogenous grounding texts, and an LLM to generate silver counterfactuals.

3 Evaluation of Explanations

LIBERTy evaluates local and global concept explanations against causal effects generated by an SCM, using error distance and a new order-based measure. Its counterfactual estimates are exact under fixed exogenous variables and deterministic decoding, while stochastic decoding introduces additional noise.

  • Local explanations target individual CaCE effects, whereas global explanations target population-level CaCE effects across the data distribution.
  • Counterfactual evaluation changes a concept while holding exogenous variables fixed, then measures the explained model’s prediction on the generated counterfactual text.
  • CaCE and ICaCE are vector-valued effects over all classes of the model’s predicted outcome.
  • ICaCE Error Distance compares reference causal effects with explanation outputs using cosine distance, L2 distance, and norm difference, whose mean is reported as ED.
  • With fixed exogenous variables and zero-temperature decoding, ICaCE is exact; stochastic decoding adds token-sampling noise.
  • Order-Faithfulness measures whether explanations preserve the relative ordering of concept importance by comparing the signs of reference-effect and explanation differences.

4 Interventional Data Generation

LIBERTy generates structural counterfactual data from explicit causal graphs, structural equations, grounding texts, and an LLM. Interventions follow Pearl’s procedure while preserving persona, template, and deterministic decoding.

  • LIBERTy’s interventional benchmark combines SCMs over concepts, exogenous grounding texts, and an LLM to generate silver counterfactuals.
  • Each dataset defines directional concept relationships and structural equations with Gaussian exogenous noise; text generation uses concepts, a persona, and a template.
  • The three causal graphs are grounded in prior literature or real-world statistical patterns but are not intended to mirror the world’s true causal structure.
  • Deterministic decoding is required to hold generation factors fixed, but it produces generic, repetitive, and inauthentic text.
  • Persona and template variables improve authenticity and diversity by providing contextual attributes and discourse structures during generation.
  • Counterfactual generation uses abduction, intervention, and prediction, then regenerates text with the same persona, template, and deterministic decoding.

5 Datasets

LIBERTy provides three high-stakes datasets with separate model-training, explanation-training, and counterfactual test subsets. The datasets cover workplace violence prediction, disease detection, and CV screening using structured text-generation procedures.

  • Each dataset has four subsets: two for training and testing the explained model, one for training explanations, and one counterfactual test set.
  • Workplace Violence Prediction: The workplace violence dataset models HR–nurse interviews predicting a nurse’s likelihood of experiencing violence, using a graph adapted from the Minnesota Nurses’ Study.
  • Workplace Violence Prediction: Its interview templates randomize concept-question order and combine opening and closing variants with personas containing concept-centered informal facts.
  • Disease Detection: The disease-detection dataset models clinical self-reports in which symptoms described in a medical forum post are used to predict disease.
  • CV Screening: The CV-screening dataset models automated resume assessment, predicting applicant quality from CV-style personal statements with labels including weak, qualified, and outstanding.

6 Experimental Setup

The experiments benchmark eight explanation methods across four method families and five explained models. They assess explanation quality and model sensitivity to concept interventions, typically averaging scores over concept changes.

  • LIBERTy experiments benchmark local and global explanations and analyze which models better capture causal effects of concept changes.
  • The five explained models are DeBERTa-v3, T5, Qwen-2.5, Llama-3.1, and GPT-4o, spanning fine-tuned and zero-shot models.
  • The eight methods cover counterfactual generation, matching, concept erasure, and concept attributions.
  • The Mediators and Confounders prompting technique is the main counterfactual-generation focus and achieves the best performance among the examined prompting techniques.

7 Results

LIBERTy evaluates local and global concept-based explanations against structural causal effects across three datasets and five models. Matching methods perform best overall, but results show substantial room for improvement and reveal differing model sensitivity to concept interventions.

  • Local explanations: Fine-tuned matching achieves the strongest local explainability results, with FT Match attaining the lowest estimation error and highest faithfulness.FT Match fine-tunes an encoder-only model for label prediction and uses its embeddings for similarity.
  • Local explanations: LLM-generated counterfactuals outperform matching on CEBaB, whereas matching performs better on LIBERTy because its candidates align with the underlying data-generating process.Human-written counterfactuals in CEBaB favor textual editing, while LIBERTy evaluates faithfulness to structural interventions.
  • Local explanations: 0.3 ED and 0.7 OF are achieved by even the best LIBERTy methods, despite ED favoring 0 and OF favoring 1, indicating substantial improvement potential.ED is Error-Distance and OF is Order-Faithfulness.
  • Global explanations: Global trends mirror local results: matching outperforms other methods, yet every method misses at least one gold top-three concept.Global Order-Faithfulness compares method rankings with rankings derived from gold concept-importance scores.
  • Concept sensitivity: LIBERTy measures model sensitivity by comparing prediction changes from structural interventions with causal effects estimated from the SCM.Sensitivity scores sum absolute ICaCE values, with larger values indicating stronger prediction shifts.
  • Concept sensitivity: Zero-shot Llama-3.1-8B and GPT-4o show lower sensitivity to demographic concepts, while Qwen2.5-1.5B most accurately reflects the causal structure among fine-tuned models.The reduced demographic sensitivity is attributed by the authors to intentional post-training alignment choices, and fine-tuning still leaves a gap from gold effects.

8 Conclusions

The paper introduces LIBERTy to address the lack of reliable explainability evaluation without gold explanations. Its findings show that structural counterfactuals expose room for improvement in explanations and reveal limited demographic sensitivity and causal-structure learning in some models.

  • LIBERTy benchmarks concept-based explanations, model sensitivity, and causal learning using interventional datasets with structural-counterfactual reference effects.
  • LLM-generated counterfactuals lose their previously reported state-of-the-art status when evaluated against structural rather than human-written counterfactuals.
  • Large room for improvement remains in both local and global explanations, providing targets for future research.
  • Some LLMs are largely insensitive to demographic interventions, while vanilla fine-tuning may fail to capture the data’s causal structure.

9 Limitations

LIBERTy’s limitations concern synthetic text, restricted explainability scope, and simplified causal data-generating processes rather than the evaluation protocol’s internal validity.

  • Synthetic Text Generation: Synthetic LLM-generated text may differ from how humans naturally express concepts, despite human validation finding the outputs coherent, relevant, fluent, and realistic.Annotators also confirmed that counterfactuals differed in only one concept.
  • Focusing on Concept-based Explanations: The framework evaluates only concept-based explanations, excluding token-level and free-text explanation methods.Concept-based methods are motivated by their focus on high-level, human-interpretable concepts.
  • DGPs as Approximations of Reality: LIBERTy’s DGPs and causal graphs simplify real-world mechanisms and do not perfectly mirror real-world causal structures.The authors state that the benchmark instead measures effects within the explained model.
  • DGPs as Approximations of Reality: The benchmark does not claim to explain real-world phenomena or reveal how LLMs internally represent them.Its stated goal is comparing explanation methods and identifying those that faithfully capture model behavior.

A.1 Real-World Data

The discussion positions LIBERTy as a controlled benchmark for model-relative faithfulness, while noting that deterministic generation and causal-inspired learning introduce practical constraints and opportunities.

  • Real-World Mechanisms: LIBERTy’s SCMs need not reproduce real-world mechanisms because faithfulness is defined relative to the explained model.The benchmark targets comparison of explanation methods and their capture of model behavior, including behavior based on spurious correlations.
  • Deterministic Decoding: Deterministic decoding is required because stochastic token sampling introduces uncontrolled generation noise that prevents structural counterfactuals.Stochastic generation can also produce entirely new narratives rather than minimally changed texts.
  • Counterfactual Generation: LLM-generated counterfactuals can mimic human-written references while failing against LIBERTy’s structural counterfactuals because their edits use heuristic assumptions.Human-written-counterfactual benchmarks may therefore assess imitation of human editing rather than causal effects.
  • Explainability Opportunities: The findings identify substantial room for improving local and global explanations and motivate causal-inspired explanation methods.LIBERTy is presented as an evaluation ground for developing such methods.
  • Causal Learning Opportunities: Vanilla fine-tuning may fail to capture the causal structure of the data, motivating learning methods explicitly aligned with the underlying DGP.The authors frame this as an opportunity for future causal learning research.

B Human Validation

Human validation assessed generated text quality, concept-value incorporation, logical consistency, and counterfactual plausibility using annotated single texts and text pairs.

  • Evaluation Criteria: The validation measured concept inclusion, linguistic quality, task relevance, logical consistency, and whether counterfactuals appeared genuine.The study used five text-level attributes and one comparative counterfactual attribute.
  • Evaluation Setup: 13 annotators produced 2,057 labels across 349 single-text and 312 text-counterfactual-pair evaluations.The average inter-annotator agreement was 0.91 across dimensions.
  • Validation Results: Coherence averaged 4.79/5, fluency 4.85/5, task relevance 4.77/5, and logical consistency 4.92/5.These scores indicate high average ratings across the listed quality dimensions.
  • Validation Results: 94.2% of concept values were correctly incorporated on average, while counterfactual plausibility averaged 4.44/5.The lowest scores occurred in CV Screening, probably because socially sensitive concepts were more heavily filtered during generation.

C Explainability Methods

The study compares counterfactual generation, matching, concept erasure, and attribution methods, with applicability varying by dataset and explanation setting.

  • Counterfactual Generation: Counterfactual-generation methods use an LLM to modify a target concept, with prompts optionally fixing other concepts, confounders, or mediators.Gemini-1.5-Pro was evaluated on CV Screening with DeBERTa-base, T5-base, and Qwen2.5-1.5B.
  • Matching: Matching methods retrieve candidate texts using semantic similarity or similarity of remaining concept values, typically with top-k matching where k = 3.Semantic matching uses text embeddings, while concept-based matching compares concept-value information.
  • Concept Erasure: Concept erasure removes linearly encoded information about a target concept from internal representations and compares predictions before and after removal.LEACE minimizes distortion to other directions while producing an erased representation.
  • Concept Erasure: LEACE is restricted to Disease Detection because its absence assumption does not fit concepts that change between two non-null states.The other datasets include concepts such as gender and occupation, for which erasure is ill-defined under that assumption.
  • Concept Attribution: ConceptShap and TCAV quantify concept influence in model activation spaces and are evaluated only for global explanations.They measure effects across a dataset rather than for individual inputs.
  • Dataset Construction: The Workplace Violence dataset models nurse interviews and predicts violence risk using a structured HR-interview template with concept-specific question banks.Templates sample one question per concept together with opening and closing variants.

E.1 Explainability Methods

The evaluation uses concept classifiers and compares explanation methods across five predictive models, three datasets, and multiple counterfactual prompt formulations. Implementations include LEACE and ConceptShap alongside standardized model-specific prompting and reproducibility details.

  • Shared evaluation components: Concept classifiers map each input–concept pair to a discrete concept level and are trained on the same 500-example subset for fair comparison.The classifiers use a 90%–10% train–validation split and fine-tune microsoft/DeBERTa-v3-base.
  • Explanation methods: LEACE estimates a separate linear erasure operator for each concept from final-layer hidden states and applies it at inference time.Concept labels are encoded as one-hot vectors while fitting the erasure operators.
  • Explanation methods: ConceptShap learns fixed concept vectors with TCAV and adapts a PyTorch implementation to use them consistently across methods.The setup supports the same three backbone models as the comparison framework.
  • Explained models: The explanation methods operate on DeBERTa-v3-base, T5-base, Qwen2.5-1.5B-Instruct, GPT-4o, and LLaMA-3.1-Instruct.Each model receives a dataset-specific input prompt suited to its interface and prompting requirements.
  • Reproducibility: The supplementary implementation materials report hyperparameters, predictive performance, and prompt templates for the three datasets.These details are provided in Tables 10–13.
  • Counterfactual generation: Counterfactual generation evaluates four prompt formulations, each encoding distinct assumptions about which concepts may change.The formulations constrain interventions to preserve causal coherence.

F Additional Results

Additional results report full outcomes for local and global explainability and concept sensitivity across workplace violence, disease detection, and CV screening. The section also documents counterfactual prompts, CV evaluation criteria, and annotation guidelines for validating generated examples.

  • Local Explainability: Local explainability reports Average ICaCE Error-Distance and Order-Faithfulness across five models and three datasets.Error-Distance is lower-is-better, while Order-Faithfulness is higher-is-better.
  • Global Explainability: Global explainability reports mean Order-Faithfulness for each model and dataset, with bold values identifying the best method per column.The results are aggregated in Table 16.
  • Concept Sensitivity Analysis: Concept sensitivity scores average summed absolute ICaCE magnitudes across examples and concept changes, alongside SCM ground-truth sensitivity.Disease Detection is an exception because disease Y is the parent of the symptom concepts.
  • CV screening evaluation: The CV classifier assigns scores 0, 1, or 2 using professional and demographic information from each candidate’s self-statement.The prompt maps these values to Regular, Good, and Exceptional classifications.
  • Counterfactual generation: Generated counterfactuals are requested to change a target concept while returning only the updated CV statement.The prompts differ in whether other concepts must remain unchanged or may require adjustment.
  • Counterfactual generation: Counterfactual prompts vary whether confounders remain fixed and whether mediators or causally linked concepts may be adjusted for logical consistency.The supplementary prompts include Fix Confounders, Fix All, and Mediators and Confounders formulations.
  • Annotation guidelines: CV annotations assess missing personal concepts and rate fluency, coherence, relevance, and consistency on a 1-to-5 scale.The guidelines define fluency as grammatical smoothness, coherence as logical connectedness, relevance as topical fit, and consistency as alignment with itself and external facts.
  • Annotation guidelines: Figure guidelines separately validate concept values, text quality, task relevance, logical consistency, and plausibility as genuine counterfactuals.The figures illustrate annotation procedures for the CV screening dataset and counterfactual plausibility.
Loading 2601.10700v2…