Source-linked AI summary
From Causal Plausibility to Causal Reliability: Evaluating LLMs as Calibrated Direct Causal-Edge Classifiers
Amit Kumar, Elnur Adl Zarabi, Suranjana Trivedy, Zhiqian Chen, Lei Zhang, Kaiqun Fu, Taoran Ji
TL;DR
The paper asks whether LLM direct-edge judgments and confidence can be trusted when used in causal discovery. It evaluates 12 open-weight models across benchmark graphs, prompts, and confidence sources, finding dense recall-dominant predictions, frequent directness and orientation errors, and more promising—but not statistically superior—agreement-based confidence. The authors therefore position LLM judgments as externally validated soft priors rather than direct causal evidence.
Problem
Existing work does not systematically identify the structural sources of pairwise LLM edge-classification errors or establish whether confidence reflects correctness.
Method
The study evaluates pairwise direct-edge classification, graph reconstruction, calibration, and structural errors across 12 models, six graphs, five prompts, and four confidence sources.
Results
LLMs produced dense, recall-dominant graphs, frequently misclassified indirect and reversed relations as direct edges, and showed unreliable conventional confidence while agreement-based confidence had better mean calibration and discrimination.
Takeaways & Limitations
LLM judgments are better treated as externally validated soft causal priors than as direct evidence of causal structure.
Takeaways & Limitations
The language-only pairwise evaluation uses neither observational nor interventional data and does not enforce global constraints such as acyclicity.
Abstract
from arXiv · showhide
Large language models (LLMs) are increasingly used to provide prior causal knowledge for structural causal discovery, yet whether their direct-edge judgments and confidence can be trusted remains unclear. We systematically evaluate 12 instruction-tuned open-weight models across six benchmark causal graphs, five prompting strategies, and four confidence sources: verbalized, logit-based, cross-prompt agreement, and cross-model agreement. Under our language-only pairwise protocol, our evaluation yields three key findings. (i) LLM-based causal judgments are strongly recall-dominant: models predict overly dense graphs with many false-positive edges, while prompting mainly shifts the precision-recall trade-off rather than resolving overprediction. Gains from model scale diminish on the largest graphs and do not eliminate miscalibration. (ii) LLMs often capture causal relatedness without reliably identifying directness or orientation. Relative to published reference graphs, models misclassify 40.0% of indirect and 36.0% of reversed non-edges as direct edges, versus 28.2% of other non-edges. Moreover, 80.8% and 84.6% of these false positives receive verbalized confidence of at least 80%, revealing substantial overconfidence in structurally incorrect predictions. (iii) Conventional confidence estimates are unreliable, whereas agreement offers a more promising signal. Logit-based confidence frequently collapses near 1.0 regardless of correctness, while cross-prompt and cross-model agreement achieve better mean calibration and discrimination, though their advantages are not statistically significant after Holm correction. A benchmark-familiarity audit further identifies potential familiarity in five model-dataset pairs, all involving AsiaM. Overall, our results suggest LLMs are better viewed as sources of externally validated soft causal priors than as direct evidence of causal structure.
1 Introduction
The paper frames LLM direct-edge judgments as a reliability problem: models may recognize causal relatedness while confusing directness, orientation, and confidence. It introduces a calibrated evaluation of these judgments across models, graphs, prompts, and confidence sources.
- Interpretation: These results distinguish causal plausibility from reliable direct-edge identification, because relatedness can be recognized without preserving directness or orientation.The resulting overprediction is especially consequential in sparse graphs.
- Motivation: The study evaluates pairwise direct-edge classification as a calibrated task spanning edge quality, graph reconstruction, and confidence reliability.The task explicitly addresses structural ambiguity and severe class imbalance.
- Evaluation scope: The evaluation covers six benchmark causal graphs, five prompting strategies, 12 instruction-tuned models, and four confidence sources.The confidence sources are verbalized, logit-based, cross-prompt agreement, and cross-model agreement.
- Main findings: LLM edge judgments are recall-dominant and prompt-sensitive, with larger models gaining mean F1 on five of six datasets but only marginally on the largest graphs.Model scale does not resolve miscalibration, while prompting shifts the precision–recall trade-off.
- Main findings: 40.0% of indirect and 36.0% of reversed non-edges were predicted as direct edges, compared with 28.2% of other non-edges.More than 80% of false positives on indirect and reversed relations received verbalized confidence of at least 80%.
2 Evaluation Design
The evaluation treats LLMs as pairwise direct causal-edge classifiers whose predictions are reconstructed into graphs and assessed for classification, calibration, and structural errors. It varies prompt guidance and derives confidence from verbal reports, logits, and prediction agreement.
- Pipeline: The framework compares pairwise predictions with published reference graphs and aggregates positive ordered pairs into reconstructed directed graphs.Graph recovery is evaluated alongside structural error analysis and confidence calibration.
- Prompt Styles: Five prompt styles progressively add metadata, reasoning, labeled examples, and combined few-shot reasoning guidance.The styles are Name-only, Metadata, CoT, Few-shot, and Few-shot + CoT.
- Direct-edge classification: The primary task asks whether an ordered pair (A, B) represents a direct causal relation, using standardized YES/NO answers and confidence.Responses are parsed deterministically, with missing fields retried and fewer than 0.2% of responses excluded.
- Graph reconstruction: Reconstructed graphs do not enforce global structural constraints such as acyclicity.This preserves the direct connection between pairwise predictions and graph reconstruction.
- Confidence Estimation: Verbalized confidence is the model-reported score, while logit-based confidence normalizes decoder logits over the YES/NO label set.Both confidence sources are normalized for evaluating confidence–correctness alignment.
- Confidence Estimation: Cross-prompt and cross-model confidence use the fraction of predictions supporting the majority label across prompts or models.Cross-model agreement is computed separately within each model-scale group.
3 Experiment
The experiment evaluates causal-edge classification and confidence calibration across diverse models, graphs, prompts, and confidence sources, using metrics for both prediction quality and structural recovery.
- Models: 12 instruction-tuned open-weight models from five families are evaluated, grouped into small (4–14B) and large (31–72B) parameter ranges.The model families are Qwen, Gemma, Llama, Mistral, and Phi.
- Datasets: Six CausalGraphBench graphs span four size categories and medical, ecological, public-health, and industrial domains.The graphs are AsiaM, River Status, COVID, Coal Gasifier, Hepar2, and Munin1.
- Datasets: Approximately 2.5M pairwise queries are required because classification scales as n(n−1), motivating a controlled six-graph subset.The authors acknowledge that this subset may not capture the full diversity of the 35-graph benchmark.
- Task design: The pairwise task is highly imbalanced because only a small fraction of ordered variable pairs are true edges, with imbalance worsening as graphs grow.This motivates focusing on calibration because aggregate accuracy can mask overconfidence on sparse edge classes.
- Evaluation metrics: Performance is assessed with Precision, Recall, F1, normalized Structural Hamming Distance, squared-gap ECE, Brier Score, and AUROC.Lower ECE and Brier indicate better confidence–correctness alignment, while higher AUROC indicates stronger discrimination between correct and incorrect predictions.
- Prompting: Few-shot prompts use five fixed demonstrations, with three positive and two negative examples held constant across models.Direct-answer prompts allow 128 generated tokens and reasoning prompts allow 384, using temperature 1.0 and repetition penalty 1.1.
4 Results and Analysis
Across datasets, models remain recall-dominant and often overpredict direct edges; scale and prompting improve some outcomes without resolving systematic miscalibration. Agreement-based confidence is descriptively stronger, but benchmark familiarity and instance-level uncertainty limit interpretation.
- Precision–Recall Behavior: 94.4% of model–dataset combinations had recall exceeding precision, indicating a consistent recall-dominant pattern that produces dense graphs with many false positives.The pattern also occurred in 84.7% of model–dataset–prompt combinations.
- Prompt Sensitivity: Prompt styles shift the precision–recall trade-off, but no style consistently achieves a better balance across models and datasets.Few-shot, chain-of-thought, and combined prompting often increase recall without comparable precision gains.
- Dataset and Graph Difficulty: Mean F1 declines on larger, more imbalanced graphs, reaching 0.031 for both model groups on MUNIN1.Performance differences also align with linguistic accessibility: RIVER STATUS outperformed COAL GASIFIER and MUNIN1 across all models.
- Effect of Model Scale: Larger models improve mean F1 on five of six datasets, with gains of 0.478 to 0.680 on ASIAM and 0.280 to 0.403 on COVID, but performance is effectively unchanged on MUNIN1.Nearly all models remain recall-dominant, so scale improves recovery without eliminating systematic edge overprediction.
- Benchmark Familiarity: Five model–dataset pairs were flagged for potential benchmark familiarity, all involving ASIAM; recall-dominant overprediction also persists on the five unflagged datasets.The flagged pairs show varying structure recovery, so the audit indicates potential familiarity rather than uniform graph memorization.
- Confidence Reliability: Agreement-based confidence has better aggregate calibration and discrimination than verbalized and logit-based confidence, but its statistical superiority is not established after Holm adjustment.Cross-model agreement achieves the highest mean AUROC for small models, while cross-prompt agreement performs best for large models.
- Confidence Thresholding: Cross-model agreement achieves the highest edge AUPRC for both small and large models; a cross-prompt threshold of 0.8 reduces large-model graph density by 44.7% while leaving F1 nearly unchanged.The threshold increases precision from 0.250 to 0.331, while F1 changes from 0.314 to 0.310.
- Instance-Level Calibration: Agreement reflects stability across prompts or models rather than a calibrated probability of correctness, because confidence can be well calibrated in aggregate without separating individual correct and incorrect predictions.Correct predictions concentrate more strongly near full agreement, but reliability remains model- and dataset-dependent.
5 Conclusion
Across the evaluation, LLMs produced dense graphs and often confused causal relatedness with direct causal structure. Confidence from verbalized and logit-based sources was unreliable, while agreement-based methods showed better average calibration and discrimination without statistically significant superiority.
- Across 12 open-weight models, six benchmark graphs, and five prompting strategies, models were recall-dominant and produced dense graphs.Prompting shifted the precision–recall trade-off, while scale gains diminished on larger graphs.
- False positives concentrated on indirect and reversed relations, indicating failures to preserve directness or orientation.A three-way label formulation did not consistently improve graph recovery, so these errors were not solely artifacts of separate binary queries.
- Verbalized and logit-based confidence were unreliable, whereas agreement-based methods achieved better mean calibration and discrimination without established statistical superiority.The agreement-based pattern persisted on the five graphs not flagged for potential benchmark familiarity.
- The evaluation supports treating LLM judgments as externally validated soft priors rather than direct evidence of causal structure.
- Future Work: Future work should test graph-aware uncertainty methods, calibrated soft priors in data-driven discovery pipelines, smaller models, paraphrased prompts, and broader graph collections.
Limitations
The study’s conclusions are bounded by its six-graph sample, fixed reference structures, language-only pairwise formulation, and exploratory analyses. These constraints limit generalization and prevent establishing that every labeled false positive is causally invalid.
- The evaluation covers six of CausalGraphBench’s 35 graphs, which may not represent the benchmark’s full diversity or noisier real-world settings.
- Published structures are treated as fixed references despite possible expert choices about causal granularity and unavailable alternative annotations or agreement estimates.
- Not every prediction labeled as a false positive can therefore be established as causally invalid.
- The language-only pairwise formulation uses neither observational nor interventional data and does not enforce global constraints such as acyclicity.
- Semantic-accessibility analysis is heuristic and exploratory, while robustness to paraphrased wording is not tested within each prompting strategy.
- Results may depend on the selected open-weight models and decoding settings, and the familiarity audit cannot establish memorization or exclude familiarity with unflagged datasets.
A.1 Prompt Templates
The appendix defines five progressively richer prompt styles, from name-only queries to metadata-supported reasoning with few-shot examples. Each template elicits a direct-edge judgment and confidence, with some variants requesting reasoning.
- A.1.1 Name-only Prompt: Name-only prompting tests whether models can infer a direct edge from dataset context and variable names alone.
- A.1.2 Metadata Prompt: Metadata prompting adds variable definitions to clarify domain-specific meanings before asking for direct causation and confidence.
- A.1.3 Chain-of-Thought Prompt: The Chain-of-Thought template requests step-by-step reasoning before the Yes/No prediction and confidence value.
- A.1.4 Few-shot Prompt: Few-shot prompting supplies five labeled direct-edge examples demonstrating the task and required output format.The examples include both positive and negative causal judgments with confidence values.
- A.1.5 Few-shot + Chain-of-Thought Prompt: Few-shot plus Chain-of-Thought combines demonstrations with brief reasoning for each answer.Its output requests answer, confidence, and reasoning.
A.1.6 Three-Way Ablation Prompt
The three-way ablation asks the model to choose one of two directed-edge labels or an explicit no-edge label, together with confidence. It uses variable context and definitions but forbids explanatory reasoning in the output.
- The prompt asks for the direct causal relationship between Variable A and Variable B given context and definitions.
- The model must choose exactly one label: A_TO_B, B_TO_A, or NO_EDGE.The labels encode A-to-B direct causation, B-to-A direct causation, or no direct edge in either direction.
- The output includes confidence from 0–100% for the selected label.
- The template requires only the selected label and confidence, excluding reasoning or explanation.
A.2 Confidence Estimation Details
The paper defines four confidence sources for pairwise causal-edge predictions: verbalized, logit-based, cross-prompt, and cross-model agreement.
- Verbalized confidence is parsed from standardized answer–confidence responses using labeled fields and answer-first forms.
- Logit-based confidence aggregates accepted YES and NO surface forms before renormalizing over the binary label set.
- Cross-prompt agreement combines predictions from five prompt styles, while cross-model agreement is computed within fixed small- and large-model groups.
A.3.1 Dataset Statistics
The appendix documents benchmark statistics, metadata complexity, model and prompt performance, calibration analyses, confidence distributions, and figure-based performance comparisons across six datasets.
- Dataset Statistics: Reference-graph statistics include the no-edge-to-edge class imbalance over ordered variable pairs.
- Dataset Statistics: Metadata complexity is summarized using variable counts, code-like names, acronym-containing descriptions, and mean words per description.
- Classification Performance: Macro-averaged performance is reported separately for small and large models, with precision, recall, and F1 higher-is-better and normalized structural Hamming distance lower-is-better.
- Classification Performance: Primary edge-classification tables report precision, recall, F1, and normalized structural Hamming distance across six datasets and five prompt styles.Normalized structural Hamming distance is defined as SHD/(n_d(n_d − 1)).
- Confidence Calibration: Calibration tables cover prompt-specific, cross-prompt, and cross-model confidence results, using AUROC, ECE, and Brier metrics.Cross-prompt agreement combines five prompt styles; cross-model agreement combines models within each size group.
- Figures: Reliability diagrams and confidence-distribution figures visualize calibration across models, datasets, prompts, and confidence sources.The confidence-distribution figure distinguishes correct from misclassified pairs and reports verbalized and logit-based confidence separately.