PaperlayerView exact revision on arXiv

Source-linked AI summary

Causally Fair Generation with Large Language Models

Patrik Okanovic, Torsten Hoefler, Drago Plecko

arXiv:2610.04444v1cs.AIcs.LGstat.ML

TL;DR

LLM-generated information can shape consequential decisions, yet causal dependencies among requested outputs may not follow the order in which a prompt supplies them. CFG supports user-selected causal-effect removal while retaining designated pathways; it is evaluated across four LLMs and three real-world settings for effect reduction, non-topological queries, and retention of permitted pathways.

  • Problem

    LLM-generated attributes can shape consequential decisions, while causally linked requested outputs may arrive outside causal order; statistical output filtering can also erase pathways an analyst wants to retain.

    Source: §1 · pp. 1–3

  • Method

    CFG is an interactive method in which users specify causal effects for removal and receive an adapted population model that retains pathways exempted as business-necessary.

    Source: §1 · p. 3, §2.2 · p. 7

  • Results

    Across four LLMs and three real-world settings, CFG is evaluated for selected-effect reductions at intermediate and final attributes, non-topological queries, and retention of permitted pathways.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Unlike statistical output filtering, fair generation can target selected disparity sources while retaining pathways judged permissible.

    Source: §1 · p. 2

    Fairness conclusions depend on causal-diagram and business-necessity judgments and representative population data; reducing evaluated effects does not ensure content is free of stereotypes or other harms.

    Source: ETHICS STATEMENT · p. 11

Abstract

from arXiv · show

Large language models (LLMs) are increasingly used to generate, complete, and transform information in settings where their outputs can shape consequential decisions, raising concerns about their impact on demographic disparities. In this context, causal inference provides a principled basis for assessing fairness, because it attributes observed disparities to the mechanisms that generated them, which a purely statistical approach cannot do even with infinite data. In LLM generation, a query may request several causally related variables, each of which is both an outcome of interest and a possible cause of other outputs, and the information supplied in the prompt need not follow a topological or a temporal order. This calls for methods that can analyze and selectively remove disparities from such a flexible generation process. In this paper we introduce Causally Fair Generation with LLMs (CFG, for short). CFG extracts relevant concepts, grounds generation in a reference population and causal diagram, and removes user-selected causal effects. CFG also allows pathways deemed justifiable for the task's utility to be retained, which is known in legal literature as business necessity. Further, we provide formal guarantees for our method when eliminating all discriminatory causal effects in the adapted population model, under appropriate causal assumptions. We evaluate CFG with four LLMs in three real-world settings based on population data and on a synthetic dataset with a known causal ground truth.

1 INTRODUCTION

LLM-generated content can influence consequential decisions and reproduce demographic disparities, while statistical comparisons alone cannot identify the mechanisms behind them. The paper formulates fair generation for causally related outputs and introduces CFG to remove selected effects while retaining permitted pathways.

1 INTRODUCTION

  • LLM outputs can shape information used by people or downstream systems, and statistical group differences alone cannot distinguish direct protected-attribute effects from inherited or background-induced disparities.The same observed difference can arise through different causal mechanisms.Source: §1 · p. 1
  • Fair generation must handle multiple generated attributes that can each mediate effects on later outputs, even when a prompt supplies later attributes and requests earlier ones.The attributes are arranged topologically in the causal model, but the prompt need not follow that order.Source: §1 · pp. 2–3
  • In the applicant-profile example, the analyst removes sex's direct effects on employment and salary while retaining education-mediated pathways, which statistical output filtering would erase.Education and employment are both generation targets and mediators of effects on salary.Source: §1 · p. 2
  • CFG combines a formulation for topological and non-topological fair-generation queries with an interactive method that grounds concepts in population data and a causal diagram, removes selected effects, and permits business-necessity exceptions.The paper also provides formal guarantees when all discriminatory causal effects are removed under appropriate assumptions.Source: §1 · p. 3
  • CFG differs from statistical bias audits, causal-model elicitation, and causal fairness methods for single-outcome prediction by targeting selective causal-effect removal across multiple generated attributes.The paper characterizes this task as fair generation, with pathways retained when deemed permissible.Source: §1 · pp. 2–3

1.1 PRELIMINARIES

  • The paper uses structural causal models to represent variables and their assignments, and decomposes observed outcome variation into direct, indirect, and spurious components.Direct and indirect effects distinguish paths through mediators from paths that do not pass through them.Source: §1.1 · pp. 3–4

2 CFG: CAUSALLY FAIR GENERATION WITH LLMS

CFG adapts population generation to user-selected causal-effect removals while supporting both topological and non-topological queries. Under recursive substitution, it gives distinct fairness guarantees for each query type.

2.1 TOPOLOGICAL & NON-TOPOLOGICAL QUERIES

  • CFG removes user-selected direct, indirect, or spurious effects while business-necessity sets retain directed paths through designated variables; backward closure suffices to identify permitted path effects.The analysis assumes an SFM-compatible process and a Markovian structure among mediators; non-topological queries condition on the fair-world distribution without a causal interpretation of that conditional.Source: §2 · p. 4, §2.1 · pp. 5–6

2.2 HARNESS FOR CAUSALLY FAIR GENERATION

  • CFG's inference-time harness analyzes supplied and requested concepts, grounds requested attributes in population data, obtains a causal diagram, adapts the population model, and inserts sampled values into the LLM prompt.The diagram is obtained from a curated repository or a second LLM-based agent, and the sampled covariates are checked against the generated story.Source: §2.2 · pp. 6–7

2.3 THEORETICAL GUARANTEES

  • For topological queries, recursive substitution makes the generated-record distribution invariant to supplied sex conditional on age; independent supplied sex and age strengthen this to independence from the complete record.These guarantees assume the user's supplied values satisfy the stated distributional condition for the stronger result.Source: §2.3 · p. 7
  • For non-topological queries, generated sex is insensitive to supplied education, employment, and salary once age is fixed; the complete population matches the debiased joint when supplied inputs follow that world.The pointwise guarantee does not require supplied age and outcomes to come from the debiased population.Source: §2.3 · p. 7

3 EXPERIMENTS

The experiments test whether CFG matches reference-population causal effects and removes selected effects across query types, datasets, and LLMs. CFG better matches reference effects than baselines, while selected effects move toward zero; business-necessity exceptions retain permitted pathways.

3 EXPERIMENTS

  • The evaluation covers three real-world survey settings and Census-Synth, using four generators and Vanilla, chain-of-thought, and retrieval-augmented baselines.The study assesses causal effects at intermediate and final generated outcomes; RAG and CFG use the same fitted population model.Source: §3 · p. 8

3.1 MATCHING REFERENCE CAUSAL EFFECTS

  • CFG matches reference causal effects better than baselines under both query types, with errors concentrated near zero while baseline errors are broader and larger.The comparison pools absolute errors in TV, DE, IE, and SE across generators, datasets, and targets.Source: §3.1 · p. 9

3.2 REMOVING DIRECT, INDIRECT, AND SPURIOUS EFFECTS

  • For both query types, CFG brings selected effects toward zero across tested models; TV confidence intervals include zero in all 20 empty-BN comparisons when all effects are removed.Figure 4 covers topological queries with empty BN sets in rows 1–3, a non-topological query in row 4, and a topological query with a BN requirement in row 5; intervals are 95% confidence intervals.Source: §3.2 · p. 9, Figure 4 · p. 9
  • With education as a business-necessity exception, topological and non-topological salary queries retain larger indirect effects and total variation than full effect removal.The remaining salary gap can reflect the analyst’s chosen exceptions.Source: §3.2 · pp. 9–10
  • CFG assumes relevant, representative population data and a credible causal diagram; LLM-proposed diagrams may need domain-expert correction.Fairness conclusions also depend on these assumptions, and reducing evaluated causal effects does not guarantee freedom from stereotypes or other harms.Source: §3.2 · p. 10, ETHICS STATEMENT · p. 11

SUPPLEMENTARY MATERIAL: Causally Fair Generation with Large Language Models

The supplementary material provides a repository for reproducing the results and reports the computational setup.

SUPPLEMENTARY MATERIAL: Causally Fair Generation with Large Language Models

A.1 PROOF OF THM. 1

The proof uses independence of downstream exogenous variables to establish a conditional counterfactual independence, then relates adapted conditional distributions to the reference-world distribution.

A.1 PROOF OF THM. 1

  • Under the standard fairness model, downstream exogenous variables are independent of those generating X and Z, yielding Yx0 independent of X conditional on Z.The proof states that Yx0 is a function only of Z and the downstream exogenous variables.Source: §A.1 · p. 16
  • Using consistency and the conditional independence result, the adapted conditional for Y given X and Z equals its reference-world conditional at x0.The displayed proof gives adapt(y | x, z) = P(Yx0 = y | X = x, Z = z) = P(Yx0 = y | Z = z) = Prw(y | x0, z).Source: §A.1 · p. 16
  • When supplied X and Z are independent, the resulting joint factorizes into a distribution of X and a distribution of (Z, Y), giving X independent of (Z, Y).The proof also gives a reverse-direction argument for the non-topological population statement.Source: §A.1 · p. 16

A.2 BUSINESS-NECESSITY POLICIES

The supplement discusses identification conditions for business-necessity pathways, hybrid completion with supplied descendants, and the Census-Synth causal model used for evaluation.

A.2 BUSINESS-NECESSITY POLICIES

  • Business-necessity path effects are identifiable under a sufficient backward-closure condition; specifications outside the stated conditions require stronger assumptions, including quantile preservation.Backward closure requires including a parent affected by X whenever that parent belongs to a selected BN variable’s ancestry.Source: §A.2 · p. 16

A.3 HYBRID COMPLETION QUERIES

  • For hybrid completion, user-supplied descendants may encode uncontrolled X relationships; fair-world inputs allow CFG to sample the remaining attributes from the adapted conditional.With real-world supplied descendants, appending only a recursively substituted suffix mixes factual and counterfactual components; overwriting supplied descendants moves the full outcome vector to the recursively substituted world.Source: §A.3 · p. 17, Figure 5 · p. 17

B CENSUS-SYNTH STRUCTURAL CAUSAL MODEL

  • Census-Synth uses an ordinal structural causal model with a shared latent cause of sex and age; Monte Carlo counterfactual calculations provide its ground-truth effect references.The model specifies direct and indirect sex effects on salary, including pathways through education and employment.Source: B CENSUS-SYNTH STRUCTURAL CAUSAL MODEL · p. 18, Figure 6 · p. 19

C.1 TARGETS AND EVALUATION PROTOCOL

The protocol evaluates CFG on intermediate and final outcomes across four datasets, using specified protected-group contrasts and outcome events. It estimates reference and generated effects, then applies selected removal settings under topological or non-topological queries.

C.1 TARGETS AND EVALUATION PROTOCOL

  • Four datasets contribute one intermediate and one final target each, with later outcomes excluded when evaluating an earlier target.The targets are income and monthly marijuana use for NSDUH, obesity and diabetes for BRFSS, employer type and salary for ACS, and employment and salary for Census-Synth.Source: §C.1 · p. 19
  • The protected contrast is Black or Hispanic versus White for NSDUH and BRFSS, and female versus male for ACS and Census-Synth.The first group is encoded as x1 and the second as x0; the protocol specifies seven intermediate and four final outcome events.Source: §C.1 · p. 19
  • Effects in generated stories use a cross-fitted one-step estimator, while Census-Synth reference effects use Monte Carlo approximation to the SCM ground truth.The three survey datasets use survey-weighted samples of 8,192 records for reference effects; intervals quantify estimator uncertainty, not variation across generation runs.Source: §C.1 · p. 19
  • Effect-removal settings are applied separately using fairadapt; non-topological generation fits a reverse conditional to the debiased joint before sampling and verbalizing.The non-topological procedure is evaluated for Census-Synth employment with an empty BN set and salary with B = {education}.Source: §C.1 · p. 19

C.2 ANNOTATING GENERATED STORIES

Generated attributes are recovered from stories by a structured annotation procedure, and CFG’s sampled values largely agree with those annotations. The section also defines prompting baselines and rules for aggregating errors.

C.2 ANNOTATING GENERATED STORIES

  • Stories are annotated attribute by attribute, and the first candidate with valid categories for every requested attribute is selected; unresolved attributes remain missing if none qualifies.The selected categories, story, prompts, raw responses, explanations, and sample identifier form one analysis row; supplied values remain query values.Source: §C.2 · p. 20
  • 98.73%–99.99%: sampled–annotated agreement across four generators on the shared topological recovery combinations.Each generator contributes 49,152 selected stories and 294,912 sampled–annotated pairs.Source: §C.2 · p. 21, Table 1 · p. 21

C.3 BASELINE CONSTRUCTION

  • Vanilla, chain-of-thought, and retrieval-augmented generation vary prompting instructions or supplied probabilities, but do not adapt the population distribution to remove selected causal effects.In recovery, CFG samples a complete record conditional on supplied attributes, preserving dependencies among requested attributes, whereas RAG presents conditional marginal probabilities.Source: §C.3 · p. 21

C.4 AGGREGATION

  • Aggregation weights datasets equally after averaging ACS employer events, and removal summaries also weight generators equally on matched event–generator pairs.Contrasts with only one protected group after filtering are excluded; weak overlap can produce estimates outside the probability scale and different available counts across methods.Source: §C.4 · p. 22

C.5 MATCHING REFERENCE EFFECTS ACROSS OUTCOMES AND QUERY ORDERS

CFG more closely matches reference causal effects than the prompting baselines across outcome positions and query orders. Its residuals are also more concentrated near zero in both query orders.

C.5 MATCHING REFERENCE EFFECTS ACROSS OUTCOMES AND QUERY ORDERS

  • 0.40–0.58 percentage points: CFG’s intermediate TV errors across generators; for Llama 3-8B, its 0.58 error compares with 6.53 for Vanilla and 6.93 for RAG.For final outcomes, CFG’s mean absolute TV errors range from 0.49 to 0.66 points across generators; Llama 3-8B’s error is 0.63, versus 4.14 for Vanilla, 8.92 for CoT, and 12.42 for RAG.Source: §C.5 · p. 22
  • Under non-topological queries, CFG’s TV errors are 1.58–1.68 points for intermediate and 1.66–1.80 for final outcomes; Llama 3-8B’s final error is 1.66 versus 10.27 for Vanilla, 7.33 for CoT, and 10.79 for RAG.Across both target positions, component errors remain at most 2.18 for DE, 0.88 for IE, and 0.53 for SE.Source: §C.5 · p. 22
  • CFG’s empirical cumulative distribution rises earlier under both query orders, indicating a larger share of absolute residuals concentrated near zero.The distributions include all 580 topological and 672 non-topological residuals; the histogram display cutoff excludes some plotted residuals but not those used in averages or cumulative distributions.Source: §C.5 · p. 22

C.6 ADDITIONAL EFFECT-REMOVAL RESULTS

Effect removal reduces selected effects across most topological comparisons, and removing all effects brings aggregate disparities and their components close to zero. Business-necessity exemptions preserve education-mediated pathways, leaving measurable disparity.

C.6 ADDITIONAL EFFECT-REMOVAL RESULTS

  • The selected effect decreases in magnitude in 169 of 176 topological comparisons; the seven exceptions are ACS events with reference effects at most 0.76 points in magnitude.At intermediate outcomes, targeting DE reduces mean absolute magnitude from 5.59 to 0.75 points, IE from 3.33 to 1.04, and SE from 0.76 to 0.09.Source: §C.6 · p. 23, Figure 9 · p. 25
  • 8.65 to 0.64 percentage points: mean absolute TV after removing all effects at intermediate outcomes; final-outcome TV falls from 6.56 to 0.54.Component residuals are 0.69, 0.40, and 0.09 points for intermediate outcomes and 0.40, 0.20, and 0.07 for final outcomes, so small TV accompanies small components rather than cancellation.Source: §C.6 · p. 24, Table 3 · p. 25
  • With B = {education}, mean absolute indirect effect remains near its 6.96-point ground-truth magnitude: 7.26 points for topological and 6.33 for non-topological queries.When all non-BN effects are selected for removal, mean absolute TV is 4.15 points for topological and 4.06 for non-topological queries, versus the 9.27-point reference magnitude; aggregates can include permitted contributions.Source: §C.6 · p. 25, Table 4 · p. 26

D CONCEPT AGENT EVALUATION

The concept agent extracts prompt variables and their specified or requested status before CFG grounds them in population data and a causal diagram. On a curated 100-prompt benchmark, Qwen3.8-27B achieves the strongest reported concept-detection scores, which are diagnostic rather than an independent estimate of arbitrary-prompt generalization.

D CONCEPT AGENT EVALUATION

  • Each concept record includes a normalized variable name, supplied value if any, specified/requested status, and an exact supporting prompt span.Requested concepts have null values and become generation targets; specified concepts provide conditioning information.Source: D CONCEPT AGENT EVALUATION · p. 26
  • The diagnostic benchmark contains 100 prompts and 528 gold records, divided among in-domain, out-of-domain, and challenge prompts.Out-of-domain means domains outside the paper's applications, not a statistically sampled population of user prompts.Source: D CONCEPT AGENT EVALUATION · pp. 26–27
  • Name F1 measures normalized concept-set recovery, while record F1 also requires matching status and normalized value.Evaluation additionally audits schema validity and whether each predicted evidence span appears verbatim in the prompt.Source: D CONCEPT AGENT EVALUATION · p. 27
  • Qwen3.8-27B achieves 97.9% name F1 and 94.3% record F1, with record F1 above 91% in every benchmark split.The results support using this model for concept detection in CFG, but the benchmark is not an independent estimate of generalization to arbitrary requests.Source: D CONCEPT AGENT EVALUATION · p. 27
  • Because prompts were curated and the alias dictionary refined after inspecting predictions, the results do not independently estimate generalization to arbitrary requests.The authors characterize them as a component diagnostic.Source: D CONCEPT AGENT EVALUATION · p. 27

E POPULATION FITTING COMPARISON

The population-fitting comparison tests conditional race-distribution estimates on NSDUH using survey-weighted held-out data. DRF has slightly better mean log loss and Brier score than XGBoost, supporting its use as CFG's primary population-fitting option.

E POPULATION FITTING COMPARISON

  • XGBoost fits categorical conditional probabilities, whereas DRF uses distributional splitting to assign query-specific weights to observed records.The comparison uses the same training and test partitions for both estimators within each split.Source: E POPULATION FITTING COMPARISON · p. 28
  • The comparison estimates race conditional on age, sex, education, income, and monthly marijuana use using 45,133 NSDUH records and five stratified 80/20 splits.The primary metric is survey-weighted held-out categorical log loss; the complementary metric is the multiclass Brier score, with lower values indicating better fit.Source: E POPULATION FITTING COMPARISON · p. 28
  • DRF achieves slightly lower mean log loss and Brier score than XGBoost in the NSDUH evaluation.Together with DRF's joint-record sampling mechanism, this supports its selection as CFG's primary population-fitting option.Source: E POPULATION FITTING COMPARISON · p. 28

F LLM CAUSAL DIAGRAM RETRIEVAL

The appendix evaluates causal-diagram retrieval and CFG's behavior when supplied attributes are conditioned on. It reports strong but assumption-dependent diagram agreement, low error in matching reference effects, and reduction of selected effects toward zero.

F LLM CAUSAL DIAGRAM RETRIEVAL

  • The diagram agent recovers baseline roles and a complete outcome order, which under the fixed SFM template determines the graph's edges.The benchmark uses reviewed reference assignments rather than uniquely identifiable ground truth from variable names.Source: F LLM CAUSAL DIAGRAM RETRIEVAL · p. 29
  • 85.0% of Astra's diagram presentations match exactly, compared with 60.0% for the strongest open-weight model, Gemma; Astra matches all presentations for ten graphs.Its nine non-exact responses occur only in ACS and moderation; a reversed pair can preserve much of a proposed structure while failing exact match.Source: F LLM CAUSAL DIAGRAM RETRIEVAL · pp. 30–31
  • Ambiguous variable names can yield different plausible roles and orders: Astra reverses one ACS outcome pair, while moderation responses reflect different meanings of message content.The authors recommend exposing roles and order for user or expert review; the chosen diagram remains a structural assumption for debiasing.Source: F LLM CAUSAL DIAGRAM RETRIEVAL · p. 31

G ILLUSTRATIVE CONVERSATION WITH CFG

  • The illustrative conversation lets users select business-necessity exceptions and direct, indirect, or spurious effects for removal before generation.Its final story illustrates output format; effect removal applies to the distribution used to generate stories, not to an individual story.Source: G ILLUSTRATIVE CONVERSATION WITH CFG · p. 32

H.1 MATCHING REFERENCE CAUSAL EFFECTS

  • CFG's mean absolute reference-effect errors are 1.59 percentage points for TV, 1.85 for DE, 1.19 for SE, and 0.73 for IE, lower than every baseline in the aggregate comparison.The aggregate averages over available datasets within each generator and then equally over the four generators.Source: Table 10 · p. 34, §H.1 · p. 34

H.2 REMOVING DIRECT, INDIRECT, AND SPURIOUS EFFECTS

  • Across both datasets and all four generators, CFG reduces the selected direct, indirect, or spurious effects toward zero.For spurious-effect removal, the supplied covariates are adjusted to a common reference-group distribution; direct- or indirect-effect removal alone leaves the supplied context distribution unchanged.Source: Figure 11 · p. 34, §H.2 · p. 34

H GENERATION CONDITIONED ON PROTECTED ATTRIBUTES AND COVARIATES

  • In the conditioned setting, the protected attribute and covariates are supplied to the generator, which produces all outcome attributes.The appendix evaluates topological queries on NSDUH and Census-Synth, with final outcomes of marijuana use and salary below $50,000, respectively.Source: H GENERATION CONDITIONED ON PROTECTED… · p. 34

Related papers

Loading 2610.04444v1…