PaperlayerView exact revision on arXiv

Source-linked AI summary

F-DACE: Fuzzy Disagreement-Aware Causal Evidence Fusion for Abstention-Safe Conversational Retail Decision Support

Sourish Dey

arXiv:2609.18238v1cs.LG

TL;DR

Observational decision-support systems can present one causal estimate even when defensible estimators disagree, and fluent interfaces can make fragile estimates persuasive. This paper pairs a causal-estimation engine with F-DACE, a fuzzy evidence-fusion policy that can abstain; its retail case withholds recommendations, and its conversational agent preserves causal status on evaluated questions.

  • Problem

    Observational decision-support systems may expose one causal estimate as a recommendation despite disagreement among defensible estimators, while fluent language can make fragile estimates persuasive.

    Source: §1 · p. 2

  • Method

    The paper combines a causal-estimation engine with F-DACE, which represents aligned evidence as fuzzy memberships and uses vetoes to produce recommendations or abstentions.

    Source: §10 · p. 15

  • Results

    The retail application withholds a recommendation despite apparently positive component estimates, and the agent preserved all 8 golden cases carrying abstain or error status, for 100.0% fidelity.

    Source: §1 · p. 2, §7.3 · p. 13

  • Takeaways & Limitations

    In the retail case, abstention is a decision result that prevents unsupported action rather than evidence of a failed predictor or causal analysis.

    Source: §8 · p. 14

    Shared unmeasured confounding that biases every component in the same direction remains undiagnosable by fusion.

    Source: §9 · p. 15

Abstract

from arXiv · show

Observational decision-support systems often expose one causal estimate as a recommendation even when plausible estimators disagree. The inherent engine of the proposed system is causal machine learning: a conditional-average-treatment-effect estimand identified by backdoor adjustment, estimated by an EconML DML causal forest and DoWhy linear regression, checked by two-way fixed effects, and converted into candidate levers by constrained optimisation. F-DACE is the decision layer on that engine. It represents precision, propensity overlap, placebo-refutation stability, interval overlap, and directional agreement as fuzzy memberships. Hard vetoes force abstention after estimand mismatch, failed diagnostics, informative sign conflict, or weak evidence. In 180 panel simulations spanning six identification conditions, F-DACE made a decision in 67.2% of runs and limited false recommendations to 17.2%; the corresponding rates were 33.3% for the causal forest and 35.6% for backdoor regression, matching deterministic unanimity rather than dominating it. Nearly all (30 of 31) false recommendations occurred under shared unmeasured confounding, which no fusion rule can diagnose when every component shares the omitted variable. The retail application aggregates a public Walmart panel to 6,435 store-weeks across 45 stores. F-DACE abstains for all five markdown indicators: some estimates are imprecise, one refutation fails, and MarkDown5 has a direct sign conflict. A LangGraph conversational agent exposes impact, what-if, and lever-optimization tools while a deterministic verifier preserves causal-layer status. On 24 live questions it achieved 100.0% tool-routing accuracy, 100.0% status fidelity, and 0.983 mean groundedness. On ten adversarial questions it resisted all injected instructions.

1 Introduction

F-DACE addresses disagreement among observational causal estimators by turning aligned evidence into a decision or mandatory abstention, while a conversational layer preserves that status. The introduction frames this policy alongside causal estimation, fuzzy evidence fusion, and conversational access to causal workflows.

1 Introduction

  • F-DACE converts aligned causal evidence into recommend, do-not-recommend, or mandatory abstention decisions, with vetoes that prevent strong estimates from overriding failed diagnostics or sign conflict.The conversational agent routes questions to the engine and cannot rewrite abstention as advice.Source: §1 · p. 2
  • The paper evaluates decision risk across six seeded identification conditions using false recommendations, decision coverage, selective error, and regret rather than estimator fit alone.Source: §1 · p. 2
  • The retail application withholds a recommendation despite apparently positive component estimates, while shared unmeasured confounding remains undiagnosable when it biases every component in the same direction.Source: §1 · p. 2

2.1 Causal machine learning and panel estimators

  • Causal machine learning and panel estimators provide distinct evidence because they address different failure risks, and F-DACE fuses them only after their estimands are aligned.The introduction identifies a DML causal forest, DoWhy backdoor regression, and a two-way fixed-effects check as engine components.Source: §1 · p. 2, §2.1 · p. 3

2.2 Fuzzy evidence and abstention

  • F-DACE uses fuzzy memberships to retain graded evidence and combines them with non-compensatory vetoes; unlike model averaging, it does not average effects into a new estimate.Source: §2.2 · p. 3

2.3 Conversational causal systems

  • The open systems question is whether conversational answer synthesis preserves the causal engine’s decision status when the engine refuses to recommend.The paper treats routing, scope continuity, grounding, and status fidelity as separate interface checks, not correctness guarantees for the underlying estimate.Source: §2.3 · p. 3

3 Causal machine learning engine

The causal engine estimates treatment effects under an explicit estimand and identification contract, then supplies aligned estimates and diagnostics for decision fusion. Its components cover backdoor identification, DML causal forests, constrained lever search, and a two-way panel check.

3.1 Estimand and identifying assumptions

  • The engine targets conditional and population-average treatment effects, interpreting the forest output as a CATE at observed covariates rather than an unobservable unit-level counterfactual.Source: §3.1 · p. 4
  • Identification from observational data requires unconfoundedness and overlap; adjustment on X blocks the assumed non-causal path from treatment to outcome through confounders.Source: §3.1 · p. 4

3.2 Graphical identification and refutation (DoWhy)

  • DoWhy identifies a median-binarized treatment contrast by backdoor linear regression and stress-tests it with placebo, random-common-cause, and subset refuters.The same linear contrast, stored as a per-unit causal beta, supports what-if impact estimates but is not itself a recommendation.Source: §3.2 · pp. 4–5

3.3 Double machine learning and the causal forest

  • DML residualizes outcome and treatment on learned nuisance functions, then a cross-fitted causal forest estimates heterogeneous effects from an orthogonal moment.Orthogonality limits the impact of small nuisance-model errors to second order; the implementation uses EconML CausalForestDML.Source: §3.3 · p. 5, Figure 2 · p. 6

3.4 From effects to candidate levers

  • For each leaf, constrained optimization allocates bounded lever changes to meet a target sales change, with an L2 objective favoring several gentle moves over one extreme move.The optimizer proposes candidates; F-DACE withholds any candidate whose required levers lack recommend status.Source: §3.4 · p. 6

3.5 Two-way fixed effects as a panel check

  • Two-way fixed effects absorbs persistent store differences and common period shocks, but is a third aligned estimate rather than an arbiter because treatment timing and heterogeneity can induce negative weighting.The specification uses store-clustered standard errors.Source: §3.5 · pp. 6–7

4 Fuzzy disagreement-aware causal evidence fusion

F-DACE preserves the engine’s estimand contract and maps its distinct causal estimates and diagnostics into fuzzy evidence without re-estimating or averaging the causal effect. Non-compensatory vetoes govern recommendation, rejection, or abstention.

4.1 Estimand contract

  • Components must share the treatment, outcome, analysis grain, population, and contrast; mismatched estimand contracts are rejected rather than fused.This prevents combining, for example, an effect per dollar with an effect of any markdown application.Source: §4.1 · p. 7

4.2 From engine outputs to fuzzy evidence

  • F-DACE maps each engine’s estimate, standard error, interval, overlap, and refutation evidence into memberships without re-estimating or averaging causal effects.The forest, backdoor OLS, and TWFE remain distinct statistical objects.Source: §4.2 · p. 7

4.3 Membership functions

  • Precision rises from zero at |τ̂ᵢ|/sᵢ = 0.5 to one at 1.96; directional memberships use normal-CDF transformations, and validity takes the weaker overlap or refutation score.Pairwise agreement averages standardized effect proximity (0.4), interval overlap (0.4), and sign agreement (0.2).Source: §4.3 · p. 7

4.4 Decision and veto rules

  • F-DACE recommends at S+ ≥ 0.65 and S− < 0.35, rejects under the symmetric negative condition, and otherwise abstains when evidence, agreement, diagnostics, or signs fail its thresholds.The decision flow applies a non-compensatory veto; thresholds were declared before the retail application and varied from 0.55 to 0.75 in sensitivity checks.Source: §4.4 · p. 7, Figure 3 · p. 8

5 Conversational causal decision-support agent

The conversational agent routes questions to typed causal tools while maintaining explicit scope across turns. A deterministic verifier preserves the causal layer’s status, and evaluation prioritizes objective routing and status measures over judge-based text-quality scores.

5.1 Stateful tool orchestration

  • A stateful ReAct graph carries conversation history through tool calls, while separate session state preserves store and department scope across turns.Explicit later-turn scope replaces remembered scope, and a ten-iteration limit bounds loops and API cost.Source: §5.1 · p. 8

5.2 Causal tool contract

  • Three typed tools handle variable impact, what-if scenarios, and constrained lever optimization, returning scope, support, estimates, diagnostics, and F-DACE status.Optimization candidates are withheld unless every required lever is recommendable; narrow or unsupported scopes return structured abstentions.Source: §5.2 · p. 8

5.3 Status-preserving answer synthesis

  • A deterministic verifier checks the latest tool result and replaces answers that fail to acknowledge error, no-solution, or abstention status.The architecture keeps decision authority with F-DACE and the verifier while the language model selects and explains.Source: §5.3 · pp. 8–9, Figure 4 · p. 9

5.4 Observability and evaluation signals

  • Evaluation records tool traces and uses routing accuracy, tool errors, and status fidelity as primary objective measures; judge scores assess text quality, not causal validity.Abstention-aware relevance treats a justified refusal as on-topic, while literal fulfillment is reported only as a diagnostic.Source: §5.4 · p. 9

6 Experimental design

The experimental design evaluates decisions across seeded synthetic identification conditions, a store-week Walmart application, and live conversational-agent tests. It includes comparator and adversarial evaluations alongside defined data and scoring protocols.

6.1 Synthetic panels

  • The synthetic benchmark uses 180 runs across six 24-unit, 30-period panels, with conditions spanning positive, negative, null, heterogeneous, poor-overlap, and time-confounded effects.The hidden confounder is withheld from all methods; comparisons include component estimators, unanimity, majority vote, and an oracle, using coverage, false recommendations, selective error, and regret.Source: §6.1 · pp. 9–10

6.2 Retail panel

  • The Walmart application aggregates department sales and markdown indicators to 6,435 store-weeks across 45 stores, matching the store-week decision grain.Department-level targeting is out of scope; the covariates and estimator-specific standard-error approaches are specified.Source: §6.2 · p. 10

6.3 Conversational-agent evaluation

  • The live evaluation uses 24 questions across three tool types and compares LLM routing with a deterministic regex/keyword baseline, plus a ten-question adversarial slice.Questions cover varied phrasing, scope, and change requests; adversarial prompts test instruction resistance, with saved answers checked by a separate judge.Source: §6.3 · p. 10

7 Results

Across synthetic simulations, F-DACE traded lower false-recommendation rates for reduced coverage; on the retail panel it abstained for every markdown. The agent routed questions correctly and preserved causal status, while adversarial testing exposed a conflict between safe refusal and task-completion metrics.

7.1 Synthetic validation

  • 17.2% false recommendations at 67.2% decision coverage contrasted with 33.3% for DML and 35.6% for backdoor OLS across 180 runs.F-DACE matched deterministic unanimity in 179 of 180 runs; under shared hidden confounding, every estimator and fusion policy recommended in all 30 replicates.Source: Table 1 · p. 11, §7.1 · p. 11

7.2 Retail application

  • All five markdowns remained abstentions at the default gate because evidence was weak or conflicting, including MarkDown2’s failed refutation and MarkDown5’s conflicting estimates.The 81-point threshold grid retained abstention in 63 configurations; recommendations for MarkDown3 and/or MarkDown4 appeared only after lowering evidence and support thresholds.Source: Table 3 · p. 12, §7.2 · pp. 12–13

7.3 Conversational-agent results

  • 100.0% routing accuracy, no tool errors, and 100.0% status fidelity were reported across 24 questions.Mean faithfulness was 0.983; the comparison with the deterministic router was limited to routing and argument extraction.Source: Table 5 · p. 13, §7.3 · p. 13
  • Answer relevance was 1.000 under an abstention-aware rubric, versus 0.804 overall and 0.438 on optimal-lever queries under literal fulfillment.The literal rubric penalized safe refusals where narrow scope left too few units for aligned multi-estimator inference.Source: §7.3 · p. 14
  • All ten injected instructions were resisted, although routing accuracy was 50.0% and answer relevance was 0.150 because safe refusals conflicted with benchmark compliance.The study reports both safety and task-completion measures to avoid classifying safe refusals as general agent failure.Source: Table 6 · p. 14, §7.3 · p. 14
  • 100.0% routing and correct remembered-scope resolution held across the four-turn follow-up evaluation, which did not test long-horizon dialogue.Source: §7.3 · p. 14

8 Discussion

F-DACE exposes graded evidence and vetoes for a tunable coverage–risk trade-off, but does not outperform unanimity on binary risk. The discussion distinguishes causal decision-making from the conversational layer that routes and explains decisions.

8 Discussion

  • F-DACE and unanimity agree on 179 of 180 runs; unanimity is slightly safer, while F-DACE exposes graded scores and vetoes for a tunable coverage–risk trade-off.Majority vote and inverse-variance pooling match DML’s false-recommendation rate in simulation and would recommend several retail markdowns withheld by F-DACE and unanimity.Source: §8 · p. 14
  • Bayesian forests, doubly robust scores, or modern difference-in-differences estimators can replace the statistical engines if they emit the same evidence contract.Experts can set asymmetric thresholds when false positives cost more than abstention.Source: §8 · p. 15
  • The agent contributes semantic routing and workflow continuity, while routing and wording evaluations assess the interface rather than causal identification or the fused decision.Perfect status fidelity coexists with flexible language routing, and adversarial results show that declining a tool call can be correct.Source: §8 · p. 15

9 Limitations

The paper identifies shared unmeasured confounding, observational-panel assumptions, policy thresholds, and limited conversational testing as boundaries. Its conclusion presents causal estimation and conversational agency as distinct-authority components; declarations describe funding, data, code, and AI assistance.

9 Limitations

  • Shared unmeasured confounding that biases every component in the same direction remains undiagnosable by fusion, so agreement is not proof of unconfoundedness.The hidden-confounder simulation makes this identification bound explicit.Source: §9 · p. 15
  • The observational retail panel assumes backdoor adjustment and overlap; unobserved promotions or demand shocks can still bias every estimator.Retail abstentions respond to residual risk but do not certify that identifying assumptions hold.Source: §9 · p. 15
  • 63 of 81 threshold-grid configurations remain all-abstain; transferring cut-offs across domains with different error and abstention costs requires local calibration.Only a lower-evidence, lower-support corner recommends MarkDown3 or MarkDown4.Source: §9 · p. 15
  • The conversational evaluation uses 24 golden questions, 10 adversarial prompts, one generator, and one judge as an implementation check, not a basis for causal claims.Causal claims rest on the synthetic decision-risk experiment and store-week retail application.Source: §9 · p. 15

10 Conclusion

  • The conclusion describes a hybrid architecture in which causal estimation and conversational agency are jointly evaluated but retain distinct authority.F-DACE abstains for all five markdowns, and the verifier prevents the conversational layer from overriding the causal decision.Source: §10 · pp. 15–16

Declarations

  • The declarations report no specific grant, identify the Walmart Kaggle competition as the data source, and state that reproduction scripts and evaluation artifacts accompany the submission.The author also reports using Cursor AI for implementation, restructuring, and language editing, with review and responsibility retained.Source: Declarations · p. 16

A Extended experimental results

The appendix identifies the scope of scenario-level benchmark tables and agent-evaluation tables, then records the orchestrator contract and function-calling schemas. The supplied passages describe table contents and schemas but do not provide table cell results.

A.1 Scenario-level synthetic results

  • Table 7 disaggregates the 180-run benchmark by scenario; coverage measures the fraction of replicates in which a method acted, and false recommendation measures action disagreement with the oracle.The hidden-confounding row is retained to show that agreement cannot repair an omitted cause shared by every component.Source: §A.1 · p. 18, Table 7 · p. 18

A.2 Agent results by query class

  • Tables 8 and 9 present golden-set results by tool/query class and detailed conversational-evaluation slices, respectively.The supplied captions identify their scope but do not state outcome values.Source: Table 8 · p. 18, Table 9 · p. 18

B.1 Orchestrator system contract

  • The orchestrator contract permits only human-reviewed recommendations, treats user override claims as untrusted, and requires tool-grounded outputs that preserve uncertainty and abstention.It specifies three tools and requires veto reasons when status is abstain.Source: §B.1 · p. 19

B.2 Function-calling schemas

  • Tool schemas accept target percentages, optional store and department scopes, variable lists, and what-if changes, with additional analysis and ranking controls.The listed controls include analyze_all_data, max_levers, top_n, and an optional date.Source: §B.2 · p. 19
  • The normalized result envelope includes execution status, analysis scope, a store-week contrast estimand, estimates with intervals and diagnostics, and an F-DACE decision.The schema names positive_support, negative_support, agreement, and veto_reasons as decision fields.Source: §B.2 · pp. 19–20

C LangGraph execution and observability

The LangGraph flow routes tool calls through execution and then a deterministic verifier before ending. Session state and optional tracing support scope continuity and observability.

C.1 State transition

  • The agent loops through ToolNode when tool calls are present, then routes to verify_response before END when no tool call remains or the iteration bound is reached.The graph records messages, iterations, final response, and session scope fields.Source: §C.1 · p. 20
  • Session state stores the current store/department list, last query scope, and last user question.The state-transition passage also lists the agent’s message and iteration state.Source: §C.1 · p. 20
  • LangSmith tracing is optional; enabled runs attach observability callbacks and judge feedback, while disabled runs write ordered messages and metrics to local JSON/JSONL artifacts.Table 10 is identified as the LangGraph/LangSmith observability schema.Source: §C.1 · p. 20, Table 10 · p. 20

D Curated conversational outputs

The curated examples show how the agent reports a what-if estimate, handles an observed abstention, and constrains recommendation outputs. No favorable markdown recommendation survived F-DACE in the retail application.

D.1 Tool calling and a quantitative what-if result

  • Baseline sales were $22,513.32, predicted sales were $22,513.32, and estimated impact was −$0.00072 for a 10% MarkDown1 increase.Source: §D.1 · p. 20

D.2 Recommendation-path contract

  • The recommendation template is a future contractual path, not an observed retail outcome, and requires reporting effect, interval, scope, and no production action.Source: §D.2 · p. 21

D.3 Observed abstention

  • The lever-optimization tool abstained because this store-department scope lacked enough units or treatment variation for aligned inference.Source: §D.3 · p. 21

D.4 Abstention override mechanism

  • The verifier replaces an unsafe lever draft after abstention, while the agent states it cannot change pricing, markdown, or inventory systems and offers prospective analysis only.Source: §D.4 · p. 21

Related papers

Loading 2609.18238v1…