PaperlayerView exact revision on arXiv

Source-linked AI summary

From the Task Boundaries of Narrative Text to Structural Anchoring, Uncertainty Triggers, and Cross-Calibration

Bowen Deng, Jiaqi Zou, Kexin Zhang, Daifeng Li

arXiv:2609.12453v1cs.HC

TL;DR

Causal graphs encode structure but leave users to interpret task-specific direction, mechanisms, and adjustment, while prior fixed-format comparisons reveal less about how users coordinate explanations. CoNS-Explorer compares fact-matched direct and story texts and studies how users switch among these texts and graphs. The studies identify a modest, task-dependent accuracy association for stories and a resource-routing process framework.

  • Problem

    Prior work often compares fixed explanation formats, leaving less evidence about how users distribute reasoning across graphs, direct explanations, and stories.

    Source: §1 · p. 2

  • Method

    CoNS-Explorer generates explanations from reviewed causal models, and complementary studies compare complete text packages and observe users freely selecting resources.

    Source: §3.1 · p. 4, §4 · p. 6, §6 · p. 14

  • Results

    Story text had a positive but modest association with accuracy, clearest in total-effect adjustment, while free-browsing participants routed local uncertainty to explanations and calibrated judgments across sources.

    Source: §1 · p. 2, §8 · p. 17

  • Takeaways & Limitations

    The framework organizes causal explanation around structural anchoring, uncertainty triggering, resource routing, and cross-calibration, with four testable design propositions.

    Source: §1 · p. 3

    The estimates compare complete presentation packages, so they do not isolate the contributions of terminology, temporal order, contextual goals, or cue salience.

    Source: §7 · p. 17

Abstract

from arXiv · show

Causal graphs represent structural relationships among variables, yet users must still interpret direction, mechanism, and adjustment conditions in relation to the task at hand. Prior work often compares explanation formats as fixed conditions and pays less attention to how users distribute reasoning across graphs, direct explanations, and stories. We developed CoNS-Explorer, which uses reviewed instructional DAGs/SCMs to maintain a shared causal-fact ledger and generate fact-matched direct explanations and contextualized stories. A controlled survey experiment ($N=240$) compared the two texts as complete presentation packages. In the primary GLMM, the Story condition had a positive but uncertain overall association with accuracy (OR $=1.55$, 95\% CI $[0.34,7.10]$, $p=.572$); a population-averaged GEE showed a significant positive effect (OR $=1.89$, 95\% CI $[1.02,3.48]$, $p=.042$). Task-type interactions localized the clearest advantage to total-effect adjustment. Story also significantly increased situational presence. In a separate system-task and interview study ($N=24$), participants freely used graphs, direct explanations, and stories across three causal models. They established structural anchors with graphs and numerical results, consulted text when direction was unclear, mechanisms were unfamiliar, or multiple paths competed, and checked their judgments against other representations or external evidence. Integrating the two studies, we develop a process framework of structural anchoring, uncertainty triggering, explanation routing, and cross-calibration, together with four testable design propositions for adaptive causal explanation.

1 Introduction

Causal graphs show structural relationships but do not by themselves answer task-specific questions about direction, mechanisms, or adjustment. This paper combines fact-matched text comparisons with observation of users switching among graphs, direct explanations, and stories to develop an explanation-routing framework.

1 Introduction

  • Causal graphs do not automatically resolve task-specific questions about direction, intervention, common causes and outcomes, or mediator adjustment.Source: §1 · p. 1
  • Prior comparisons often change factual content and presentation cues together, while fixed-format studies miss how users combine resources during interaction.Source: §1 · p. 2
  • CoNS-Explorer generates direct explanations and contextualized stories from the same causal-fact ledger, and two studies examine package effects and free resource switching.Source: §1 · p. 2
  • Story text showed a positive but modest accuracy association, clearest in total-effect adjustment; participants also used text when uncertainty arose and checked judgments across sources.Source: §1 · p. 2
  • The studies yield a framework of structural anchoring, uncertainty triggering, explanation routing, and cross-calibration, alongside four testable design propositions.Source: §1 · p. 3

2 Related Work

Prior work supports task-based evaluation, narrative explanations, and causal visualization, but leaves open how explanation resources complement one another during users’ reasoning.

2.1 Human-Centered XAI Requires Task-Based Validation

  • Human-centered XAI distinguishes task-valid understanding from subjective ratings, motivating objectively scored causal-discrimination performance as Study 1’s primary outcome.Source: §2.1 · p. 3

2.2 From Structured Explanations to Stories

  • Narratives can make abstract relationships imaginable, but characters, event order, context, and cue salience often change together, complicating comparisons with direct explanations.Source: §2.2 · p. 3

2.3 Causal Visualization and Interactive Causal Reasoning

  • Causal visualization supports structural exploration, while this paper focuses on how users select, switch, and verify across resources in response to current uncertainty.Source: §2.3 · pp. 3–4

3 CoNS-Explorer: Fact-Matched Causal Explanation

CoNS-Explorer compares direct and story explanations grounded in a shared reviewed causal model. Its design preserves core facts while allowing presentation cues to differ, and applies the materials to causal tasks in education and macroeconomics.

3.1 Comparing Expressions Under Fact Consistency

  • CoNS-Explorer generates both texts from a shared instructional package containing the reviewed model, intervention states, changes, boundaries, and question set.Source: §3.1 · p. 4

3.2 Two Explanation Formats

  • Direct text foregrounds variables and relationships for compact verification, whereas a story adds fictional characters, goals, a baseline, and event sequence.Source: §3.2 · p. 4, §3.1 · p. 4

3.3 Generation, Execution, and Verification

  • The system validates and executes the model, records results in a shared fact ledger, generates either text, and checks factual coverage and unsupported claims.Source: §3.3 · p. 5

3.4 Study Materials and Causal Scenarios

  • Education and macroeconomic scenarios support reasoning about confounding, outcome selection, and total-effect adjustment, with fixed interventions specified for each model.Source: §3.4 · p. 6

4 Study 1: Controlled Survey Experiment

Study 1 compared fact-aligned Direct and Story texts in a randomized survey, measuring accuracy on causal tasks and post-task experience ratings. Complementary analyses found a task-specific accuracy advantage and a clearer experiential change in situational presence.

4.4 Outcomes and Statistical Analysis

  • Overall correctness across six questions was the sole confirmatory outcome, analyzed with a binomial logistic mixed-effects model and complementary robustness and task-interaction analyses.The GLMM estimated a conditional effect while GEE estimated a population-averaged effect; six post-task ratings assessed experience.Source: §4 · p. 6, §4.4 · pp. 7–8

4.1 Design and Participants

  • The between-subjects survey randomly assigned 240 participants equally to Direct or Story; participants were fluent Chinese readers, and causal-inference training was not required.Both conditions covered education and macroeconomic scenarios.Source: §4.1 · p. 6

4.2 Procedure and Causal-Discrimination Tasks

  • Participants examined a graph, baseline state, and assigned text, answered questions before and after a fixed intervention, and completed a second scenario under the same text condition.Graphs, interventions, values, and questions were the same across conditions; only text format differed.Source: §4.2 · p. 6

4.3 Data Preparation and Quality Control

  • All 240 responses were retained; six binary questions yielded 1,440 participant–item observations, split evenly across confounding, outcome-selection, and total-effect-adjustment tasks.The answers were prespecified from reviewed DAGs/SCMs, and the audit found all objective responses and ratings complete.Source: §4.3 · p. 7

4.5.1 Boundary of the Overall Effect.

  • Story favored accuracy in both models: confirmatory GLMM OR = 1.55 (95% CI [0.34, 7.10], p = .572), while overall GEE OR = 1.89 (95% CI [1.02, 3.48], p = .042).The authors characterize the positive accuracy effect as limited in magnitude.Source: §4.5.1 · p. 8

4.5.2 Story Effects Varied Across Task Types.

  • Story’s clearest accuracy advantage was in total-effect adjustment: GEE gain 9.08 percentage points (95% CI [3.06, 15.10], Holm p = .009).The interaction appeared in both frameworks; cross-model comparisons showed a stronger Story effect for total-effect adjustment than confounding, and this task is identified as the priority for larger preregistered replication.Source: §4.5.2 · pp. 8–9, Figure 3 · p. 9

4.5.3 Story Significantly Increased Situational Presence.

  • Story significantly increased situational presence, with mean ratings of 5.77 versus 5.22 for Direct and cumulative-logit OR = 2.95 (95% CI [1.81, 4.79], Holm p < .001).Vividness, mental simulation, and narrative transportation were similar across conditions; the experiential change was concentrated in feeling situated in the scenario.Source: §4.5.3 · p. 9

5 Study 2: System Tasks and Interviews

In the free-browsing study, participants used graphs and intervention results to anchor structure, then routed local uncertainties to text or analytic support and checked judgments across representations. These observations informed a four-stage, iterative explanation-routing framework.

5.1 Participants and Research Setting

  • Study 2 recruited 24 independent participants for six interactive tasks across agriculture, education, and supply chains, with graphs, Direct text, and Story available to browse.Tasks covered mediator chains, competing paths, observation versus intervention, and mediator adjustment.Source: §5 · p. 9, §5.1 · pp. 9–10, §5.2 · p. 10

5.2 Three Domains and Six Interactive Tasks

  • Participants manipulated intervenable nodes, observed value and color changes, and judged results; retrospective prompts reconstructed observed switching before separate design-preference probes.The probes asked about Story’s mechanism information and desired LLM explanation timing and granularity.Source: §5.2 · pp. 10–11

5.3 Qualitative Analysis

  • Researchers manually developed themes from interview material using iterative coding, case comparison, and a second researcher’s recheck of source alignment and assignments.An LLM proposed initial codes, but researchers made the axial-analysis, theme-development, and other analytic decisions.Source: §5.3 · p. 11

5.4.1 Reasoning Began with Structural Anchors in the Graph and Intervention Results.

  • All 24 participants used a graph, interaction, color, numerical value, or arrow as a structural anchor, though their starting resources varied.Participants then sought text when the graph alone did not clarify direction, mechanism, or which relationship dominated.Source: §5.4.1 · p. 12

5.4.2 The Two Texts Supplied Different Missing Information.

  • Direct text supported verification of direction, adjustment conditions, and local logic, while Story helped explain unfamiliar relationships and contextualize mechanisms.Twenty of 24 participants used Direct text to verify details; 23 of 24 used Story or recognized its contextual or mechanistic value.Source: §5.4.1 · p. 12, §5.4.2 · p. 12
  • Twenty of 24 participants described switching when local uncertainty arose; familiarity and reading cost shifted how much explanation they sought.Unfamiliar domains increased text demand, while long or distracting text led participants to search selectively or return to another resource.Source: §5.4.2 · pp. 12–13

5.4.4 Competing Paths Prompted Cross-Checking and Requests for Controls.

  • Competing paths left structural anchors insufficient for some judgments, prompting requests for controls or trend comparisons and cross-checking against other representations or evidence.Fourteen of 24 participants calibrated judgments among the graph, texts, real-world experience, or external evidence; layered explanations were a stated preference, not observed switching behavior.Source: §5.4.4 · p. 13

5.5 An Explanation-Routing Process Framework

  • The framework describes an iterative cycle of structural anchoring, uncertainty triggering, explanation routing, and cross-calibration, with familiarity and reading cost shaping explanation use.Users may seek controls, local trends, or a dynamic chain when competing paths remain unresolved, then retain, revise, or challenge their judgment.Source: §5.5 · pp. 13–14

6 Discussion

The discussion integrates fixed-format experiment results with free-browsing evidence to frame explanation choice as matching resources to users’ current uncertainty. It proposes testable mechanisms and design implications for routing, verification, and evaluation.

6 Discussion

  • The two studies extend fixed-format comparisons by showing how users choose among resources in response to current uncertainty, with each resource serving different functions and carrying different costs.The integrated account distinguishes empirical findings from propositions for future testing.Source: §6 · p. 14

6.1 Story Supports Gaps in Context and Mechanism

  • Story significantly increased situational presence and was consulted around unfamiliar variables or mechanistic questions, while Direct text supported precise checking.Characters and plot raised reading costs for some participants, and familiarity reduced demand for Story.Source: §6.1 · p. 15

6.2 Total-Effect Adjustment Suggests a Testable Mechanism

  • Total-effect adjustment showed the clearest Story advantage; the authors propose testing whether process continuity matters more than general immersion for these tasks.The proposed mechanism is that Story’s sequence may help retain a mediating chain, while Direct text supports checks of direction, adjustment conditions, and model boundaries.Source: §6.2 · p. 15

6.3 Four Testable Propositions for Explanation Routing

  • The framework yields four propositions: users anchor structure first, route by local uncertainty, adjust explanation expansion by familiarity and reading cost, and cross-calibrate amid conflict or competing paths.The propositions map to observable behavior and implementable interface mechanisms for instrumentation and testing.Source: §6.3 · p. 15

6.4.1 Keep Structure Visible and Offer a Minimal Sufficient Explanation First.

  • Keep intervention structure visible and offer a short explanation first, with longer Direct text or Story available as expandable detail.This progressive disclosure is proposed to retain the graph’s navigational value while responding to long-text reading costs.Source: §6.4.1 · pp. 15–16

6.4.2 Route Explanations by Type of Uncertainty.

  • Route direction, adjustment, boundary, or wording questions to searchable Direct text; make Story available for unfamiliar terms, real-world meaning, or mechanisms.For simultaneous competing paths, the interface can support controls, mediator-chain highlighting, path comparisons, or trend displays.Source: §6.4.2 · p. 16

6.4.3 Support Cross-Checking and Contestability.

  • Because participants checked explanations against experience and external evidence, interfaces should link Story back to graph relations, values, model boundaries, and data sources.The authors also propose marking fictional characters and settings as contextual devices and allowing users to revisit structure or more precise mechanisms.Source: §6.4.3 · p. 16

6.5 Evaluate Situated Experience and Task Understanding Separately

  • Measure situated experience separately from task understanding, combining objective performance and subjective ratings with interaction traces such as switching and answer revision.Adaptive systems additionally need evaluation of whether explanation timing and granularity match the task.Source: §6.5 · p. 16

7 Scope and Future Validation

The authors delimit the evidence to short-term judgments on a small set of instructional causal tasks and identify priorities for broader validation. The texts were compared as complete presentation packages, not as isolated components.

Scope of Study 1

  • Study 1 used two reviewed instructional DAGs/SCMs and six diagnostic questions; generally easy questions limited group-difference detection, and two items per task type make task estimates directional.The authors propose harder, parallel total-effect-adjustment questions and a preregistered replication.Source: §7 · p. 16

Scope of Study 2

  • Study 2 covered three domains with 24 participants and fixed task order; future work can counterbalance order and use larger samples to test the propositions directly.The recurring patterns complemented the survey findings, but the fixed order made exploratory experiences comparable rather than counterbalanced.Source: §7 · p. 17

Presentation Packages and Measurement

  • Because terminology, temporal order, contextual goals, and cue salience differed, Study 1 estimates complete-text package differences; experience measures were single items in a short session.The authors suggest factorial studies to isolate components and item-level confidence paired with accuracy for fuller calibration analysis.Source: §7 · p. 17

8 Conclusion

The paper combines controlled and free-browsing studies to identify when Story text may help and how users coordinate explanation resources during causal-graph reasoning. It presents a routing framework and testable propositions for design.

8 Conclusion

  • Across the 240-person experiment and independent 24-person free-browsing study, Story’s clearest accuracy benefit was in total-effect adjustment, while it increased situational presence.Participants anchored judgments with graphs and values, routed uncertainty to text or analytic support, and checked against other representations or external evidence.Source: §8 · p. 17

A Overview

  • The appendix documents study-data, fact-matching, system-material, and interaction-trace verification questions, distinguishing appendix evidence from separate companion artifacts.Participant-facing screenshots remain in Chinese because the study was administered in Chinese.Source: A Overview · p. 20

B Study 1: Survey Data Processing

Study 1 used item-level causal-discrimination data for its primary analysis, while retaining total scores only as a robustness summary. Distinct causal operations and post-task experience indicators were analyzed separately rather than combined into unsupported scales.

B Study 1: Survey Data Processing

  • 1,440 participant-by-item observations formed the primary model dataset; six-item total scores served only as a robustness summary.The study analyzed 240 complete responses, with 120 participants per condition, against a frozen DAG/SCM answer key.Source: B Study 1: Survey Data Processing · p. 20
  • The six objective items covered three distinct causal operations, so the study did not treat them as one psychometric scale.The item groups covered confounding, outcome selection, and total-effect adjustment.Source: B Study 1: Survey Data Processing · p. 20
  • Narrative-experience indicators were analyzed separately after their composite showed insufficient internal consistency; trust and cognitive effort remained separate planned indicators.The narrative-experience composite had pooled α = .599.Source: B Study 1: Survey Data Processing · p. 20

C Study 1: Main Statistical Results

Study 1 found high accuracy and a positive but model-sensitive Story advantage, with the clearest task-specific difference in total-effect adjustment. Story’s association with accuracy therefore warrants interpretation alongside ceiling effects and uncertainty.

C Study 1: Main Statistical Results

  • 91.5% in Story versus 87.9% in Direct was the raw accuracy; 76.7% of participants answered all six items correctly, indicating a ceiling pattern.Four of six items had pooled accuracy at or above 90%.Source: C Study 1: Main Statistical Results · p. 21
  • Story’s effect was stronger for total-effect adjustment than confounding (GLMM ratio OR = 7.18; GEE ratio OR = 2.21), while the within-task advantage was model-sensitive.For total-effect adjustment, GEE estimated +9.08 pp (95% CI [3.06, 15.10], Holm p = .009); the supplied GLMM estimate is incomplete.Source: C Study 1: Main Statistical Results · p. 21
  • The Story × Task Type interaction was supported in both frameworks: GLMM likelihood-ratio χ2(2) = 8.06 and GEE robust Wald χ2(2) = 7.02.The corresponding p-values were .018 and .030.Source: C Study 1: Main Statistical Results · p. 21

D Study 1: Experimental Materials

The experimental materials held causal content constant while varying explanatory expression. CoNS-Explorer separates deterministic scenario and fact generation from the paired language forms, with static scenarios used at runtime.

D Study 1: Experimental Materials

  • Direct and Story shared causal structures, values, interventions, questions, and answer keys, differing only in explanatory expression.Paired texts were length-matched within 10%.Source: D Study 1: Experimental Materials · p. 21

E CoNS-Explorer System and Validation

  • CoNS-Explorer’s frozen causal source and offline scripts produce a shared fact ledger; the language model cannot change numerical values or causal relations.The source includes the DAG/SCM, baseline and intervention states, changes, boundary conditions, and question set.Source: E CoNS-Explorer System and Validation · p. 22
  • For k controllable factors, the system enumerates 6^k scenarios; the public v4 interface uses static lookup rather than runtime text generation.The three active datasets yield 288 participant-facing static scenarios.Source: E CoNS-Explorer System and Validation · p. 22

F Study 2: Exploratory Workflow and Qualitative Analysis

In Study 2, participants freely switched among graphs and two text forms while completing tasks across three domains. Their workflow combined graph-based structural anchoring, uncertainty-driven text consultation, cross-checking, and explicit study boundaries.

F Study 2: Exploratory Workflow and Qualitative Analysis

  • The independent Study 2 sample comprised 24 participants who freely switched among graph, Direct, and Story views across six fixed-order tasks in three domains.Sessions used a PC interface and lasted about 40 minutes, including tasks and interview.Source: F Study 2: Exploratory Workflow and… · p. 22
  • Graphs provided the structural anchor; participants consulted Direct for local verification and Story for contextual mechanism understanding when domains or processes felt unfamiliar.Participants returned to graphs, compared text forms, or invoked external knowledge before deciding.Source: F Study 2: Exploratory Workflow and… · pp. 22–23

G Reproducibility, Anonymization, and Boundaries

  • The planned reproducibility package includes frozen models, generation and analysis scripts, static scenarios and narratives, survey materials and outputs, and the final codebook.Separate R scripts reproduce the GLMM, GEE, bootstrap summaries, and experience analyses.Source: G Reproducibility, Anonymization, and… · p. 23
  • The causal models are teaching models rather than estimates from real observational datasets, and Study 2 qualitative counts describe this sample rather than a population rate.Other stated boundaries include discrete intervention values and evaluation of Story as a full expression package.Source: G Reproducibility, Anonymization, and… · p. 23
Loading 2609.12453v1…