Source-linked AI summary
NarrativeSteward: Coordinating Delegation, Guidance, and Verification in Agent-Assisted Interactive Narrative Authoring
Wenjin Wang, Jiazhen Lei, Yuxin Sha, Nuwa Xi, Meng Zhao, Xingxi Yin, Qi Liu, Yuliang Shen, Zixun Sun
TL;DR
Authors delegating interactive-narrative creation and revision to autonomous agents can struggle to understand the evolving work well enough to guide it. NarrativeSteward links narrative artifacts with agent dialogue, structural review, change records, and execution verification. In a 12-participant within-subject study, it supported easier revision-request formulation and change inspection, and greater perceived understanding than general-purpose agents.
Problem
As agents generate and revise extensive narrative content, authors need to understand its overall structure, local details, and relationships to guide continued creation.
Source: §1 · p. 2Method
Results
In a within-subject study with 12 participants, NarrativeSteward supported easier revision-request formulation and change inspection, alongside greater perceived understanding of changes and story structure, than general-purpose agents.
Source: §9 · p. 19Takeaways & Limitations
Authors used structural review and feedback to develop subsequent requests and direct further revisions, highlighting linked artifacts’ role in sustaining guidance across successive requests.
Source: §9 · p. 19The study compared complete authoring environments with different models, tools, representations, interaction support, familiarity, and story content, so it does not isolate individual system components.
Source: §8.4 · p. 18
Abstract
from arXiv · showhide
Autonomous AI agents can turn authors' goals into interactive narratives by independently organizing and carrying out generation and revision. As agents generate and revise extensive content, authors struggle to grasp its overall structure, local details, and relationships, complicating continued guidance. We present NarrativeSteward, an authoring environment that organizes outlines, worldbuilding, and narrative graphs as linked artifacts for agent implementation and author guidance. Agent dialogue and project-wide structural review help authors understand the evolving work and guide local and cross-layer revisions, while change records and execution verification help authors assess the resulting work. Technical tests validated the system's change records, recovery mechanisms, and execution diagnostics. In a 12-participant within-subject study, NarrativeSteward supported easier formulation of revision requests and inspection of changes, and greater perceived understanding of changes and story structure, than general-purpose agents. Qualitative findings show how reviewing the work and feedback helps authors develop requirements and guide subsequent delegation. We open-source NarrativeSteward at https://github.com/Tencent/NarrativeSteward.
1 Introduction
Autonomous agents can implement and revise interactive narratives, but authors need to understand the evolving work to guide further creation. NarrativeSteward links artifacts, dialogue, structural review, and feedback to support that guidance; its evaluation found improved revision-request formulation, change inspection, and perceived understanding compared with general-purpose agents.
1 Introduction
- Authors delegating extensive narrative work need to grasp its overall structure, local details, and relationships to determine what it contains and what to request next.Continuing to understand the developing work is part of delegation.Source: §1 · p. 2
- NarrativeSteward links outlines, worldbuilding, and narrative graphs as a shared basis for agent implementation and author review, supporting local and cross-layer changes without a fixed authoring sequence.Dialogue and project-wide structural review help authors inspect content and relationships, connect concerns to requests, and guide revisions.Source: §1 · p. 2
- In a 12-participant within-subject study, NarrativeSteward supported easier revision-request formulation and change inspection, alongside greater perceived understanding of changes and story structure, than general-purpose agents.Technical tests validated change records, recovery mechanisms, and execution diagnostics.Source: §1 · p. 2
- Structural review helped authors identify concerns and develop them through dialogue into subsequent requests, while execution diagnostics and agent feedback informed what to inspect and revise next.The findings connect examining the work with specifying initial goals as parts of ongoing guidance.Source: §1 · pp. 2–3
2 Related Work
Prior interactive-narrative tools support branching-content organization, LLM-assisted editing and analysis, and generation from higher-level requirements. NarrativeSteward focuses on a different division of work: agents organize and execute implementation while authors continue to understand and direct the evolving narrative.
2.1 Interactive Narrative Authoring
- Interactive-narrative tools organize branching content and connect structural editing with generation, analysis, or transformation from higher-level requirements into narrative structures and playable content.Other systems also help authors explore possible experiences through specifications, examples, and sampled playthroughs.Source: §2.1 · pp. 3–4
2.2 Autonomous Delegation and Structured Work Environments
- Autonomous agents can organize and carry out implementation, but users still need to interpret developing work; NarrativeSteward links narrative change records and execution diagnoses to artifacts and their structural connections.This context lets authors judge how changes or problems relate to creative intentions and delegate further work on the same artifacts.Source: §2.2 · p. 4
2.3 Author Guidance and Creative Participation
- When agents perform edits, authors may still develop and assess creative decisions; NarrativeSteward examines how reviewing artifacts and feedback supports identifying concerns, judging proposals, and forming subsequent requests.The focus includes when authors inspect the work and how they express what should happen next.Source: §2.3 · p. 4
2.4 Execution Analysis and Revision Feedback
- NarrativeSteward connects execution feedback to ongoing author guidance: verification follows choices and state updates across event and beat graphs, while authors inspect diagnoses and use them to decide what to ask the agent to change.Playtesting provides experience of selected routes, connecting whether a narrative can proceed with how the author wants it to unfold.Source: §2.4 · p. 5
3 The NarrativeSteward System
NarrativeSteward is a web-based environment for delegating interactive-narrative creation and revision while authors guide the developing work. Linked artifacts support agent implementation and author review; dialogue, structural views, change records, and execution verification help authors inspect and direct revisions.
3 The NarrativeSteward System
- NarrativeSteward lets authors delegate creation and revision of interactive narratives to an autonomous agent while continuing to guide the work through dialogue, structural views, and feedback.The resulting narratives support branching choices, delayed consequences, and multiple endings.Source: §3 · p. 5
3.1 Design Objectives
- The design objectives are to support delegation and review of local and cross-layer changes, connect focused dialogue with project-wide structural review, and make feedback useful for subsequent creative decisions.Change records, execution diagnostics, and playtesting feedback are connected to narrative content to inform what to retain, revise, or investigate.Source: §3.1 · p. 5
3.2 Linked Artifacts across Dialogue and Structural Views
- Linked artifacts include intent, outline, world cards, an event graph, and per-event beat graphs; references and state rules connect world context, local choices, and later consequences.Authors and agents inspect the same work through dialogue and structural views, while authors can move from a concern to relevant content and back to dialogue.Source: §3.2 · p. 5, Figure 3 · p. 6
3.3 Executable Narrative Model
- Choices update shared state variables through beat effects, and conditions on outgoing edges determine available continuations across event and beat graphs.The graphs are directed acyclic graphs with a single entry; variables are Boolean flags, finite enumerations, or bounded integers, and single-variable edge conditions support a finite execution state space.Source: §3.3 · pp. 6–7
3.4 Delegating, Reviewing, and Guiding Revisions
- An agent reads and edits project artifacts in response to an author request, then presents saved changes in a changeset that authors can locate, inspect, keep, or undo.Changes are prepared in an isolated draft; validation precedes saving, and a failed save restores the prior saved state.Source: §3.4 · p. 7, Figure 4 · p. 7
3.5 Execution Verification and Diagnostic Feedback
- Execution verification exhaustively analyzes saved event and beat graphs to identify unreachable content, never-taken transitions, and reachable non-ending states with no continuation.The analysis merges states at the same position when their values agree on variables needed for later condition checks; authors can inspect diagnostic traces and locate relevant narrative objects.Source: §3.5 · p. 8
4 Usage Scenario
Maya develops The Last Delivery, a story about a courier boarding the last ferry. The scenario follows her from delegating a draft through reviewing its content, resolving an execution problem, and playtesting the result.
4 Usage Scenario
- Maya’s story centers on a courier choosing between carrying medicine to a stranded clinic and a sealed letter to the harbor keeper on the last ferry.The courier cannot serve both destinations that night.Source: §4 · p. 8
- Maya wants an early choice to determine the later destination while the prose conveys the cost of leaving someone waiting.Source: §4 · p. 8
- The scenario follows Maya from delegating an initial draft through reviewing content, resolving an execution problem, and playtesting the result.Source: §4 · p. 8
Delegating the Initial Narrative
Maya delegates a compact story with two endings to the agent, then reviews how its linked artifacts and cargo variable implement the intended choice.
Delegating the Initial Narrative
- Maya requests a compact story with Departure, Crossing, and two endings, each reachable in a different playthrough.She authorizes continuous generation of the linked narrative artifacts.Source: Delegating the Initial Narrative · p. 8
- The generated project includes intent, an outline, world cards, an event graph, and beat graphs for each event.Source: Delegating the Initial Narrative · p. 8
- A shared cargo variable records the chosen parcel and determines which destination becomes available.Source: Delegating the Initial Narrative · p. 8
Reviewing the Narrative and Guiding Revisions
Maya reviews generated narrative content, requests a revision that preserves its rules and choices, and then refines wording directly. The checkpoint example shows how reachable states determine available exits.
Reviewing the Narrative and Guiding Revisions
- Maya asks the agent to make Crossing feel more like a personal cost while preserving its cargo rules, connections, choices, and endings.The agent revises the event summary and opening narration.Source: Reviewing the Narrative and Guiding… · p. 8
- The changeset lets Maya compare the saved text before and after the agent’s revision.Source: Reviewing the Narrative and Guiding… · p. 8
- Using Locate, Maya opens Crossing’s summary in the structural view and refines its wording directly, retaining the rules connecting parcel choice to destination.Source: Reviewing the Narrative and Guiding… · pp. 8–9
- At the checkpoint, each gate condition is tested against reachable states; having neither a pass nor a key leaves every exit unavailable.Source: Figure 5 · p. 9
Running Execution Verification and Choosing a Repair
Execution verification identifies an unreachable Crossing branch, and Maya chooses a repair that preserves the intended choice between helping the clinic and the harbor.
Running Execution Verification and Choosing a Repair
- Verification identifies a Crossing transition requiring cargo=none even though earlier choices set cargo to medicine or letter.The transition is unreachable because the parcel choice changes cargo before the courier boards the ferry.Source: Running Execution Verification and… · p. 9
- Maya uses Locate to inspect the transition and its condition before choosing how to repair it.Source: Running Execution Verification and… · p. 9
- Adding a leave-both option would make the branch reachable but let the courier avoid choosing which shore to help, so Maya removes the unreachable beat and incoming transition.This keeps the original two choices and their consequences.Source: Running Execution Verification and… · p. 9
Verifying the Revision and Playtesting the Narrative
After removing the unreachable branch, Maya verifies the narrative and playtests the parcel choice through to its destination. The linked artifacts and playtest connect her revisions and repair decisions to story content and consequences.
Verifying the Revision and Playtesting the Narrative
- After the unreachable branch is removed, execution verification passes.Source: Verifying the Revision and Playtesting… · p. 9, Figure 8 · p. 11
- In the medicine playtest, the clinic is available at Crossing while the harbor remains locked because it requires the letter.Cite for Agent attaches the locked choice and current playtest context to dialogue as Maya drafts a question about its availability.Source: Verifying the Revision and Playtesting… · p. 9, Figure 8 · p. 11
- Across the scenario, Maya’s original dilemma guides her requests, prose assessment, and repair choice; playtesting brings the rules’ consequences back into the story experience.Source: Verifying the Revision and Playtesting… · pp. 9–10
- The generated event graph connects Departure and Crossing to the two endings; continuing the medicine route reaches Clinic Ending.Source: Verifying the Revision and Playtesting… · p. 9, Figure 6 · p. 10, Figure 8 · p. 11
5 Technical Evaluation
Controlled tests assessed whether NarrativeSteward accurately records and recovers changes and verifies narrative execution. The tests matched expected outcomes across change tracking, recovery, fault detection, report applicability, and runtime.
5.1 Change Records and Recovery
- All 11 expected change details across four editing cases were identified without extras, and every affected narrative object was correctly located.Cases covered local edits, batched beat-graph edits, cross-layer additions, and text-and-asset changes.Source: §5.1 · p. 11
- All four failed or stopped turns restored the pre-turn project; undo restored prior content in three conflict-free cases and rejected three conflicting cases without partial reversion.Rejected undo preserved the saved work.Source: §5.1 · p. 11
5.2 Execution Verification
- Execution verification found and located expected faults in all three faulty projects, while all three corrected versions passed.The faults covered an unreachable event, a never-enabled edge, and a reachable dead-end state.Source: §5.2 · pp. 11–12
- Verification reports were invalidated after all five changes to checked content and retained after both description or image-path edits.The latter edits do not affect execution checks.Source: §5.2 · p. 12
- Across five test cases, median verification times ranged from 3.5 to 59.4 ms; the longest median occurred when exploring 45,051 execution states.All 25 timed runs completed.Source: §5.2 · p. 12
6 User Study
The study compared NarrativeSteward with participants’ familiar general-purpose agents in a within-subject design, combining questionnaires, system-use records, and interview accounts. It examined delegation, understanding and refinement, and feedback-informed judgments and revision decisions.
6 User Study
- The within-subject study examined delegation experience, understanding and continued refinement, and feedback-informed judgments using questionnaires, interviews, and system-use records.Twelve participants completed the study; they ranged from no creative experience to seven or more years, and all had used their selected general-purpose agent on a multi-file project.Source: §6 · p. 12, §6.1 · p. 12
6.2 Study Design and Procedure
- Each participant created a browser-runnable interactive story from blank content with both NarrativeSteward and a familiar general-purpose agent, with condition order counterbalanced.The baseline comprised Cursor for eight participants and Codex for four; NarrativeSteward used gpt-5.5 with default medium reasoning effort, while baseline participants chose their own model and settings.Source: §6.2 · pp. 12–13
- Participants could choose when to submit without a fixed time limit, and researchers confirmed that every submitted project in both conditions was runnable.After both tasks, participants completed cross-tool comparisons and interviews; NarrativeSteward participants received usage instructions before their task.Source: §6.2 · p. 13
6.3 Data Collection and Analysis
- Thirteen seven-point measures covered the three research questions, alongside feature-helpfulness ratings, feature-use reports, and a standard-scored System Usability Scale.Paired ratings were analyzed item by item with two-sided Wilcoxon signed-rank tests, paired rank-biserial effects, and Holm–Bonferroni correction within each research question.Source: §6.3 · p. 13
- Researchers thematically analyzed interview notes and written accounts, and NarrativeSteward logged authoring events, changesets, and verification metadata.Two researchers reviewed and coded relevant passages, discussed themes, and revisited material for supporting and contrasting experiences.Source: §6.3 · p. 13
7 User Study Findings
Across the user study, participants rated NarrativeSteward more favorably for formulating and inspecting revisions, understanding story structure, and making decisions about changes. Their accounts describe how review, dialogue, diagnostics, and playtesting informed successive revision choices.
7.1 RQ1: Delegation Experience and Effort
- For revision formulation, request expression, and change inspection, NarrativeSteward medians were 6, 6, and 6.5 versus agents’ 3.5, 4, and 3; all differences were significant after correction.The three adjusted p-values were 0.003, 0.008, and 0.002 across 12 paired responses.Source: §7.1 · p. 13, Figure 9 · p. 14
- Participants described using access to specific content and broader context to form and review requests, while related-content changes and intended narrative effects required additional coordination.The fit between requests and results did not differ significantly after correction (medians 5 versus 4.5; adjusted p=0.375).Source: §7.1 · pp. 13–14
- Two participants had desired but unfinished revisions with NarrativeSteward versus five with general-purpose agents; among the latter, four cited anticipated inspection effort and two communication effort.Participants could select up to two stopping reasons; accounts also described checking effort, satisfaction, and available effort shaping whether work continued.Source: §7.1 · p. 14
7.2 RQ2: Understanding and Continuing to Refine the Work
- All 12 participants rated structural understanding higher with NarrativeSteward (medians 7 versus 4), and locating content and understanding agent changes also rated higher.Locating ease medians were 6 versus 4 (adjusted p=0.002); understanding changes medians were 6 versus 4 (adjusted p=0.029).Source: §7.2 · p. 15
- Participants used structural review to identify revision targets, then combined agent requests with direct edits while moving between related narrative parts.Feature helpfulness medians were 6.5 for the structural view and 7 for content inspection; seven participants preferred NarrativeSteward overall, while three reported insufficient experience to choose.Source: §7.2 · pp. 15–16
7.3 RQ3: Feedback, Judgment, and Revision Decisions
- NarrativeSteward was favored for deciding whether to retain a change (medians 6 versus 3.5) and whether to check further (5 versus 3); task-completion confidence was also higher (6 versus 5).All eligible pairs favored NarrativeSteward on the first two judgments; recovery ease did not differ significantly after correction.Source: §7.3 · p. 16
- Participants used content inspection, agent feedback, execution diagnoses, and playtesting to judge implemented changes and direct further checks or revisions.Execution verification and playtesting each received a median helpfulness rating of 7; one account describes repeatedly alternating semantic review and execution verification with delegated repairs.Source: §7.3 · p. 16
8 Discussion
NarrativeSteward’s linked artifacts and structural review help authors turn emerging creative aims into revision requests, while connected feedback supports judging and guiding successive changes. The discussion also identifies review flexibility and study and representation constraints as important boundaries.
8.1 Structural Review Helps Authors Develop Creative Direction
- Structural review helps authors inspect generated relationships, develop concrete revision requests, and continue shaping their direction as the narrative takes form.Linked artifacts keep the concern, changed content, and subsequent review connected.Source: §8.1 · pp. 16–17
8.2 Review Effort and Flexibility in Authoring Shape Continued Delegation
- Review effort can discourage continued revisions, motivating focused and flexible ways to inspect large generated works and adapt how authors approach them.The discussion proposes focused review, intermediate inspection, and adapting presentations or connecting other tools to authors’ current needs.Source: §8.2 · pp. 17–18
8.3 Feedback Connects Execution Evidence with Creative Judgment
- Authors combine change records, execution diagnostics, agent analysis, and playtesting as distinct evidence for creative judgments and successive revision decisions.A reachable branch may still feel inconsequential; resolving a reachability issue can either make it accessible or remove it, with different story possibilities.Source: §8.3 · p. 18
8.4 Limitations
- Because the study compared complete environments with different models, tools, representations, and familiarity, it cannot isolate model capability or individual component effects.The authors suggest comparing interface variants with the same model.Source: §8.4 · p. 18
- The 12 participants each completed one task per condition, limiting generalization to other author populations and longer creative projects.The evaluation also emphasizes perceived support and accounts of decision-making; how these relate to implemented revisions and story quality needs further assessment.Source: §8.4 · p. 18
- Executable narratives use acyclic event and beat graphs, finite state-variable domains, and single-variable edge conditions, constraining recurring situations and complex conditions.Verification costs for substantially larger or differently structured narratives remain to be examined.Source: §8.4 · p. 18
9 Conclusion
The conclusion presents NarrativeSteward as a linked-artifact authoring environment connecting dialogue, structural review, and feedback, and reports improved revision experience and perceived understanding in a 12-participant comparison. The accompanying materials document its implementation, study measures, interview guide, and story scale.
9 Conclusion
- In a 12-participant within-subject study, NarrativeSteward supported easier revision requests and change inspection, with greater perceived understanding of changes and story structure than general-purpose agents.Participants described using structural review to develop requests and combining execution feedback with agent-led story analysis to direct revisions.Source: §9 · pp. 18–19
A Implementation Details
- The system uses a web client, Python backend, typed JSON artifacts, and a main agent that can delegate generation or rewriting to four specialized agents.Structural validation checks formats, references, and graph relationships; turn work occurs on an isolated project copy, and verification reports are tied to fingerprints of checked content.Source: A Implementation Details · p. 21
B.1 Questionnaires
- Questionnaires used seven-point scales for 13 authoring-experience measures, with applicability conditions for reporting revisions, inspecting changes, and rating feature helpfulness.Usability used the published Simplified Chinese SUS wording on a five-point scale.Source: §B.1 · p. 21, §B.1.1 · p. 21, §B.1.2 · p. 21
B.1.3 Tool Preferences and Usability.
- Background questions covered creative experience, AI and narrative-tool use, and selected-agent capabilities; participants also reported tool preferences across authoring activities.The study asked participants about their experience with each tool and the reasons behind their judgments and choices.Source: §B.1.4 · p. 21, §B.2 · p. 21, Table 8 · p. 23
B.2 Interview Guide
- The interview guide covered creative direction, delegation, finding and understanding changes, revision or recovery choices, checking work, and future tool preferences.It asked participants to describe concrete experiences and the information they consulted when deciding what to do next.Source: §B.2 · pp. 21–22
B.3 Narrative Topics and Scale
- The 24 completed narratives spanned six genres and were summarized by story stages, decision points, and distinct reachable endings across branches.The counts were manually checked; stages denote major story phases, while decision points count distinct choice locations affecting content, later actions, or endings.Source: §B.3 · p. 23
B.4 Supplementary Analyses and Results
Supplementary analyses describe paired rating distributions and the technical evaluation of change, recovery, and execution-verification mechanisms. The appendix details the verification algorithm, test cases, and resource measurements.
B.4 Supplementary Analyses and Results
- Table 10 supplements the 13 authoring-experience comparisons with medians, quartiles, and within-participant rating differences for eligible paired responses.Wilcoxon tests used nonzero within-participant differences and assumed independent participants and differences symmetric about zero under the null.Source: §B.4 · p. 23, Table 10 · p. 24
C Execution Verification and Technical Evaluation Details
- Technical tests used predefined inputs in isolated project copies to exercise change, recovery, and verification mechanisms without invoking a language model.The appendix describes the verification algorithm and technical test cases summarized in the main paper.Source: C Execution Verification and Technical… · p. 24
C.1 Execution Verification Algorithm
- Verification expands event and beat graphs into an acyclic execution graph and propagates required variable values to check reachable content and execution behavior.It computes needed-variable sets in reverse topological order; fixed assignments remove overwritten incoming values, while constant increments preserve the need for an incoming value.Source: §C.1 · p. 24
- At each execution position, verification applies effects, tests outgoing conditions, and forwards distinct joint assignments while preserving variable correlations.It records reached events and beats, enabled edges, and reachable non-ending states without continuations, reconstructing a route to reported dead ends.Source: §C.1 · pp. 24–25
C.2 Change Records and Recovery
- Change-record tests covered four edit patterns; recovery and undo tests checked restoration, asset removal, repeated undo, and rejection of conflicting whole-turn undos.Cases included exceptions, timeouts, recursion-limit exceptions, and user stops after project writes.Source: §C.2 · p. 25
C.3 Execution Verification Tests
- Verification tests checked hand-computed flows and faulty/corrected project pairs for unreachable events, never-enabled edges, and reachable dead ends.Report-applicability tests included edits to checked inputs and to world-card descriptions and image paths.Source: §C.3 · p. 25
C.4 Runtime and Resource Measurements
- Resource measurements covered one reference story and four generated structures, with one warmup and five measured runs per case; all runs completed.The distant-fault case required route reconstruction, and Table 11 reports time, peak memory, and explored execution states.Source: §C.4 · p. 25, Table 11 · p. 26