PaperlayerView exact revision on arXiv

Source-linked AI summary

SENSE: State-aware Emotion Navigation Storytelling Engine

Yi Xia, Pablo Carrasco Velo, Mudit Paliwal, Ibrahim Khan, Yifan Geng, Mustafa Can Gursesli, Juho Hamari, Ruck Thawonmas

arXiv:2610.07666v1cs.HCcs.AIcs.MM

TL;DR

Branching visual novels can suffer from graph-level inconsistencies and emotional drift, while existing path repairs do not guide emotional intent. SENSE combines state-aware planning, topology validation, and path-aware synthesis; its evaluations report gains in emotional richness, narrative diversity, and visual-textual alignment.

  • Problem

    LLMs can generate branching narratives with missing passages, trap-like cycles, and causal asymmetries, while physical-path repairs do not address emotional intent and drift.

    Source: §I · p. 1

  • Method

    SENSE generates playable VNs from minimal inputs by modeling narrative state, branching history, choices, and emotional trajectories.

    Source: §I · p. 1

  • Results

    The authors report significant improvements in emotional richness, narrative diversity, and visual-textual alignment while maintaining end-to-end integration in a deterministic game environment.

    Source: §VII · p. 11

  • Takeaways & Limitations

    SENSE integrates structurally coherent, emotionally resonant branching narratives within a deterministic playable-VN environment.

    Source: §VII · p. 11

    Generated prose remains below human-authored visual novels in literary quality, with narrative depth bounded by LLM creative limitations.

    Source: D. Limitations and Future Work · p. 10

Abstract

from arXiv · show

This paper presents SENSE, a state-aware framework for generating playable branching visual novels with multi-track emotional navigation. Integrating a state-based narrative architecture called MIND, a structure analyzer, and a path-aware context management module, SENSE produces narratives that are both structurally coherent and emotionally rich. From minimal high-level inputs, it generates multiple intersecting routes while preserving character consistency and narrative causality. Evaluations using LLM judges, affective metrics, and visual assessments indicate SENSE outperforms baselines in narrative diversity and robust asset integration, while preliminary human trials show directional improvements in emotional fidelity alongside comparable enjoyment.

I. INTRODUCTION

SENSE addresses graph-level failures and emotional drift in branching visual novels by combining state-aware emotional planning with topology validation and path-aware script generation. Its design builds on hierarchical storytelling while targeting long intersecting routes that remain underexplored in LLM-based VN generation.

I. INTRODUCTION

  • LLMs can produce missing passages, trap-like cycles, and causal asymmetries, while physical path repairs leave emotional intent—and thus emotional drift—unaddressed.The related-work discussion says integrating emotional trajectories into LLM-generated VNs with long texts and intersecting branches remains underexplored.Source: §I · p. 1, A. Emotional Arcs and Narrative… · p. 2
  • SENSE generates playable VNs from minimal inputs by modeling narrative state, branching history, player choices, and emotional trajectories.Its MIND architecture combines macro-level emotional planning with path-aware script generation to guide emotional progression and causal consistency.Source: §I · p. 1
  • The Structure Analyzer corrects node errors, cycles, and orphans, while topology-driven, path-aware synthesis improves context consistency across shared nodes.The framework separates high-level narrative planning from surface-text realization, a longstanding strategy for mitigating long-range dependencies.Source: §I · p. 1, A. Emotional Arcs and Narrative… · p. 2
  • SENSE compiles validated scripts and visuals into playable VNs, with the stated goal of improving semantic fidelity and interactive coherence.The paper also situates this planning-to-execution separation within hierarchical narrative-generation approaches.Source: §I · p. 1, A. Emotional Arcs and Narrative… · p. 2

B. LLM-Based Branching Narrative Generation

LLM-based branching narratives remain vulnerable to global path inconsistencies, and prior multi-agent approaches also leave text-image alignment challenges. SENSE combines topology-aware generation with a decoupled pipeline that turns minimal inputs into playable visual novels.

B. LLM-Based Branching Narrative Generation

C. LLM-driven Multi-modal Narrative Generation

  • The system uses role-specialized agents and iterative multimodal selection and validation to improve visual-textual alignment.The related-work discussion identifies text-image alignment as an area where hierarchical frameworks still have room to improve.Source: C. LLM-driven Multi-modal Narrative… · p. 2

A. System Overview

  • SENSE generates playable VNs from theme, genre, and emotional-arc inputs while separating narrative prose from deterministic routing and code compilation.Audio generation is excluded, while textual and visual assets are retained.Source: A. System Overview · p. 2
  • The seven-phase pipeline expands inputs into a world setting, creates an emotionally tagged outline, validates routing, derives chapter outlines, and synthesizes path-dependent scripts.It then generates context-aware images and deterministically compiles tagged scripts and visuals into executable code.Source: A. System Overview · p. 2

B. The MIND state-based Architecture

MIND governs emotional planning and route-specific script synthesis through STAR and two routing-triggered modes. It treats convergent branches as unresolved choice histories and unique branches as specific player choices, with rules intended to preserve logical consistency.

MIND state-based Architecture

  • MIND is an inference-time narrative controller that uses prompt constraints and dynamic context injection rather than changing the underlying LLM architecture.Its governance combines STAR’s macroscopic planning with path-aware microscopic synthesis.Source: B. The MIND state-based Architecture · pp. 2–3

Macroscopic Governance: The STAR Protocol

  • STAR sets a global emotional blueprint: the Spine follows hard constraints for the given arc, while branching Limbs develop trajectories through autonomous reasoning.Emotional tags mark each node as Rise or Fall, supporting variation across branches while mitigating affective monotony.Source: 1) Macroscopic Governance: The STAR… · p. 3

Microscopic Logic Safety

  • At convergent chapters, Mode A keeps merged scenes valid across possible player histories by withholding conflicting physical details, making NPC reactions unreadable, or focusing on shared mysteries.The system injects possible player choices when multiple diverging choices lead to the same subsequent chapter.Source: 1) Macroscopic Governance: The STAR… · p. 3
  • At unique branches or endings, Mode B states the definite player choice and requires the script to describe its specific consequences.If dimensions remain in conflict, the prompt directs the model toward general consequences or a sudden external event to advance the story without logical friction.Source: 1) Macroscopic Governance: The STAR… · pp. 3–4

C. Structure Analyzer

The Structure Analyzer deterministically repairs graph errors before script synthesis. Subsequent context injection and sequential, route-specific synthesis use topology and path histories to produce distinct scripts at shared nodes.

C. Structure Analyzer

  • The Structure Analyzer consolidates outlines because LLMs can hallucinate node identifiers and create illogical loops even when events are converted to structured JSON.Its validated graph provides the foundation for subsequent script synthesis.Source: C. Structure Analyzer · p. 4
  • Heuristic node resolution repairs broken transition links by prioritizing full-title, explicit-ID, cleaned-title, then branch-and-keyword matches.This procedure addresses numerical chapter-ID mismatches when semantic titles remain accurate.Source: C. Structure Analyzer · p. 4
  • A DFS detects cycles, then Connection Quality Scores break the weakest link while favoring more logical transitions.The scoring favors natural index progression or same-branch links and penalizes structural regressions.Source: C. Structure Analyzer · p. 4
  • Orphan resolution distinguishes true orphans from disconnected subgraphs, reconstructs missing edges through reverse validation, and prunes only confirmed true orphans.Type-agnostic identifier comparisons help address integer-string confusion.Source: C. Structure Analyzer · p. 4

D. Topology-Driven Context Injection and Script Synthesis

  • Path-aware synthesis retrieves each route’s transcript so shared chapters can receive distinct scripts conditioned on their ancestor sequences.The context-injection logic also supplies possible choices at convergences and can generate unique variant files for terminal-node divergence.Source: D. Topology-Driven Context Injection… · p. 4

E. VN Image Assets Generation

SENSE generates identity-preserving visual assets and converts its intermediate scripts into playable Ren’Py games. Its pipeline combines structured identity and context cues, facial-expression editing, iterative validation, and automated script integration.

E. VN Image Assets Generation

  • Structured world-setting attributes establish a Base Identity, while script cues are merged with location metadata to ground backgrounds and mitigate character drift.The pipeline uses image-to-image generation and facial-region editing to vary expressions without changing core character structure or attire.Source: E. VN Image Assets Generation · pp. 4–5
  • A VLM scores three candidate images per iteration for Base Identity consistency and visual fidelity, adding correction feedback until one passes or reaches two iterations.The image-generation loop evaluates candidates against predefined quality criteria.Source: E. VN Image Assets Generation · p. 5

F. Script Integration and Backend Generation

  • A shared script representation supports backend-specific outputs; the demonstrated integration targets Ren’Py, with similar conversion logic proposed for platforms such as Unity.Platform-specific integration stages would still be needed for other backends.Source: F. Script Integration and Backend… · p. 5

G. Game script conversion and asset integration

  • Two sequential parsers link images to script labels and asset identifiers, then convert the resulting script into a playable Ren’Py game.Regular-expression parsers transform the intermediate script and bind generated assets to narrative elements.Source: G. Game script conversion and asset… · p. 5
  • The generated game initializes assets, implements branching choices, matches dialogue characters to images, and replaces the oldest displayed character when a fourth appears.At most three characters are displayed at once, positioned left, right, and center.Source: G. Game script conversion and asset… · p. 5

IV. EXPERIMENTS

The experiments compare SENSE with an external baseline and an ablated version under a shared generation setup. They reconstruct readable routes for evaluation and use LLM judges to assess narrative quality across multiple criteria and route pools.

1) Frameworks and Generation Setup:

2) Comparative Integrity and Route Extraction Protocol:

  • A Ren’Py-aware parser reconstructs player-facing text routes and bypasses non-fatal execution errors to assess the baseline’s readable narrative output.This protocol evaluates the baseline at its peak narrative performance rather than penalizing its architectural limitations.Source: 2) Comparative Integrity and Route… · pp. 5–6

B. Primary Evaluation: LLM-as-a-Judge Protocol

  • Three independent, zero-temperature LLM-judge trials score six narrative criteria on a five-point scale, aggregated into Overall quality across AllMean and Top-3 route pools.AllMean covers successfully generated routes; Top-3 compares the highest-scoring routes per case to address route-yield differences.Source: B. Primary Evaluation: LLM-as-a-Judge… · p. 6

C. Quantitative Analysis of Affective Salience

The analysis measures affective intensity and richness in generated routes, evaluates text-image alignment and character identity, and applies statistical tests to compare groups and modules.

C. Quantitative Analysis of Affective Salience

  • A context-aware DeBERTa model estimates sentence-level valence and arousal, with reported concordance correlation coefficients of 0.8221 and 0.7125, respectively.Its sliding window uses a radius of four sentences.Source: C. Quantitative Analysis of Affective… · p. 6
  • Affective Salience measures mean distance from neutral, while High-Salience Proportion measures the share of sentences with valence or arousal outside [0.35, 0.65].Sentence-level deviations from 0.5 are aggregated into route-level metrics.Source: C. Quantitative Analysis of Affective… · p. 6

D. Visual Evaluation: Cross-Modal Alignment and Character Identity Preservation

  • Visual evaluation uses VQAScore for prompt alignment and DINO and CLIP-I embedding similarity against base character images to assess identity preservation across expressions.VQAScore is the probability of answering “Yes” to whether an image shows its prompt.Source: D. Visual Evaluation: Cross-Modal… · p. 6

E. Statistical Methodology and Significance Testing

  • Normality tests determine whether group comparisons use one-way ANOVA or Kruskal-Wallis, while planned pairwise contrasts use Welch’s t-tests or Mann-Whitney U tests.The planned contrasts report unadjusted p-values.Source: E. Statistical Methodology and… · pp. 6–7

F. Implementation Details

SENSE produced more routes than the baseline and achieved stronger Top-3 Emotional Fidelity and Overall Quality scores. The section also reports implementation details and quality across the full set of generated routes.

F. Implementation Details

  • Generated scripts and multimedia assets were compiled using Ren’Py, alongside GPT-5-mini narrative generation, three open-weight LLM judges, and Qwen3-VL for visual evaluation.Images were generated with waiIllustriousSDXL in ComfyUI.Source: F. Implementation Details · p. 7

B. Narrative Quality

  • Full SENSE achieved the highest Top-3 Emotional Fidelity (EM: M = 4.58) and Overall Quality (OV: M = 4.52), significantly outperforming Baseline and WithoutMIND.The reported 95% confidence intervals are [4.53, 4.63] for EM and [4.48, 4.56] for OV.Source: Table II · p. 7, B. Narrative Quality · p. 7

A. Framework Robustness and Complexity: Narrative Route Yield

  • Full SENSE generated 226 routes, more than 2.2 times the Baseline’s 99; WithoutMIND generated 94 routes.The reported route-yield difference reflects multiple emotional trajectories and the Path-Aware Context mechanism and Schrödinger Rule for convergences.Source: A. Framework Robustness and… · p. 7

B. Narrative Quality

  • Across 226 routes, Full SENSE maintained high average narrative quality in the AllMean robustness analysis, which evaluates all generated routes rather than only each VN’s Top-3.WithoutMIND’s per-route Complexity score fell to 4.27 under structural restrictions and single-emotion assignment; Full SENSE restored Complexity to 4.532 and Surprise to 3.562.Source: B. Narrative Quality · p. 7

C. Affective Salience and Narrative Pacing Results

Full SENSE increased valence salience while leaving arousal largely unchanged; visual evaluations found strong character identity preservation alongside mixed text-image alignment results.

C. Affective Salience and Narrative Pacing Results

  • Valence AS reached 0.103 (95% CI [0.101, 0.105]) and HSP 23.6% (95% CI [22.8, 24.4]), significantly exceeding Baseline and WithoutMIND (p < 0.001).The increase from WithoutMIND coincided with higher affective salience without reducing structural diversity.Source: C. Affective Salience and Narrative… · p. 8, Table III · p. 8
  • Full SENSE showed no significant difference from Baseline in Arousal AS (p = 0.105), while Baseline had higher Arousal HSP than Full SENSE (86.9% vs. 85.0%, p < 0.001).Source: C. Affective Salience and Narrative… · p. 8

D. Visual Evaluation Results

E. End-to-End System Deployment

SENSE ran as a playable Ren’Py visual novel, with user-study findings favoring some relevance and emotional-arc measures but not overall enjoyment. In an open-weight model case study, it produced more routes and retained affective and character-identity advantages.

E. End-to-End System Deployment

  • The system ran as a fully playable visual novel using Ren’Py, executing the full pipeline without manual intervention and integrating narrative, visual, and interactive components.Source: E. End-to-End System Deployment · p. 8

F. User Study

  • Participants rated SENSE higher in Genre Relevance (Mdiff = 0.67, p = 0.013), and 66.7% (12/18) perceived a clearer emotional arc, while enjoyment preferences were evenly split (9 vs. 9).Source: F. User Study · p. 9, Table V · p. 9

G. Generalizability to Open-Weight Models

VI. DISCUSSION

SENSE’s discussion reports stronger valence-focused emotional outcomes and greater route coverage, alongside trade-offs in enjoyment, visual variety, and literary quality.

A. Valence Guidance and Multi-Track Emotional Support

  • Full SENSE led LLM-judged Empathy and Overall Quality, with higher emotional engagement and expressive depth than Baseline and WithoutMIND.The discussion attributes its varied emotional arcs to MIND’s multiple simultaneous emotional tracks.Source: A. Valence Guidance and Multi-Track… · p. 9
  • 0.103 Valence AS and 23.6% HSP were highest for Full SENSE, with HSP above Baseline (15.3%) and WithoutMIND (21.0%).These sentence-level measures complement the LLM-judge results.Source: A. Valence Guidance and Multi-Track… · p. 9
  • Arousal differences remained minor: Full SENSE had Arousal AS 0.244 and HSP 85.0%, versus Baseline at 0.246 and 86.9%.The discussion says overall arousal remains largely driven by the baseline language model’s generative tendencies.Source: A. Valence Guidance and Multi-Track… · pp. 9–10

B. The Complexity Paradox: Global vs. Per-Route Narrative Complexity

  • 226 routes for Full SENSE versus 99 for Baseline illustrate the coverage–per-route complexity trade-off; SENSE reports logical and affective integrity across the expanded narrative space.The framework attributes this balance to MIND, the structure analyzer, and path-aware context management.Source: B. The Complexity Paradox: Global vs.… · p. 10

C. User Study Findings

  • Enjoyment split evenly (9 vs. 9), while 66.7% of participants identified SENSE’s version as having a clearer emotional arc.Some participants found the baseline’s unpredictability entertaining and SENSE’s visual stability tedious; experienced participants rated theme relevance higher, while both versions drew criticism for prose and character depth.Source: C. User Study Findings · p. 10

D. Limitations and Future Work

SENSE’s reported limits include literary depth, expressive poses, arousal guidance, and preliminary user-study scope; the paper also identifies deployment safety needs and frames the system as a co-creation aid.

D. Limitations and Future Work

  • Generated prose and character motivations remain less nuanced than in human-authored visual novels, while identity-consistent expressive body poses and fine-grained arousal guidance remain challenging.The paper also notes that its user study included 18 participants and warrants validation in broader contexts.Source: D. Limitations and Future Work · p. 10
  • Future work targets pose-constrained generation, stronger arousal guidance, deeper prose, and larger participatory studies with multimodal physiological metrics.The proposed pose work includes context-aware inpainting for cross-prompt visual consistency.Source: D. Limitations and Future Work · p. 10

VII. CONCLUSION

  • The paper presents SENSE as a Human-AI co-creation tool that supplies an affective draft for creators to refine and expand.It is intended to lower the barrier for indie developers and small studios by automating branching logic and repetitive asset generation.Source: E. Ethical Considerations · p. 10, §VII · p. 11

E. Ethical Considerations

  • Routes involving negative emotions or extreme conflict carry risks of harmful, biased, or inappropriate content; industrial use requires stricter safety alignment and filtering.Source: E. Ethical Considerations · p. 10

VII. CONCLUSION

  • SENSE integrates MIND, a structure analyzer, and path-aware context management to generate playable branching VNs with multi-track emotional navigation from minimal input.The conclusion reports improvements in emotional richness, narrative diversity, and visual-textual asset alignment within a deterministic game environment.Source: §VII · p. 11

Related papers

Loading 2610.07666v1…