Source-linked AI summary
Codified Foreshadowing-Payoff Text Generation
Longfei Yun, Kun Zhou, Yupeng Hou, Letian Peng, Jingbo Shang
TL;DR
Story-generation models often produce locally fluent text without fulfilling long-range narrative setups, while existing evaluations under-test this form of coherence. CFPG encodes foreshadow–trigger–payoff commitments, mines aligned examples from BOOKSUM, and reports consistent improvements over prompt-based baselines in payoff timing, localization, and narrative alignment.
Problem
Language models frequently neglect, contradict, or mishandle foreshadowed commitments, while existing benchmarks emphasize local coherence instead of later payoff realization.
Method
CFPG represents foreshadows as executable Foreshadow–Trigger–Payoff predicates and mines structured foreshadow–payoff pairs from BOOKSUM.
Results
CFPG shows consistent improvements over prompt-based baselines in payoff timing, localization, and narrative alignment.
Takeaways & Limitations
Explicitly codifying narrative commitments enables grounded payoff detection and controlled realization under incremental context.
Takeaways & Limitations
CFPG targets explicit, textually grounded relations and is evaluated on summary-level narratives, while extraction errors may limit coverage.
Abstract
from arXiv · showhide
Foreshadowing and payoff are ubiquitous narrative devices through which authors introduce commitments early in a story and resolve them through concrete, observable outcomes. However, despite advances in story generation, large language models (LLMs) frequently fail to bridge these long-range narrative dependencies, often leaving "Chekhov's guns" unfired even when the necessary context is present. Existing evaluations largely overlook this structural failure, focusing on surface-level coherence rather than the logical fulfillment of narrative setups. In this paper, we introduce Codified Foreshadowing-Payoff Generation (CFPG), a novel framework that reframes narrative quality through the lens of payoff realization. Recognizing that LLMs struggle to intuitively grasp the "triggering mechanism" of a foreshadowed event, CFPG transforms narrative continuity into a set of executable causal predicates. By mining and encoding Foreshadow-Trigger-Payoff triples from the BookSum corpus, we provide structured supervision that ensures foreshadowed commitments are not only mentioned but also temporally and logically fulfilled. Experiments demonstrate that CFPG significantly outperforms standard prompting baselines in payoff accuracy and narrative alignment. Our findings suggest that explicitly codifying narrative mechanics is essential for moving LLMs from surface-level fluency to genuine narrative competence.
1 Introduction
The paper identifies a gap in story generation: models can remain locally fluent while failing to fulfill long-range narrative commitments. CFPG addresses this by explicitly representing and evaluating foreshadow–payoff realization.
- Narrative coherence depends on concretely resolving commitments introduced through objects, intentions, or conditions earlier in the story.
- Existing story-generation benchmarks emphasize short-range coherence rather than whether earlier narrative setups are subsequently paid off.
- Language models may produce fluent, locally coherent text while neglecting, contradicting, or inappropriately resolving previously introduced setups.
- CFPG reformulates narrative coherence as explicit causal realization using structured foreshadow–trigger–payoff predicates.
- The framework supports evaluation based on whether commitments are realized, postponed, or violated as stories unfold.
- Controlled experiments report improved payoff realization accuracy and narrative alignment over standard prompting baselines.
2 Related Works
Prior story-generation and controllable-generation methods improve local fluency, consistency, planning, or style, but generally do not explicitly encode causally verifiable narrative commitments. CFPG targets this gap with executable foreshadow–payoff constraints.
- Prior story-writing systems primarily emphasize fluent, locally plausible, and stylistically consistent narratives.
- Planning mechanisms improve structural consistency but leave foreshadowed events implicit and payoff realization neither enforced nor causally verifiable.
- Multi-agent and feedback-based systems enhance creativity and engagement while steering narrative progression through qualitative rather than explicit causal conditions.
- CFPG codifies foreshadow–payoff relations as executable narrative constraints for grounded detection and controlled realization of long-range commitments.
- Most controllable text-generation formulations regulate static attributes such as topics, styles, or domains rather than when earlier narrative commitments should be resolved.
3 Method
CFPG represents narrative commitments as Foreshadow–Trigger–Payoff triples and tracks them in an explicit state throughout generation. A select–generate–update loop activates eligible commitments, guides continuation, and maintains temporal grounding.
- Structured Representation of Narrative Commitments: A Foreshadow–Trigger–Payoff triple comprises a setup creating causal debt, a prerequisite condition, and the event that fulfills the commitment.
- Structured Representation of Narrative Commitments: Explicit trigger modeling distinguishes premature payoff, which can spoil suspense, from missing payoff, which creates logical inconsistency.
- CFPG externalizes narrative causality as a codified finite-state abstraction that tracks unfulfilled commitments as explicit resolvable predicates.
- Select–Generate–Update Cycle: At each step, a codification gate compares the current narrative context with each trigger and promotes only satisfied commitments to the active subset.
- Select–Generate–Update Cycle: The language model generates the next scene conditioned on payoffs associated with active commitments as explicit narrative requirements.
- Select–Generate–Update Cycle: After generation, verification removes realized commitments and encodes newly introduced setups as triples for the next state.
4 Dataset
The dataset mines sentence-anchored foreshadow–payoff instances from BOOKSUM summaries through candidate extraction, causal verification, and rubric-based filtering. Each retained instance links a setup to a later, textually supported resolution.
- BOOKSUM provides hierarchical abstractive summaries that distill long-range plot points into discourse-salient events for scalable extraction.
- Candidate identification uses GPT-4.1 to extract pairs anchored to two specific summary sentences, prioritizing recall before filtering.
- A symbolic verification gate rejects metaphorical, anticipatory, or unsupported links and retains genuine causal or narrative resolutions.
- Two independent verifier models apply rubric criteria covering setup validity, payoff validity, temporal separation, and foreshadow justification.
- Pairs enter the final dataset only when both verifier models accept them on all four criteria.
- Each instance records the summary, setup and payoff sentence indices, a relation description, and a categorical foreshadow type.
5 Experiments
The experiments test CFPG under oracle-timed and incrementally revealed contexts, measuring payoff activation, narrative alignment, causal saliency, timing, and error patterns. Across these settings, CFPG improves grounded payoff realization by suppressing premature triggers, sharpening resolution decisions, and linking detection to trajectory-consistent generation.
- Experimental Setup: The experiments evaluate oracle-timed activation, grounded payoff decisions under incremental context, and error attribution during narrative progression.Behavioral evaluation uses a three-class entailment scheme, while grounded tracking analyzes activation timing, localization accuracy, and generative fidelity.
- Conditional Payoff Activation: CFPG achieves near-ceiling Should-Payoff Rates above 0.96 and higher narrative alignment than standard prompting under oracle timing.The controlled setup truncates narratives immediately before known payoff points to isolate the transition from setup to resolution.
- Mechanistic Evidence: CFPG increases attention to foreshadowing setup tokens, producing a pronounced causal-saliency surge relative to baseline prompting.Baseline attention is diffuse and sparse, whereas CFPG concentrates attention on foreshadowing anchors during payoff generation.
- Conclusion: Overall, separating payoff eligibility from surface generation provides model-agnostic control over foreshadow resolution and preserves narrative commitments.The framework aims to ensure that narrative hooks are not merely introduced but meaningfully resolved.
- Grounded Payoff Tracking: CFPG reduces early payoff triggers by 29.3%, helping distinguish genuine causal resolution from semantic proximity.Prompting produced 235 early triggers in GPT-4.1-mini when relevant characters or keywords appeared without satisfying logical prerequisites.
- Grounded Payoff Tracking: CFPG improves trajectory alignment by 43%, bridging the baseline gap between detecting a payoff window and generating a consistent continuation.The baseline Continuation Score was 0.453 for GPT-4.1-mini, indicating that passive recognition did not ensure active narrative commitment.
- Grounded Payoff Tracking: CFPG reduces premature activation, makes the payoff-boundary transition steeper by +0.22, and sustains high activation after resolution.The baseline activates more before the true payoff and rapidly decays afterward, while CFPG maintains a stable causal state.
- Error Attribution: Premature triggering remains the dominant error mode, but CFPG reduces these cases by 31% and lowers Thematic Confusion from 34 to 26 cases.Indirect Failure cases also decrease from 7 to 3, indicating improved linkage between implicit or non-linear payoffs and their original setups.
6 Conclusion
CFPG models narrative coherence as explicit realization of causal commitments by representing foreshadows as executable Foreshadow–Trigger–Payoff predicates. Experiments show improvements over prompt-based baselines in payoff timing, localization, and narrative alignment.
- CFPG models narrative coherence as the explicit realization of causal commitments.
- Executable Foreshadow–Trigger–Payoff predicates support grounded payoff detection and controlled realization under incremental context.
- CFPG improves over prompt-based baselines in payoff timing, localization, and narrative alignment.
Limitations
CFPG targets explicit, textually grounded foreshadow–payoff relations rather than highly abstract or purely symbolic narrative devices.
- CFPG does not aim to model highly abstract or purely symbolic narrative devices.
- The experiments use summary-level narratives, which may not capture all stylistic or discourse-level phenomena in full-length texts.
- Automatically extracted Foreshadow–Trigger–Payoff structures may limit coverage when extraction errors or omissions occur.
A.1 Statistics
The extracted corpus contains 629 validated foreshadow–payoff pairs from 148 books and emphasizes explicit, long-range causal dependencies grounded in observable narrative events. Most foreshadows are object- or event-based, while symbol-based cues are rare.
- 629 validated foreshadow–payoff pairs were extracted from 148 books.
- Payoff distance has a heavy-tailed distribution, with a median of 13 sentences, a mean of 20.9, and dependencies exceeding 200 sentences.
- Object-based foreshadows account for 48.2% and event-based foreshadows 35.3% of the dataset.
- Symbol-based foreshadows are rare at 1.7%, consistent with conservative filtering of purely thematic or interpretive cues.
- The dataset emphasizes explicit, long-range causal dependencies grounded in observable narrative events.
A.2 Quality Check and Reliability Analysis
A quality check of 100 extracted foreshadow–payoff pairs found high agreement overall, with perfect agreement on payoff accuracy and lower agreement on causal connection validity.
- Two annotators independently assessed overall validity and component-level correctness for a random sample of 100 pairs.
- 88% of samples received identical pair-level judgments.
- Setup accuracy agreement reached 95%, while payoff accuracy agreement was perfect.
- Connection validity agreement was 88%, reflecting greater subjectivity in judging long-range narrative reinterpretation.
- Remaining disagreements primarily involved borderline causal-relevance cases rather than factual inconsistencies.
B Metric Definitions for Grounded Payoff Tracking
Grounded payoff tracking is evaluated in an incremental setting where stories are processed sentence by sentence without revising past decisions. The metric suite covers decision timing, localization precision, and continuation quality after payoff detection.
- Decision Timing: Correct Detection Rate measures narratives where payoff decisions occur within ±3 sentences of the annotated payoff location.It evaluates end-to-end detection accuracy under partial context.
- Decision Timing: Early Triggers counts premature payoff decisions before any ground-truth payoff becomes observable, with lower values indicating greater resistance to premature inference.Such errors are typically associated with surface-level cues or anticipatory signals.
- Decision Timing: Late Triggers counts payoff decisions made after the ground-truth payoff point, with lower values indicating better temporal alignment.These errors reflect delayed recognition or overly conservative decision-making.
- Temporal Localization Precision: Localization Error is the average absolute sentence distance between predicted and annotated payoff trigger points across triggered cases.Lower values indicate more accurate temporal alignment during online narrative sensing.
- Generation Fidelity: Continuation Score evaluates generated one-sentence payoff realizations against ground-truth continuations after detection within the tolerance window.A trajectory-based evaluator assesses narrative consistency, causal progression, and outcome.
- Metric Summary: Together, the metrics assess decision timing, temporal localization precision, and generation fidelity under incremental narrative context.The model processes the story sentence by sentence and cannot revise past decisions.