Source-linked AI summary
Narrative Interpolation for Generating and Understanding Stories
Su Wang, Greg Durrett, Katrin Erk
TL;DR
Controlled storytelling requires coherent narratives that follow user guidance without demanding a detailed storyline. The paper introduces INTERPOL, which interpolates bridging events between a beginning and ending using GPT-2 and coherence reranking. Human evaluation finds the resulting stories on-par or better than prior guidance methods, while the paper notes limited commonsense inference by large language models.
Problem
Controlled storytelling must produce logically consistent, topically coherent, and pragmatically reasonable stories while existing topic and storyline guidance have important control or usability limitations.
Source: §1 · p. 1Method
Results
Human evaluation finds the method on-par or better than prior state-of-the-art guidance techniques in coherence, faithfulness, and overall preference despite reduced user guidance.
Source: §1 · p. 2Takeaways & Limitations
Ending-guided interpolation provides flexible narrative control with less manual guidance than prior storyline-based approaches.
Source: §1 · p. 2, §5 · p. 4, §6 · p. 4The findings indicate that large-scale language models manage commonsense inference only to a small extent, leaving a need for human-provided knowledge.
Source: §6 · p. 4
Abstract
from arXiv · showhide
We propose a method for controlled narrative/story generation where we are able to guide the model to produce coherent narratives with user-specified target endings by interpolation: for example, we are told that Jim went hiking and at the end Jim needed to be rescued, and we want the model to incrementally generate steps along the way. The core of our method is an interpolation model based on GPT-2 which conditions on a previous sentence and a next sentence in a narrative and fills in the gap. Additionally, a reranker helps control for coherence of the generated text. With human evaluation, we show that ending-guided generation results in narratives which are coherent, faithful to the given ending guide, and require less manual effort on the part of the human guide writer than past approaches.
1 Introduction
The paper introduces ending-guided narrative interpolation to generate coherent stories with less detailed human guidance. Human evaluation finds the method comparable or superior to prior guidance techniques in coherence, faithfulness, and overall preference.
- Unlike topic or storyline guidance, the method aims to reduce manual guidance while avoiding loosely connected events and sensitivity to keyphrase choices.Source: §1 · p. 1, Table 2 · p. 1
- Ending-guided interpolation generates bridging events between a beginning and user-specified ending to form a coherent story.Source: §1 · p. 1
- The generator uses GPT-2 to propose intermediate sentences from left-right context, while a RoBERTa coherence ranker rejects poor generations.Source: §1 · pp. 1–2
- Human evaluation shows stories are on-par or better than prior state-of-the-art guidance in coherence, faithfulness, and overall preference despite reduced user guidance.Source: §1 · p. 2
2 Related Work
Prior controlled-story methods range from simple topic conditioning to fine-grained storylines. INTERPOL occupies an intermediate position by using more guidance than a topic but less than a full storyline.
- Topic-conditioned models generate longform stories from simple topic inputs, while providing less fine-grained content control than full storylines.Source: §2 · p. 2
- Storyline methods provide finer content control but require suitable granularity and can be sensitive to keyphrase choices.Source: §2 · p. 2
- INTERPOL balances these approaches with less fine-grained guidance than full storylines but more guidance than a general topic.Source: §2 · p. 2
- Prior rerankers target errors such as repetition, contradiction, and other generation failures using automatically generated negative examples.Source: §2 · p. 2
3 Task and Data
The task is to construct a fixed-length coherent story from a beginning and user-specified ending by interpolating the intervening sentences. Experiments use the ROCStories corpus and a 0.9/0.05/0.05 split.
- Given beginning b, ending e, and length k, the system generates k−2 intermediate sentences forming a coherent k-sentence story.Source: §3 · p. 2
- The experiments use ROCStories, a collection of 98,162 human-written five-sentence English stories.Source: §3 · p. 2
- The dataset has a vocabulary of 33,215 words, average sentence length of 10, and a 0.9/0.05/0.05 train/dev/test split.Source: §3 · p. 2
4 Modeling
INTERPOL iteratively fills gaps between story sentences, combining GPT-2 candidate generation with RoBERTa-based coherence ranking. The procedure uses left-right context and a bisectional insertion order, while sequential interpolation showed no significant perplexity difference.
INTERPOL procedure
- INTERPOL starts with a beginning and ending, repeatedly proposes intermediate candidates at insertion points, and selects the best candidate using story-level coherence.Source: §4 · p. 2
Text generator
- The generator uses GPT-2 to sample an interpolated sentence from concatenated right and left context, placing the future sentence first.Source: §4 · pp. 2–3
Insertion order
- Compared with purely sequential interpolation, the bisectional approach produced no statistically significant difference in wordpiece perplexity.Source: §4 · p. 2
- Bisectional interpolation fills the middle before adjacent gaps, such as generating s3 from s1 and s5 before s2 and s4.Source: Figure 1 · p. 3
Coherence ranker
- A RoBERTa ranker scores complete story segments for coherence and is trained with real stories plus negatives representing repetition, irrelevance, and out-of-order sentences.Source: §4 · p. 3
5 Experiments and Analysis
The experiments evaluate ending-guide informativeness, coherence ranking, and ending-guided generation against sequential and storyline-based baselines using human judgments and wordpiece perplexity. INTERPOL is preferred across coherence, faithfulness, and overall story preference, while requiring less manual guidance than storyline conditioning.
Evaluation
- The evaluation measures story quality through coherence, faithfulness to the beginning and ending, and overall human preference.Human evaluation is used because the cited prior work identifies it as the reliable assessment method for story-generation quality.Source: §5 · p. 3
Baselines
- The baselines include LEFTTORIGHT without right context, NORANKING without candidate selection, and PAW using storyline conditioning.PAW is reformulated from Yao et al. (2019), with storylines represented as chains of keyphrases.Source: §5 · pp. 3–4
Is the ending guide informative?
- Ending conditioning provides useful context for generating intermediate sentences, as measured by lower wordpiece perplexity for models using the story ending.Table 3 compares LEFTTORIGHT and NORANKING in both single-sentence and full-story generation settings.Source: Table 3 · p. 4, §5 · p. 4
Is coherence ranking helpful?
- In a human evaluation of 30 entries, INTERPOL was significantly preferred over NORANKING on all three evaluated axes, demonstrating the effectiveness of coherence ranking and higher overall story quality.The evaluation sampled 10 beginning–ending pairs, crowdsourced two alternative endings for each, and had three workers assess stories generated by both models.Source: Table 4 · p. 4, §5 · p. 4
Storyline-guide or ending-guide?
- Humans prefer INTERPOL over PAW across coherence, faithfulness, and overall preference, indicating stronger performance for ending-guided than storyline-guided generation.The comparison uses 90 sampled inputs with respective control sources.Source: Table 5 · p. 4, §5 · p. 4
6 Conclusion
The paper presents interpolation with pretrained language models as a simple method for controllable storytelling. Human evaluation finds more coherent, flexibly controlled, and preferred stories, while the authors note limited commonsense inference as a remaining challenge.
Conclusion
- The method uses modern pretrained language models and is presented as a simple, intuitive approach to storytelling and narrative understanding.The conclusion frames the method as a contribution to better narrative understanding.Source: §6 · p. 4
- Human evaluation shows the models generate more coherent stories than previous work and allow users to control story content flexibly.Users also prefer the generated stories as stories.Source: §6 · p. 4
Future directions
- Large-scale language models manage commonsense inference only to a small extent, motivating human-provided knowledge in future systems.The authors identify more specific organization of events and participants as another future direction.Source: §6 · p. 4