Source-linked AI summary
Plan-And-Write: Towards Better Automatic Storytelling
Lili Yao, Nanyun Peng, Ralph Weischedel, Kevin Knight, Dongyan Zhao, Rui Yan
TL;DR
Automatic storytelling must produce long, coherent sequences of events, while prior methods are limited in plot planning or domain scope. This paper plans a storyline before realizing a title-conditioned story, comparing dynamic and static schemas. Explicit planning yields more diverse, coherent, and on-topic stories than systems without full planning, though the models remain limited by simplified storylines and coherence challenges.
Problem
Automatic storytelling seeks to generate coherent natural-language stories from titles, but prior work is limited in plot planning or domain scope.
Method
The plan-and-write framework first plans a storyline and then generates a story from it, comparing dynamic and static planning schemas.
Results
Explicit storyline planning produces more diverse, coherent, and on-topic stories than systems without planning, with both dynamic and static schemas outperforming no-planning baselines.
Takeaways & Limitations
Storyline planning improves title-conditioned open-domain story generation and allows human–computer interaction at the abstract storyline level.
Takeaways & Limitations
The models use simplified word-sequence storylines and still struggle to generate sequences of coherent and logically consistent sentences.
Abstract
from arXiv · showhide
Automatic storytelling is challenging since it requires generating long, coherent natural language to describes a sensible sequence of events. Despite considerable efforts on automatic story generation in the past, prior work either is restricted in plot planning, or can only generate stories in a narrow domain. In this paper, we explore open-domain story generation that writes stories given a title (topic) as input. We propose a plan-and-write hierarchical generation framework that first plans a storyline, and then generates a story based on the storyline. We compare two planning strategies. The dynamic schema interweaves story planning and its surface realization in text, while the static schema plans out the entire storyline before generating stories. Experiments show that with explicit storyline planning, the generated stories are more diverse, coherent, and on topic than those generated without creating a full plan, according to both automatic and human evaluations.
Introduction
The paper addresses open-domain story generation from titles by combining explicit storyline planning with natural-language realization. It compares planning strategies and reports gains in diversity, coherence, and topicality over systems without planning.
- Automatic storytelling requires coherent natural language describing a sensible, causally linked sequence of events, making it harder than generation from an existing plan or knowledge fragment.
- The framework decomposes story generation into story planning, which generates plots, and surface realization, which composes text from those plots.
- Storylines provide a simple representation that supports automatic extraction of training data without additional human annotation.
- The paper explores dynamic planning, which adjusts the plot during writing, and static planning, which plans the entire plot before writing.
- The paper develops diversity metrics and analyzes which story aspects matter for human evaluation.
- The plan-and-write model generates more diverse, coherent, and on-topic stories than systems without planning, according to the experiments.
Plan-and-Write Storytelling
The framework takes a title, plans an intermediate storyline represented as an ordered word sequence, and generates a story conditioned on that plan. Storyline training data are extracted automatically from existing story corpora.
- The system receives a title as input and generates a multi-sentence story as output.
- A storyline is an intermediate plot representation consisting of an ordered sequence of words, with one word corresponding to each storyline position.
- The plan-and-write framework always plans a storyline and includes dynamic and static schema variations.
- Training storylines are extracted from existing corpora by selecting one important word from each sentence using the RAKE algorithm.
Methods
The framework alternates or separates storyline planning and story generation, using title and contextual information to guide coherent text production. It compares dynamic planning during writing with static planning before writing.
- Dynamic schema: The dynamic schema generates the next storyline word and story sentence at each step, conditioning on existing storyline and previously generated sentences.It emphasizes flexibility by interleaving planning with surface realization.
- Dynamic schema: Dynamic storyline planning models the next word from the title, previous sentences, and previous storyline word using a BiGRU context encoder.The context is encoded bidirectionally, then auxiliary storyline information is incorporated during decoding.
- Dynamic schema: Story generation proceeds incrementally by alternating planning and writing, with each sentence generated from context and an additional storyline-word cue.The storyline and story models differ because the former generates a word while the latter generates a variable-length sentence sequence.
- Representation and optimization: Storylines contain one important word per sentence, extracted with RAKE, and the separately trained planning and story models are combined in a pipeline.A heuristic also forbids repeated words in generated storylines to reduce repetition.
- Static schema: The static schema first generates a complete storyline from the title, then generates the story from the fixed plan and title.This sacrifices flexibility but provides look-ahead information intended to enhance coherence.
Experimental Setup
The experiments use ROCStories to compare planned story generation with baselines that omit planning. Evaluation combines diversity-oriented automatic metrics with pairwise human judgments of story quality.
- Dataset: ROCStories provides five-sentence commonsense stories with causal and temporal relations; training data are split 8:1:1 for training, validation, and testing.Titles are available only in the original training set, which supplies the title-conditioned input.
- Baselines: The plan-and-write methods are evaluated against representative baselines without a planning module, including incremental sentence-to-sentence generation and whole-story conditional language modeling.Inc-S2S conditions each sentence on the title and previously generated sentences, whereas Cond-LM generates the story word by word from the title.
- Training: All models use RNN-based conditional generation with SGD and receive the same hyper-parameter optimization procedure, including grid search over embedding size, hidden-vector size, and dropout.Word-embedding dimensions are tuned over [100, 200, 300, 500].
- Automatic evaluation: Automatic evaluation targets diversity because n-gram-overlap metrics such as BLEU are considered unsuitable for human-like story generation.The proposed measures quantify inter-story repetition at each sentence position and intra-story repetition against preceding sentences using trigram statistics.
Results and Discussion
The plan-and-write framework improves story diversity, human-rated quality, and storyline–story alignment, with static planning generally strongest. Error analysis nevertheless identifies off-topic, repetitive, and logically inconsistent outputs as remaining problems.
- Objective evaluation: 9816 generated stories were evaluated for repetition, with lower inter- and intra-story repetition indicating greater diversity.Figure 3 uses sentence-level curves and whole-story bars; human-written training data provides 34% inter-story and 0.3% intra-story aggregate reference rates.
- Objective evaluation: Plan-and-write methods significantly reduce inter-story repetition versus all non-planning methods and improve aggregate intra-story repetition against corresponding baselines.The dynamic schema produces more repetitive final sentences than the baselines, despite the broader reductions.
- Subjective evaluation: Both dynamic and static schemas significantly outperform their corresponding baselines on fidelity, coherence, interestingness, and overall user preference.Human evaluation compares generated stories pairwise across four aspects using 300 sampled titles and 233 Turkers.
- Subjective evaluation: Coherence and interestingness weigh more heavily than fidelity in predicting human overall preferences.The authors estimate relative importance with a linear regression over human assessments.
- Analysis: Static planning generates higher-quality storylines and stronger storyline–story correlation than dynamic planning, partially explaining its better performance.The reported storyline coverage is 75% for dynamic and 78% for static planning.
- Analysis: Manual review of 50 static-schema stories finds three major problems: off-topic, repetitive, and logically inconsistent generations.The system can produce locally coherent grammatical sentences, but globally coherent and logically consistent sequences remain an open challenge.
Related work
Earlier storytelling systems often focused on plot planning, specialized domains, or additional structure, while this work combines automatic storyline generation with surface realization from titles. The paper distinguishes its approach from close two-stage systems through full-story experiments, title-only inputs, and comparison of dynamic and static planning.
- Earlier approaches: Traditional systems used symbolic planning or case-based reasoning to select, sequence, or adapt story actions.These approaches aimed to compose sensible plots under specified criteria or storytelling requirements.
- Domain expansion: Some learned-domain approaches broadened story settings but remained at the plot-planning level without surface realization.Examples mined personal stories or used crowd-sourced corpora to learn domain models for unknown domains.
- Event structures: Event-structure research extracted or modeled sequences of events, while other work generated story variants from symbolic representations or used additional information.The cited line of work centers on representing story event structures rather than automatically generating paper-style storylines from titles.
- Closest prior work: Closest two-stage work decomposed generation into story structure modeling and structure-to-surface generation, but differs in evaluation scope and input setting.Martin et al. did not test full story generation; Xu et al. used more specific story prompts, whereas this paper requires only a title and compares two planning strategies.
- Neural generation: Deep learning work in story generation includes RNNs trained on highly specialized textual corpora.This line of research is presented as part of broader neural natural-language-generation approaches.
Conclusion and Future Work
The paper concludes that explicit storyline planning improves title-to-story generation, with both proposed schemas outperforming unplanned baselines. Future work targets richer plot representations, longer documents, and reduced pipeline error propagation.
- Both dynamic and static plan-and-write strategies outperform baselines without planning components.
- The static schema generates more coherent and relevant stories than the dynamic schema by planning the storyline holistically.
- Current models approximate storylines with word sequences, simplifying meaningful entity, event, and relation structures in real plots.
- Future work will explore richer plot representations and extend the framework to longer documents.
- The authors also plan storyline induction and joint storyline-story generation to avoid error propagation in the current pipeline.
Appendix
The appendix documents hyper-parameter settings, testing-set BLEU scores, and the AMT human-evaluation survey interface.
- Figure 5 shows a snapshot of the AMT survey used for human evaluation.
- Table 7 lists the best hyper-parameter settings for the baselines and proposed models.It defines suffixes for dimensions, dropout, embedding and hidden layers, and storyline and story settings.
- Table 8 reports BLEU scores on the testing set.The suffix “-W.O.” denotes no optimization for storylines.