Source-linked AI summary
Event Representations for Automated Story Generation with Deep Neural Nets
Lara J. Martin, Prithviraj Ammanabrolu, Xinyu Wang, William Hancock, Shruti Singh, Brent Harrison, Mark O. Riedl
TL;DR
The paper asks how automated story generation can learn coherent stories from text despite limitations of character-, word-, and sentence-level representations. It introduces event-based preprocessing and a two-stage event2event/event2sentence pipeline, finding that generalized event representations improve successor prediction while requiring translation back to readable language.
Problem
Automated story generation seeks to select event sequences that form stories, but character-, word-, and sentence-level recurrent models have had limited success producing coherent narratives.
Method
The paper preprocesses sentences into semantic event sequences, generates successor events with event2event, and translates abstract events into natural-language sentences with event2sentence.
Results
Generalized events substantially improve perplexity, while bigrams and genre information further improve evaluation outcomes across successor-generation measures.
Takeaways & Limitations
Mid-level generalized events provide reusable story structure for prediction, while event2sentence supplies the details needed to make generated events readable.
Takeaways & Limitations
The successor-event formulation is simplistic because it models story generation as event co-occurrence prediction rather than planning for communicative goals.
Abstract
from arXiv · showhide
Automated story generation is the problem of automatically selecting a sequence of events, actions, or words that can be told as a story. We seek to develop a system that can generate stories by learning everything it needs to know from textual story corpora. To date, recurrent neural networks that learn language models at character, word, or sentence levels have had little success generating coherent stories. We explore the question of event representations that provide a mid-level of abstraction between words and sentences in order to retain the semantic information of the original data while minimizing event sparsity. We present a technique for preprocessing textual story data into event sequences. We then present a technique for automated story generation whereby we decompose the problem into the generation of successive events (event2event) and the generation of natural language sentences from events (event2sentence). We give empirical results comparing different event representations and their effects on event successor generation and the translation of events to natural language.
Introduction
The paper addresses open story generation by learning from textual corpora rather than relying on manually engineered domains. It proposes mid-level event representations and a two-stage neural pipeline to generate event sequences and readable sentences.
- Symbolic and case-based systems require human-authored domain models, restricting stories to covered topics and complicating attribution of quality to algorithms.
- Open story generation aims to create stories about any topic without prior manual domain knowledge engineering.
- Character- and word-level models can produce grammatical sentences but often fail to maintain coherent narratives, whereas sentence-level events are too sparse.
- The proposed event representation preserves core semantic information while reducing event sparsity, allowing models to learn reusable story skeletons.
- The paper evaluates the event representation against sentence and alternative representations within its neural story-generation pipeline.
- The event2event network generates successive abstract events, while event2sentence translates them into human-readable natural-language sentences.
Related Work
Related work spans symbolic planning, case-based reasoning, learned domain models, content retrieval, recurrent neural networks, script-based causal chains, and classification-based story completion.
- Early story-generation systems used symbolic planning or case-based reasoning but generated stories only in predetermined, well-defined domains.
- SayAnything retrieves relevant existing story content from online blogs, while Scheherazade learns a domain model from crowdsourced example stories.
- Recurrent neural approaches model story sequences using representations such as Skip-thought sentence vectors and high-dimensional sentence embeddings.
- Script-learning work represents events through verbs and their dependency relations to a protagonist, emphasizing causal chains around that protagonist.
- Pichotta and Mooney represent events as verb, subject, direction object, prepositional object, and preposition, inspiring the paper’s alternative representation.
- The story cloze test frames generation as choosing between two candidate fifth sentences using embeddings, sentiment, and stylistic features.
Event Representation
The paper represents stories as structured events positioned between words and sentences, then tests abstraction and contextual variants for successor prediction. Its core representation extracts semantic roles while allowing generalization and genre information.
- Story generation predicts a successor event from a sliding window of preceding events using a parameterized probability distribution.
- The representation targets a balance between semantic knowledge and predictive power because excessive event sparsity can make generation effectively random.
- The basic event is the 4-tuple ⟨s, v, o, m⟩: subject, verb, object, and a wildcard modifier for additional dependencies.
- Dependency parsing extracts events from sentences, inserting EmptyParameter when an object or modifier cannot be identified.
- Conjoined subjects or verbal structures can yield multiple events per sentence; the reported average was 2.69 events per sentence.
- Experiments used 42,170 Wikipedia movie-plot stories averaging 14.515 sentences per story.
- Generalized representations replace named entities and nouns with abstract tags or WordNet hypernyms to increase abstraction and reduce sparsity.
- Genre information adds a 100-cluster topic-model category as a fifth tuple element ⟨s, v, o, m, g⟩.
Event-to-Event
The event2event model predicts successor events using recurrent encoder-decoder networks over alternative event representations. Results favor generalized representations for perplexity, history-aware bigrams for BLEU, and unordered multi-event inputs for perplexity, while highlighting a trade-off between prediction and language realization.
- The experiments compare 11 event representations against a sentence-level baseline for predicting successor events.The representations vary in abstraction, entity handling, genre information, history, and multiple-event treatment.
- Adding genre information improves perplexity, but its equal weighting with other event elements artificially inflates BLEU during evaluation.Genre is represented as a fifth tuple parameter during training and removed from inputs and outputs before testing.
- Generalizing named entities, words, and verbs substantially improves perplexity over original-word and sentence representations.Original word events have similar perplexity to original sentences, while further generalization produces much better scores.
- Generalized event bigrams improve BLEU to nearly the level of original-word events by incorporating preceding-event history.The bigram experiments provide additional context for successor prediction and increase generated-to-expected n-gram overlap.
- The best perplexity occurs when all generalized events extracted from one sentence jointly predict all events from the next sentence.This setup avoids forcing an arbitrary order among events originating in the same sentence.
- Greater generalization aids successive-event generation but makes translating events back into natural-language sentences more difficult.The pipeline therefore separates event2event prediction from event2sentence realization, and evaluates both representation effects.
Event-to-Sentence
The event2sentence network translates abstract event representations back into natural-language sentences, addressing the lossiness and unreadability introduced by event preprocessing. Experiments compare generalized and original-word representations, including split-and-pruned inputs, showing a trade-off between sparsity reduction and recoverable detail.
- Event2Sentence: Because event extraction is linear and lossy, event2sentence must restore details that are absent from the abstract event representation.A single event can correspond to multiple plausible source sentences, making back-translation non-trivial.
- Event2Sentence: Event2sentence translates events into natural-language sentences using parallel training pairs of events and the sentences from which they were extracted.In the complete pipeline, event2event generates successor events before event2sentence renders them for human readers.
- Evaluation: Beam search with beam width B=5 maintains five partial sentence hypotheses and expands them using the highest-probability next tokens.Decoding continues until an end-of-sentence tag is reached.
- Evaluation: Generalization drastically improves perplexity, while perplexity in these experiments appears to correspond to vocabulary size.For event2sentence, BLEU is considered the more meaningful metric because the task is translation.
- Representation comparisons: Generalized events with full-length generalized sentences achieve better BLEU scores than representations using original words.The generalized setup replaces named entities and nouns with abstract categories or WordNet synsets while retaining verbs.
- Representation comparisons: When split-and-pruned sentences retain original words, BLEU performance improves relative to the generalized alternative, reversing the full-length pattern.The authors attribute the reversal to combined sparsity reduction removing too much distinguishing information.
Future Work
Future work extends the pipeline with memory modules for restoring omitted story details and with planning-oriented generation for communicative goals. The proposed memory design reuses recent and historical entities, while reinforcement learning is intended to replace successor-event prediction.
- Memory modules: Working Memory would retain removed character names and nouns so event2sentence can refill placeholders after generating a partially generalized sentence.Long-Term Memory would preserve a history of entities and nouns together with how long ago they were last used.
- Memory modules: Continuous named-entity numbering is designed to help event bigrams maintain character turn-taking patterns across successive events.The motivating example alternates John and Andrew as the acting character.
- Memory modules: The memory design assumes that stories are more likely to reuse existing entities and concepts than introduce new ones.Long-Term Memory is intended to support reuse when the recent event bigram lacks the needed placeholder content.
- Planning and communicative goals: The current successor-event model is simplistic because it assumes stories can be generated from generalized event co-occurrence patterns.The authors identify topic, theme, and specified endings as communicative goals that planning could address.
- Planning and communicative goals: Future work proposes replacing event2event with reinforcement learning that performs lookahead over whether successor events satisfy communicative intent.The proposed planning formulation contrasts with treating story generation solely as successor-event prediction.
Conclusions
The experiments support event representations that abstract away from natural language while preserving useful semantics, improving event successor generation but creating a readability challenge addressed by event2sentence. Story coherence remains unevaluated.
- Conclusions: More abstract event representations improved the generative ability of recurrent neural networks during event2event story generation.The conclusion reports that experiments with different representations backed the hypothesis that representation choice affects sparsity and performance.
- Conclusions: Event bigrams did not significantly harm the generative model and may help coherence by incorporating more history.The paper notes that coherence was difficult to measure and was not evaluated.
- Conclusions: Generalizing events away from natural language can improve successor generation but makes generated story content unreadable.The event2sentence network translates generalized or original-word events back into natural-language sentences.
- Conclusions: Event2sentence is intended to recover readable sentences and fill in missing specifics for dynamic storytelling.The paper presents a pipeline architecture for filling in details in partially generalized sentences.
- Conclusions: Neural-network methods are presented as a path toward open story generation from comprehensive story-text corpora rather than hand-authored domain knowledge.The paper contrasts this direction with earlier systems that relied on human-authored symbolic domain knowledge.