Source-linked AI summary
Improving Neural Story Generation by Targeted Common Sense Grounding
Huanru Henry Mao, Bodhisattwa Prasad Majumder, Julian McAuley, Garrison W. Cottrell
TL;DR
Neural language models can write stylistically coherent stories while producing logical inconsistencies, and next-token perplexity alone may be insufficient for sensible story generation. The paper uses auxiliary common-sense ranking tasks in a two-stage fine-tuning pipeline, reporting improved common-sense reasoning and state-of-the-art perplexity on WritingPrompts.
Problem
Neural language models can generate stylistically coherent stories with logical inconsistencies, raising the question of whether next-token perplexity alone is sufficient for sensible story generation.
Source: §1 · p. 1Method
The method combines BookCorpus domain adaptation with WritingPrompts fine-tuning and auxiliary perplexity ranking on common-sense and synthetic examples.
Source: §2.2 · p. 2, §3 · p. 3Results
The two-stage multi-task method improves common-sense reasoning and achieves state-of-the-art perplexity on WritingPrompts.
Source: §1 · p. 1Takeaways & Limitations
The authors demonstrate a simple way to improve the likelihood that a language model generates sensible text by leveraging transfer and multi-task learning.
Source: §7 · p. 4The approach depends on SWAG, which provides only short-term common-sense grounding, and story-level coherence is unlikely from this method alone.
Source: §7 · pp. 4–5
Abstract
from arXiv · showhide
Stories generated with neural language models have shown promise in grammatical and stylistic consistency. However, the generated stories are still lacking in common sense reasoning, e.g., they often contain sentences deprived of world knowledge. We propose a simple multi-task learning scheme to achieve quantitatively better common sense reasoning in language models by leveraging auxiliary training signals from datasets designed to provide common sense grounding. When combined with our two-stage fine-tuning pipeline, our method achieves improved common sense reasoning and state-of-the-art perplexity on the Writing Prompts (Fan et al., 2018) story generation dataset.
1 Introduction
Neural language models can produce stylistically coherent stories that nevertheless violate common sense. The paper proposes evaluating and training for common-sense reasoning through likelihood-based ranking, reporting improved common-sense reasoning and state-of-the-art perplexity on WritingPrompts.
1 Introduction
- GPT-2 can produce stylistically coherent text with logical inconsistencies, suggesting next-token perplexity alone may be insufficient for sensible stories.The passage illustrates the issue with a sentence describing a sunny, warm summer night.Source: §1 · p. 1
- The proposed automatic evaluation ranks model perplexity on plausible but nonsensical completions from SWAG and Story Cloze.These datasets are designed for common-sense grounding.Source: §1 · p. 1
- The two-stage multi-task approach improves common-sense reasoning and achieves state-of-the-art perplexity on WritingPrompts.The contribution combines an auxiliary multi-task objective with a two-stage fine-tuning pipeline.Source: §1 · p. 1
2 Tasks
The primary task models concatenated WritingPrompts prompts and stories as text sequences. The auxiliary task trains the model to assign higher likelihood to sensible choices than to plausible but spurious alternatives.
2.1 Primary Task: Language Modeling
- WritingPrompts language modeling trains on prompt-story pairs as a sequence, using cross-entropy between true next tokens and predicted probabilities.The task is reduced from conditional generation by concatenating the prompt before the story.Source: §2.1 · p. 2
- WritingPrompts prompts are concatenated with their stories and treated as one input to a Transformer decoder language model.The format allows sampling from a supplied prompt or generating a prompt first.Source: §2.1 · p. 2
2.2 Auxiliary Task: Perplexity Ranking
- The auxiliary task defines better common-sense behavior as assigning a sensible sequence higher probability than the average probability of spurious sequences.Perplexity ranking optimizes the model to rank sensible text below spurious text in perplexity.Source: §2.2 · p. 2
- SWAG provides four-way common-sense choices expressed as full sentences that can be concatenated and scored as a single string.Its adversarially filtered choices are designed to seem plausible to language models.Source: §2.2 · p. 2
- The synthetic dataset pairs 250K human-written WebText passages with samples generated by the original 1.5B-parameter GPT-2.The pairs are framed as a two-way ranking task under the assumption that human-written text is more sensible on average.Source: §2.2 · p. 2
3 Training Pipeline
The pipeline first adapts pretrained GPT-2 to the story domain using BookCorpus, then fine-tunes on WritingPrompts with alternating language-modeling and perplexity-ranking objectives.
3 Training Pipeline
- The pipeline is designed to improve both perplexity and common-sense reasoning in story generation.The model uses the 117M-parameter GPT-2 architecture with pretrained weights.Source: §3 · p. 2
- The pipeline first fine-tunes pretrained GPT-2 on BookCorpus for domain adaptation, then alternates WritingPrompts language modeling with perplexity ranking on SWAG and synthetic data.BookCorpus contains over 10,000 free books; the target-stage objectives use WritingPrompts as the primary task and SWAG plus synthetic examples as auxiliary tasks.Source: §3 · p. 3
4 Evaluation
The evaluation measures readability, prompt matching, and common-sense reasoning (CSR). It uses perplexity-based measures for readability and ranks candidate prompts or answers by likelihood.
Readability
- WritingPrompts readability is measured with test-set perplexity, reported at both subword and estimated word level.Word-level perplexity is estimated by multiplying the probabilities of a word’s subword tokens.Source: §4 · p. 3
Prompt ranking
- Prompt ranking tests whether a story is most likely under its true prompt among ten candidates, using accuracy over 1,000 random samples.Nine prompts are randomly sampled; the true prompt is the tenth candidate.Source: §4 · p. 3
Common-sense reasoning
- CSR accuracy on SWAG and Story Cloze counts an example correct when the answer with the lowest perplexity is correct.SWAG validation estimates generalization to unseen examples; Story Cloze validation provides a zero-shot test in story-telling.Source: §4 · p. 3
5 Results and Discussion
The results report improvements from intermediate and two-stage fine-tuning, with multi-task training further improving common-sense reasoning. The added auxiliary tasks negligibly affect other metrics.
Intermediate Fine-Tuning
- Intermediate fine-tuning on BookCorpus improves zero-shot WritingPrompts perplexity and prompt ranking compared with GPT2, indicating domain adaptation.The reported comparison evaluates the BookCorpus-fine-tuned model without further WritingPrompts fine-tuning.Source: §5 · p. 3
Pre-training
- Zero-shot GPT2 is a strong baseline; GPT2 fine-tuned on WritingPrompts beats the Fusion Model in perplexity, and all models beat it in prompt ranking.The authors suggest task-specific models are unnecessary for prompt ranking given pretraining.Source: §5 · p. 3
Multi-tasking Fine-Tuning
- Multi-tasking on WritingPrompts and SWAG improves SWAG validation and zero-shot Story Cloze performance; synthetic examples further boost CSR, while other metrics are negligibly affected.The table reports WritingPrompts test results and SWAG and Story Cloze validation results.Source: §5 · p. 4, Table 3 · p. 4
Intermediate Fine-Tuning
- Two-stage fine-tuning (GPT2 →BC →WP) further improves perplexity and CSR compared with GPT2.The authors hypothesize that CSR improves because BookCorpus is higher quality and author-written compared with WebText.Source: §5 · p. 4
6 Related Work
The paper places its work among hierarchical story-generation methods, common-sense benchmarks, and transfer and multi-task learning. It uses auxiliary perplexity ranking to target common-sense reasoning in generated text.
Story Generation
- Prior hierarchical story-generation methods generate an outline or prompt before expanding it into a story (Kiros et al., 2015; Roemmele, 2016; Yao et al., 2018; Fan et al., 2018).This work also performs hierarchical generation but focuses on common sense rather than task-specific architectures.Source: §6 · p. 4
Common Sense Reasoning
- Common-sense reasoning has been studied on benchmarks including SWAG, Story Cloze, the Winograd Schema Challenge, and CommonsenseQA (Zellers et al., 2018; Mostafazadeh et al., 2016; Levesque et al., 2012; Talmor et al., 2018).Recent benchmark methods emphasize large-scale pretraining (Peters et al., 2018; Radford et al., 2018), which this fine-tuning pipeline builds upon.Source: §6 · p. 4
- The authors say their SWAG and Story Cloze results are far from state of the art (Devlin et al., 2018), since those datasets serve as constraints and proxies, not direct targets.Source: §6 · p. 4
Multi-task Learning
- Unlike conventional auxiliary multi-task learning with an added classifier (Howard and Ruder, 2018), their parameter-free perplexity ranking is argued to favor sensible text.The work builds on multi-task learning principles (Caruana, 1993; Ruder, 2017; McCann et al., 2018) and auxiliary tasks for common-sense reasoning (Søgaard and Bingel, 2017).Source: §6 · p. 4
7 Conclusion
The paper concludes that transfer and multi-task learning can improve a language model’s perplexity and common-sense reasoning. Its grounding is limited by SWAG’s short-term coverage, and story-level coherence is unlikely from this method alone.
Conclusion
- The pipeline combines transfer and multi-task learning to improve perplexity and common-sense reasoning in the language model.Source: §7 · p. 4
Limitations
- The approach depends on SWAG, which provides only short-term common-sense grounding; the synthetic dataset may mitigate this limitation.Source: §7 · pp. 4–5
- Story-level coherence is unlikely to be achieved using this method alone.Source: §7 · p. 5
8 Training Details
Training uses a two-stage pipeline: BookCorpus domain adaptation followed by multi-task fine-tuning on WritingPrompts, SWAG, and a synthetic dataset.
8 Training Details
- The two-stage pipeline first adapts on BookCorpus, then fine-tunes on WritingPrompts, SWAG, and a synthetic dataset.Training alternates among the three second-stage datasets, with WritingPrompts trained every iteration, the synthetic dataset every 15 iterations, and SWAG every 30.Source: §8 · p. 6
- Stage one uses triangular learning-rate scheduling, warming up for 1,000 iterations to 5 × 10−5 before decaying to zero over 99,000 iterations.Experiments use batch size 16 and eight NVIDIA 2080Ti GPUs.Source: §8 · p. 6
- Stage two weights WritingPrompts more heavily than the auxiliary tasks and uses its validation set for early stopping.The same learning-rate schedule as stage one is applied.Source: §8 · p. 6
9 Samples
The section presents generated samples from models with and without auxiliary objectives, including outputs seeded with prompts used by Fan et al. (2018).
9 Samples
- The examples include unconditional full stories, in which the model also generates the prompts.Source: §9 · p. 6
- Table 5 presents samples from GPT2 →BC →WP, whereas Table 6 presents samples from GPT2 →BC →WP + SWAG + SYNTH; the paper also includes stories from models with and without auxiliary multi-task objectives.Source: §9 · p. 6, Table 5 · p. 7, Table 6 · p. 8
- Table 7 shows samples from GPT2 →BC →WP + SWAG + SYNTH primed with the same prompts as Fan et al. (2018).The best model is also sampled using the exact prompt used to seed Fan et al.’s samples.Source: §9 · p. 6, Table 7 · p. 9