PaperlayerView exact revision on arXiv

Source-linked AI summary

Re3: Generating Longer Stories With Recursive Reprompting and Revision

Kevin Yang, Yuandong Tian, Nanyun Peng, Dan Klein

arXiv:2210.06774v3cs.CLcs.AI

TL;DR

Automatically generating stories beyond two thousand words requires sustained plot coherence and premise relevance, challenges less central in shorter-story generation. Re3 addresses them with structured planning, recursive context injection, continuation reranking, and factual-consistency editing; human evaluators reported 14% and 20% absolute gains in plot coherence and premise relevance over similar-length GPT3-based rolling-window baselines.

  • Problem

    Automatically generating stories over two thousand words requires maintaining long-range plot coherence and premise relevance, alongside style and factual consistency, over thousands of words.

  • Method

    Re3 prompts GPT3 to create a structured plan, recursively injects relevant plan and story-state context during drafting, reranks continuations, and edits the selected continuation for factual consistency.

  • Results

    14% absolute increase in stories judged coherent in overarching plot and 20% increase in stories judged relevant to the initial premise were reported against similar-length GPT3-based rolling-window baselines.

  • Takeaways & Limitations

    Re3 improves long-range coherence and premise relevance while generating stories over 2000 words using general-purpose language models.

  • Takeaways & Limitations

    Evaluation difficulty limited experiment sample sizes and detailed ablations, and prevented formal evaluation of stories longer than 2000–2500 words.

Abstract

from arXiv · show

We consider the problem of automatically generating longer stories of over two thousand words. Compared to prior work on shorter stories, long-range plot coherence and relevance are more central challenges here. We propose the Recursive Reprompting and Revision framework (Re3) to address these challenges by (a) prompting a general-purpose language model to construct a structured overarching plan, and (b) generating story passages by repeatedly injecting contextual information from both the plan and current story state into a language model prompt. We then revise by (c) reranking different continuations for plot coherence and premise relevance, and finally (d) editing the best continuation for factual consistency. Compared to similar-length stories generated directly from the same base model, human evaluators judged substantially more of Re3's stories as having a coherent overarching plot (by 14% absolute increase), and relevant to the given initial premise (by 20%).

1 Introduction

Re3 targets automatically generated stories of 2000–2500 words, where maintaining overarching plot coherence, premise relevance, style, and factual consistency creates challenges beyond shorter-story generation. It uses a structured plan, recursive context injection, and revision modules to improve long-range generation quality.

  • Motivation: Stories of 2000–2500 words extend substantially beyond prior automatically generated short stories and are intended to resemble human-consumable short fiction.Prior work often generated five sentences to one or two paragraphs, whereas the paper targets several-page-length stories.
  • Challenges: Long-form generation must preserve an overarching plot and premise relevance over thousands of words while also maintaining narration style and avoiding factual contradictions.These challenges are described as qualitatively new compared with prior work on shorter stories.
  • Framework: Re3 first prompts GPT3 to augment a premise with a setting, characters, and outline, then recursively reprompts it using selectively injected information from the plan and story so far.The Draft prompt dynamically reconstructs context at each generation step.
  • Framework: Re3 revises continuations by reranking alternatives and postediting the selected continuation for factual consistency, following a fully automatic analogue of planning, drafting, rewriting, and editing.The framework is inspired by human writing but operates without human intervention.
  • Results: 14% absolute increase in stories judged coherent in overarching plot and 20% increase in stories judged relevant to the initial premise were reported against similar-length GPT3-based rolling-window baselines.Human evaluators also predicted up to 83% of Re3 stories to be human-written.

2 Related Work

Prior story-generation research has separately explored planning, revision, human interaction, and prompting, usually for shorter outputs. Re3 combines these ideas for fully automatic long-form generation using recursive reprompting rather than one-shot output.

  • Automatic Story Generation: Earlier automatic story-generation approaches commonly modeled individual parts of the writing process rather than the full planning-and-revision workflow.Related methods include outlines, structured schemas, latent variables, slot filling, keywords, control codes, and reranking.
  • Automatic Story Generation: Many prior systems focused on stories of about five sentences, even when using language models with hundreds of billions of parameters.Longer story datasets exist, but research often remained centered on much shorter outputs.
  • Human-In-The-Loop Story Generation: Human-in-the-loop systems commonly combine planning and revision to maintain quality in longer stories, whereas Re3 focuses on fully automatic generation.The paper notes that Re3 remains potentially controllable through human interaction because its modules operate largely in natural language.
  • Prompting: Prompting research has demonstrated strong zero-shot capabilities, but most prompting methods target shorter-answer tasks rather than long-form generation.Re3 instead treats recursive reprompting as a subroutine for producing the final long-form output.

3 Recursive Reprompting and Revision

Re3 decomposes automatic long-story generation into planning, recursive drafting, reranking, and factual editing. Its modules maintain structured context, select coherent and relevant continuations, and correct character-attribute inconsistencies.

  • Framework overview: Re3 decomposes the human writing process into Plan, Draft, Rewrite, and Edit modules for fully automatic story generation.The framework uses no human intervention.
  • 3.1 Plan Module: The Plan module augments a premise with a setting, characters, and numbered outline, whose components are repeatedly reused during passage generation.It prompts GPT3-Instruct-175B to generate these structured planning elements and filters malformed outputs.
  • 3.2 Draft Module: The Draft module generates fixed-length continuations from prompts combining relevant plan context, prior-story summaries, previous-section outlines, and the immediately preceding passage.Relevant context is dynamically updated with named-entity recognition, generated character descriptions, and retrieval of information relevant to the latest passage.
  • 3.3 Rewrite Module: The Rewrite module reranks draft continuations for coherence with the previous passage and relevance to the current outline point, while filtering repetition and narration-inconsistency errors.Its coherence and relevance models are trained using passages and negative examples derived from WritingPrompts, and rule-based heuristics remove detectable writing problems.
  • 3.4 Edit Module: The Edit module detects contradictions in character attributes using compact attribute dictionaries, then makes local corrections to the selected continuation.Dictionaries are updated as new passages and characters appear, and the system extracts attribute-value pairs with a high-precision information-extraction procedure.

4 Evaluation

RE3 is evaluated as a fully automatic long-story generation system against two GPT3-175B rolling-window baselines, using human judgments of quality and writing problems. Its stories are judged significantly more coherent, relevant, and less problematic, while examples and ablations clarify strengths and remaining issues.

  • Task and setup: RE3 generates roughly 2000–2500-word stories from initial premises, using three outline points and four 256-token continuations per point.The implementation targets roughly 3000 tokens while allowing shorter or longer stories in principle.
  • Task and setup: Human workers compare RE3 with ROLLING and ROLLING-FT on interest, coherence, relevance, humanlikeness, and writing issues.For pairwise metrics, workers choose which story is better, both, or neither; each pair is judged by three workers.
  • Main results: RE3 significantly outperforms both rolling-window baselines in coherence and premise relevance, while receiving fewer miscellaneous writing-problem judgments.The comparisons use stories of similar length and report each baseline comparison separately.
  • Main results: 83.3% and 80.0% of RE3 stories were judged human-written in the two baseline comparisons.These percentages are reported as an absolute measure of perceived humanlikeness.
  • Qualitative findings: RE3 can recover the original high-level plot after early generation errors, unlike rolling-window methods.The planning infrastructure provides a mechanism for returning to the intended plot trajectory.

5 Analysis

The analysis examines how Re3’s modules contribute to coherence, relevance, and factual consistency. Plan and Rewrite are critical for plot-level metrics, while Edit provides limited improvement and leaves broader continuity problems unresolved.

  • Ablated Modules: DRAFT-REWRITE-EDIT removes recursive reprompting and generates text like the ROLLING baseline before applying Rewrite and Edit.
  • Ablated Modules: Plan and Rewrite are critical for overall plot coherence and premise relevance, whereas Edit contributes little to these metrics.The ablations remove each module in turn, except Draft, whose removal would make the system inoperable.
  • Further Analysis of Edit Module: Generated stories still contain continuity issues involving settings, changing facts, and omitted outline plot points that an attentive human editor could fix.
  • Further Analysis of Edit Module: STRUCTURED-DETECT is evaluated against entailment-based baselines for identifying character-based factual inconsistencies.The evaluation frames consistency detection as classification over consistent and contradictory setup-story pairs.
  • Further Analysis of Edit Module: The naive ENTAILMENT baseline achieves a ROC-AUC barely above chance, underscoring the need for highly precise inconsistency detection.STRUCTURED-DETECT is designed to scale to longer passages, where the authors hypothesize its advantage would widen.
  • Further Analysis of Edit Module: Current Edit revisions do not effectively improve factual consistency across multiple-thousand-word stories because detection and correction errors compound.The module can fix isolated details but may struggle with larger changes and sometimes introduces undesired edits or additions.

6 Discussion

The discussion presents Re3 as an initial approach to generating over-2000-word stories with stronger long-range coherence and premise relevance. It also identifies evaluation, factual continuity, and scaling story length as continuing challenges.

  • Discussion: Re3 generates stories over 2000 words long while showing strong performance on long-range coherence and premise relevance.
  • Discussion: Re3 emulates human long-form writing with general-purpose language models and dynamically reinjects contextual information into prompts.Because its modules operate largely in natural language, the framework is described as potentially adaptable and controllable.
  • Discussion: The current system remains an initial attempt, with the authors’ main goal being further improvement of automatic long-form story generation.
  • Discussion: Although Re3 stories are an order of magnitude longer than prior work, they remain short stories by human standards rather than novellas or novels.The authors suggest hierarchical outline generation as one possible route to greater length, including an Appendix M example reaching 7500 words.
  • Discussion: The greatest barrier to increasing story length is evaluation, which is currently noisy and costly because comparisons rely solely on human annotation.The authors call for metrics capturing topic drift, contextual nonsensicality, and long-range factual contradictions.
  • Discussion: Despite plot coherence and premise faithfulness, Re3 remains below beginner human writers on long-range factual continuity and other continuity dimensions.The current Edit module is described as not human-level, and human stories maintain continuity beyond character facts.

Limitations

Re3’s evaluation is constrained by the difficulty and cost of assessing long-form generation. Its editing, domain portability, language coverage, and detailed design also remain limited.

  • Evaluation scope: Limited evaluation sample sizes and ablations constrain the experiments and prevent formal evaluation of stories longer than 2000-2500 words.The authors attribute these constraints primarily to evaluation difficulty, not story-generation API costs.
  • System development: Many prompts, reranking heuristics, and Edit-module thresholds were selected manually rather than through careful validation.The authors therefore expect substantial room for improvement in individual module designs.
  • Portability: Adapting the custom-designed story-generation modules to other domains may require manually redesigning prompts and tuning parameters.The structured attribute-value dictionary for characters is especially tailored to stories.
  • Editing limitations: The current Edit module does not improve the main metrics and handles only contradictions involving character attributes.Even in a controlled setting where it outperforms baselines, its absolute ROC-AUC remains low.
  • Language coverage: Re3’s performance may decrease in languages without strong general-purpose language models such as GPT3.

Ethics Statement

The ethics discussion focuses on misuse and bias risks from strong pretrained language models. Re3’s factual-consistency editing is presented as a possible safeguard, while framework-level debiasing remains possible.

  • Misuse: Strong natural language generation systems can be abused for applications such as fake news generation.The authors mitigate this concern in part by focusing on story generation.
  • Bias: Because Re3 relies heavily on GPT3, it may inherit and amplify pretrained-model biases in stories, potentially affecting readers negatively.The framework is not tied to GPT3 and could incorporate debiasing or controlled-generation methods.
  • Language coverage: Re3’s performance is tied to the quality of its base language model and may suffer on non-English languages.

A Character Name Generation

The Plan module generates character names with GPT3-Instruct-175B using the premise, setting, and prior character descriptions, then applies heuristic filtering. The approach is sufficient for this work but leaves quality and fairness concerns.

  • Prompting: The name-generation prompt includes the premise, setting, and previous character descriptions, enabling names from the premise to be copied.
  • Prompting: Generated names are followed by descriptions, and each name-description pair is appended before generating the next character.
  • Filtering: The system samples 10 names and filters problematic role or attribute strings, punctuation, and repeated out-of-premise strings.
  • Limitations: The heuristics still leave room to improve name quality and fairness, including through less biased generation systems.
  • Filtering: Additional rewrite heuristics remove empty outputs and reduce repetition by checking repeated five-word sequences and similar sentences.
  • Filtering: Further filters target jarring narration changes, analysis-like headers, and first- or second-person continuations.

C Details on Editing System Information Extraction

The Edit module extracts structured character information from generated passages, validates and reconciles attributes, and completes relations between characters. Its controlled evaluation converts contradiction decisions into probabilities for ROC-AUC scoring.

  • Information extraction: The Edit detector extracts structured information about a character from each newly generated passage.
  • Running example: The running example uses a dark-cabin premise, setting, characters, and a generated plan to illustrate attribute-value detection.
  • Fact extraction: The system generates three candidate fact lists and retains facts agreed upon by at least two outputs according to an entailment model.This agreement filter is intended to reduce hallucination.
  • Attribute extraction: It extracts attribute keys from facts with few-shot GPT3 prompting and discards generated values because they frequently hallucinate.
  • Attribute extraction: Low-confidence or unanswered attribute keys are filtered with a T5-large UnifiedQA model before values are regenerated from the original facts.
  • Conflict resolution: When an attribute conflicts with an existing value, an entailment model compares converted attribute sentences to reconcile the old and new values.
  • Relation completion: The system can infer reciprocal relations between characters, but its one-value-per-attribute assumption may incorrectly flag multiple values as contradictions.
  • Evaluation: For controlled evaluation, the detector returns the maximum contradiction probability across attribute conflicts to compute ROC-AUC.

D Data on API Usage

RE3 story generation uses substantial API resources, with filtering and reranking driving many tokens and each story costing a few dollars.

  • Usage measurement: The reported API-usage analysis averages calls and processed tokens for each GPT3 endpoint across five RE3 runs.Counts include both prompts and generations.
  • Usage drivers: Filtering and reranking in the Plan and Rewrite modules typically generate 10 outputs per call, increasing token usage.The Edit module accounts for most GPT3-Instruct-13B usage and some GPT3-Instruct-175B usage.
  • Cost: A single RE3 story costs a few dollars under the reported settings.Baselines and ablations require fewer API calls than RE3.

E Dataset Usage

The work relies primarily on the WritingPrompts dataset for reranker and baseline training, while reporting API-usage information in a dedicated table.

  • Dataset usage: WritingPrompts is the only preexisting story dataset used in the work.It trains the relevance and coherence rerankers and the ROLLING-FT generator.
  • Generated examples: The authors also generated contradictory story setups and beginnings through GPT3 prompting for Edit-module analysis.These examples did not rely on a preexisting dataset.
  • Reporting: Table 12 reports average API calls and tokens processed per RE3 story for each endpoint.The table includes prompts and outputs, with Edit API tokens counted together.

F Length vs. Story Quality Analysis

The length analysis compares shorter and longer RE3 variants, but its 50-story sample does not support clear quantitative conclusions about quality.

  • Experimental setup: RE3-SHORT and RE3-LONG vary the number of 256-token passages generated per outline section from two to six.The main configuration uses three outline sections and four passages per section.
  • Findings: A sample of 50 stories was insufficient to draw clear quantitative conclusions about how length affects RE3 story quality.The comparison used RE3-SHORT, RE3, and RE3-LONG.
  • Qualitative observations: Annotators judged longer stories more interesting, while longer outputs may accumulate writing problems and shorter outputs may seem truncated.The generator may also become repetitive or lose the plot thread over longer horizons.

G Full Metrics for Miscellaneous Writing Problems

Individual writing-problem metrics generally show few significant differences, whereas aggregate miscellaneous-problem measures can reveal significant differences between systems and ablations.

  • Individual metrics: Differences in individual writing problems are largely not significant, although RE3 is never significantly worse in the reported comparisons.Aggregate differences can become significant when individual problems are combined.
  • Baseline comparison: Table 14 compares fractions of stories with individual writing problems between RE3 and the ROLLING baselines.Significance is marked at p < 0.05, with aggregate Misc. Problems differences becoming significant.
  • Ablation comparison: Table 15 compares individual writing-problem fractions between RE3 and ablations removing the Plan, Rewrite, or Edit modules.The caption states that individual differences are largely not significant.

H Mechanical Turk Evaluation Details

The evaluation used Mechanical Turk pairwise comparisons of lengthy stories, with annotators skimming passages and judging subjective qualities. Agreement was generally low, reflecting noisy individual labels and the cost of expert annotation.

  • Annotators compared two fairly lengthy stories in Mechanical Turk surveys to evaluate generated-story qualities.Workers were paid $1.50 per hit.
  • The evaluation compared RE3 with baselines and ablations using pairwise metric annotations.Fleiss’ kappa was reported for both comparison types.
  • Annotators were instructed to quickly read or skim lengthy passages, so many details could be missed.The authors expected expert annotators to achieve better agreement, but at substantially higher cost.
  • Agreement was usually positive but low across most comparisons.The paper attributes this partly to subjective metrics and noisy individual labels.

J Example Stories

The example stories illustrate that RE3 generally preserves an overarching, premise-relevant plot, while baseline stories can become disconnected or irrelevant. RE3 examples still contain confusing passages and premature or imperfect resolutions, partly reflecting evaluation constraints.

  • Example Stories: The examples are drawn from the first five evaluation premises and are presented as i.i.d., non-cherry-picked samples.Because many stories can fit the same premise, regeneration could yield different strengths and problems.
  • Example Stories: The authors provide qualitative strengths and weaknesses in example captions to facilitate faster reading.These captions supplement the general qualitative trends described in the text.
  • Example Stories: RE3 stories generally maintain a coherent overarching plot and relevance to the initial premise at the same length.Some confusing passages remain, and stories may deviate from the detailed generated outline.
  • Example Stories: ROLLING and ROLLING-FT commonly produce coherent short passages that become largely irrelevant to the premise and to one another.This contrasts with RE3’s broader plot coherence in the examples.
  • Example Stories: Some RE3 problems may result from concessions made because of evaluation costs and limitations.A 7500-word example without these limitations is offered as a possible indication of the framework’s ceiling.

L Example Data for Editing System Evaluation

The editing-system examples contrast an original setup with an altered setup and show the corresponding generated stories. Changing Beth’s relationship to Julie changes how she is introduced and how the interaction is framed.

  • Example Data for Editing System Evaluation: The example compares an initial setup s and altered setup s′ with their corresponding stories t and t′.The altered setup resamples Beth Christensen’s description.
  • Example Data for Editing System Evaluation: In the initial setup and story, Beth Christensen is Julie’s mother.The corresponding story introduces Beth in that parental role.
  • Example Data for Editing System Evaluation: In the altered setup and corresponding story, Beth Christensen is Julie’s friend instead of her mother.The relationship is changed by resampling Beth’s description.
  • Example Data for Editing System Evaluation: The initial story depicts Beth confronting Julie and Tommy after learning that Julie is pregnant.Beth reacts with shock, embraces Julie, and scolds Tommy.

M Longer Story Example

Re3’s longer-generation example removes fixed component-length limits and uses a two-level hierarchical outline to organize story passages. A single 7,500-word story followed its premise and outline closely while maintaining a coherent overarching plot, despite some inconsistencies.

  • Longer generation: Re3 removes fixed length limits on outlines and story expansions, allowing generation to proceed through outline points based on reranking quality.The system advances to the next high-level point when the best continuation falls below a sufficient log-probability threshold.
  • Hierarchical outline: A two-level hierarchy expands each high-level outline point into minor plot points, with one story passage generated for each lower-level point.Completed lower-level groups are collapsed into their corresponding high-level point in the Draft prompt.
  • Scalability: Recursive hierarchy could enable exponentially longer stories without substantially increasing prompt length; the example reached roughly 7500 words with a 1024-token generator context.This extends the framework beyond the fixed lengths used in the main experiments.
  • Example quality: The single non-cherry-picked story followed its premise and outline closely and maintained a highly coherent plot with a clear beginning, middle, and end.The authors still observed a couple of odd details or inconsistencies, which did not significantly reduce overall understandability.
  • Example setup: The example’s premise described a typical but not completely mundane apocalyptic scenario, and its initial setup and outline were judged sensible for that premise.The outline’s lettered sub-points were nevertheless somewhat repetitive.
Loading 2210.06774v3…