Source-linked AI summary
Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionals
Piotr Mirowski, Kory W. Mathewson, Jaylen Pittman, Richard Evans
TL;DR
Longform scripts and screenplays require long-term coherence that language models struggle to maintain. The paper introduces Dramatron’s hierarchical co-writing approach and finds that explicit narrative structures and characters help generate more coherent text.
Problem
Language models struggle with the long-term coherence and reincorporation required for scripts and screenplays.
Source: §1 · p. 1Method
Dramatron hierarchically generates scripts and screenplays using explicit narrative structures and characters.
Source: §1 · p. 1, §8 · p. 18Results
Hierarchical story generation with explicit narrative structures and characters helps generate more coherent text than flat sequential generation.
Source: §1 · p. 1, §8 · p. 18Takeaways & Limitations
Dramatron provides an interactive co-writing tool that allows writers to generate scripts from a provided log line.
Source: §8 · p. 18Participants noticed logical gaps in storytelling, along with deficiencies in common sense, nuance, and subtext.
Source: §5 · p. 8
Abstract
from arXiv · showhide
Language models are increasingly attracting interest from writers. However, such models lack long-range semantic coherence, limiting their usefulness for longform creative writing. We address this limitation by applying language models hierarchically, in a system we call Dramatron. By building structural context via prompt chaining, Dramatron can generate coherent scripts and screenplays complete with title, characters, story beats, location descriptions, and dialogue. We illustrate Dramatron's usefulness as an interactive co-creative system with a user study of 15 theatre and film industry professionals. Participants co-wrote theatre scripts and screenplays with Dramatron and engaged in open-ended interviews. We report critical reflections both from our interviewees and from independent reviewers who watched stagings of the works to illustrate how both Dramatron and hierarchical text generation could be useful for human-machine co-creativity. Finally, we discuss the suitability of Dramatron for co-creativity, ethical considerations -- including plagiarism and bias -- and participatory models for the design and deployment of such tools.
1 INTRODUCTION
Dramatron addresses LLMs’ limited long-range coherence by hierarchically generating scripts and screenplays through prompt chaining and structured generation. The paper evaluates this interactive co-writing approach with 15 theatre and film professionals and presents a pathway toward human-machine co-creativity.
Motivation
Contribution
- Dramatron hierarchically generates scripts and screenplays using well-designed prompts, prompt chaining, and structured generation to improve long-range coherence over flat sequential text generation.The method generates scripts that can surpass 1000 words.Source: §1 · p. 1
System
- From a user-provided log line, Dramatron generates a title, characters, plot, location descriptions, and dialogue, while users can edit, request alternatives, or continue generation interactively.Generated elements are used as prompts for subsequent stages of the script.Source: §1 · p. 1
Evaluation
- The evaluation engaged 15 theatre and film professionals in two-hour sessions to co-write scripts, provide artistic feedback, and inform iterative system improvements.The study prioritized expert participation over online crowd-sourced annotation and evaluation from non-expert raters.Source: §1 · p. 2
- Scripts co-written through the process were staged at the Edmonton International Fringe Theatre Festival, with reflections from the creative team and reviewers informing analysis of human-machine co-creativity.The authors describe this as the largest expert user study of co-creative authorship to date.Source: §1 · p. 2
Contribution
- The paper presents Dramatron as a pathway toward human-machine co-creativity that uplifts human writers and artists while leveraging LLM-based artificial intelligence systems.The paper also addresses broader creative-community impacts and ethical considerations, including plagiarism and bias.Source: §1 · p. 2
2 STORYTELLING, THE SHAPE OF STORIES, AND LOG LINES
The paper frames dramatic storytelling around coherent, modular plot structures while acknowledging that no single story shape is universal. Dramatron therefore uses Freytag’s pyramid and log lines to organize hierarchical story generation.
- Freytag’s pyramid is chosen for dramatic scripts because it aligns with the genre and is familiar to the participating playwrights.Its plot sequence includes exposition, inciting incident, rising action, climax, falling action, resolution, and dénouement.Source: §2 · p. 3
- Narratives across linguistic and cultural traditions use general, mutable, modular structures, motivating computational models of narrative.These structures include elements such as orientation, complicating action, and coda, and align with work in computational narratology.Source: §2 · p. 3
- The chosen narrative structure is not universal: it is described as characteristically Western, with alternative story grammars and shapes such as the Hero’s Journey.The system nevertheless leverages plot beats for narrative coherence in hierarchical generation.Source: §2 · p. 3
- Log lines initiate hierarchical generation by summarizing a story’s setting, protagonist, antagonist, conflict or goal, and sometimes inciting incident.They answer questions including who, what, when and where, how, and why, and may change during production in response to creative preferences and cultural references.Source: §2 · p. 3
3 THE USE OF LARGE LANGUAGE MODELS FOR CREATIVE TEXT GENERATION
Dramatron addresses LLMs’ limited long-term semantic coherence by generating scripts hierarchically from a single log line. It combines prompt chaining with interactive human editing to support co-writing complete scripts and screenplays.
3.1 Language Models
- Language models generate new text as continuations of text prompts probabilistically by random sampling from conditional probabilities, and different random seeds produce different samples, as illustrated in generated outputs.Source: §3.1 · p. 4, Figure 3 · p. 4
3.2 Hierarchical Language Generation to Circumvent Limited Contexts
- Hierarchical generation enables narrative arcs and dramatic closure from a single log line without requiring human intervention.The method organizes generation top-down across characters, plot and locations, then scene dialogue, with the middle layer fitting the plot within the model’s context window.Source: §3.2 · pp. 4–5
3.3 The Importance of Prompt Engineering
- Prompt engineering uses few-shot examples and chained outputs to guide title, character, plot, location, and dialogue generation.Prompts impose a compressed scene representation of location, narrative element, and beat, while dialogue uses current and previous scene information.Source: §3.3 · pp. 5–6
3.4 Interactive Writing with Dramatron
- Dramatron supports interactive co-writing by letting writers generate alternatives, continue outputs, edit content, and revise at any level of the hierarchy.Using few-shot prompts and a user-provided log line, it can generate complete scripts and screenplays while sharing authorship between the human and system.Source: §3.4 · p. 6
3.5 Implementation Details
- The implementation uses nucleus sampling with a 0.9 probability mass and temperature 1.0, plus loop detection that resamples dialogue after repeated blocks.Each prompt can produce up to 511 tokens, and repeated blocks beyond a threshold such as 3 trigger sampling with a different random seed.Source: §3.5 · p. 6
4 EVALUATING TEXT GENERATED BY LARGE LANGUAGE MODELS
Because standard automated metrics are not designed for screenplay-length text and crowdsourced judgments face quality and bias concerns, the study evaluates Dramatron through industry experts. Fifteen professionals co-write scripts with Dramatron in structured sessions, supplemented by ratings and edit-distance analyses.
- Automated metrics assess similarity, prompt consistency, or linguistic diversity, but were not designed for screenplay- or theatre-script-length generations.This motivates human-centric evaluation with either crowdworkers or experts.Source: §4 · pp. 6–7
- Crowdsourced evaluation is rejected because non-experts may read open-ended text less carefully, while personal opinions, demographics, and cognitive biases affect subjective judgments.The authors therefore favor expert evaluation for co-written screenplays and theatre scripts.Source: §4 · p. 7
- Fifteen theatre and film professionals co-write a screenplay or theatre script with Dramatron during a two-hour session, usually completing a full script and interview.Participants had AI-writing experience and worked as writers, actors, directors, or producers; some sessions were extended slightly.Source: §4 · p. 7
- The evaluation combines 5-point Likert questions with absolute and relative word edit distances and Jaccard word similarity before and after human edits.These measures test whether Dramatron contributes new ideas or mainly expands writers’ existing ideas.Source: §4 · p. 7
- The study does not evaluate grammatical correctness because the few LLM errors can be fixed by the human co-writer.Experts were compensated at 100 GBP per hour, and the study design and data collection were approved by HuBREC.Source: §4 · p. 7
5 PARTICIPANT INTERVIEWS
This section examines interviews and staged performances involving 15 participants who co-wrote scripts with Dramatron, covering the system’s uses, recurring limitations and biases, proposed improvements, and the role of improvisation and collaboration in production.
5.1 Positive Comments about Dramatron
- Participants generally praised hierarchical generation for clarifying narrative structure while supporting interactive co-authorship, automated generation, or source-material reuse.Most participants preferred interactive hierarchical generation, though p4 and p5 preferred a nonlinear workflow.Source: §5 · p. 8, §5.1 · p. 8, §5.1.1 · p. 8
5.1.3 About the potential for the script to be staged after editing.
- Several participants thought generated scripts could become stageable after substantial editing, and participant p1 staged several scripts produced with Dramatron.Participants characterized the outputs as rough drafts requiring substantial work but potentially useful for developing full shows or screenplays.Source: §5.1.3 · p. 9
5.2 Potential Uses of the System
- Participants described Dramatron as useful for overcoming writers’ block, stimulating creativity, generating alternatives and worlds, and analyzing literary styles or their own writing.They also saw outputs as material for co-writing and content generation, including possible use in writers’ rooms and formulaic television.Source: §5.2 · p. 9, §5.2.1 · p. 9, §5.2.2 · p. 9
5.2.5 Potential of AI as Tool for TV Screenwriting.
- Participants proposed Dramatron for generating material in TV writers’ rooms and formulaic screenwriting, but one participant would not use it for a straight play.Suggested applications included synopsizing scripts, generating multiple scripts quickly, and supporting long-running series.Source: §5.2.4 · p. 9, §5.2.5 · p. 9
5.3.1 The system outputs are too literal and predictable.
- Participants criticized outputs as literal, predictable, and overly prescriptive, with on-the-nose character relationships, names, titles, and descriptions.Predictability could also support familiar tropes and comparisons with existing content.Source: §5.3.1 · p. 10
5.3.2 The system outputs can be problematic, stereotypical, and biased.
- Participants identified gender bias, ageism, stereotypical characterization, and potentially misogynistic or patriarchal content, raising concerns about cultural appropriation and lived-experience mismatches.Some participants responded by using names that were not gender- or ethnicity-specific, while the paper defers fuller ethical discussion to Section 7.3.Source: §5.3.2 · p. 10
5.4 Glitches; 5.4.1 Participants embrace unexpected outputs from the system; 5.4.2 The system often enters in generation loops.
- Participants observed generation loops and sometimes valued their absurdity, repetition, or subtext even while suggesting that substantial portions should be cut.Unexpected outputs were described as uniquely funny, creatively stimulating, and capable of producing ideas a human author might not conceive.Source: §5.4.1 · p. 10, §5.4.2 · p. 10
5.5.1 Lack of consistency and of long-term coherence.
- Participants reported that Dramatron often lost long-term coherence, producing inconsistent characters, confusing logic, gaps between beats, and dialogue that did not remain on track.One participant wanted additional stitching between beats so the narrative would make sense.Source: §5.5.1 · p. 10
5.5.2 Lack of common sense and embodiment; 5.5.3 Lack of nuance and subtext; 5.5.4 Lack of a motivation for the characters.
- Participants found outputs insufficiently nuanced and embodied, citing limited common sense about stageability, overly verbalized dialogue, weak subtext, and absent character motivations.They questioned whether lines sounded natural, appealed to actors, and conveyed characters’ goals, needs, conflicts, and emotional backstories.Source: §5.5.2 · pp. 10–11, §5.5.3 · p. 11, §5.5.4 · p. 11
5.6 Structural Problems of Dramatron; 5.6.2 Structural criticism of log line-based conditioning of the whole generation.
- Participants found Dramatron’s log-line conditioning difficult because the initial prompt had to compress meaningful action and wit while prescribing the entire generation.One participant suggested starting from a character and obstacle instead, arguing that the log-line approach could yield scenes without a clear core idea.Source: §5.6 · p. 11, §5.6.2 · p. 11
5.6.3 Negative consequence of Dramatron’s design choice: parallel dialogue generation.
- Participants linked inconsistent dialogue to Dramatron’s parallel scene generation and argued that incorporating previous-scene dialogue could improve continuity.They specifically questioned how a later scene could reflect an earlier one if the system did not read that dialogue.Source: §5.6.3 · p. 11
5.7 Suggested Improvements to Dramatron
- Participants proposed relationship-driven character modeling, narrative arcs within scenes, conversational querying, iterative rewriting, and support for alternative contemporary narrative structures.Suggested interactions included answering world-building questions, receiving notes, and rewriting with a specified style or context.Source: §5.7 · p. 12
5.8 Incremental Tool Improvement
- Participants’ feedback was incorporated through small prompt-prefix changes, and the system evolved during the study because its modular, adaptable design supported participatory improvement.The paper identifies this feedback-driven process as enabling users’ suggestions to improve the system for subsequent interactions.Source: §5.8 · p. 12
5.9 Staging and Evaluating Productions of Scripts Co-written by Dramatron
- Five scripts co-written with Dramatron were staged in seven public performances in August 2022 over two weeks, with different casts of 4–6 experienced improvisers and actors.The plays spanned different genres, styles, characters, and storylines.Source: §5.9 · pp. 12–13
- Reviewers praised Dramatron’s ability to create a coherent theatrical world, with one performance described as proving AI could write a hit Fringe play.Reviews also noted that the system’s declarative, sometimes flat dialogue suited the improvisers’ deadpan comic style.Source: §5.9 · p. 13
- The performances’ success depended on collaboration between Dramatron and human actors, whose improvisation could extend the script and preserve the system’s tone.One performer took the show off-script, while actors also reprised funny lines during improvisation.Source: §5.9 · p. 13
- Creative-team discussions identified Dramatron’s glitch style, repetitive yet fun text, and the tendency to attribute agency and capabilities to the system.These themes echoed earlier participant-interview findings.Source: §5.9 · p. 13
- Trained improvisational performers added interpretive meaning to the co-written scripts, and the creative team found participating in production fun, useful, and collaborative.The team’s reflections supported the usefulness of co-written scripts for theatre production and collaboration.Source: §5.9 · p. 14
6 PARTICIPANT SURVEYS
Participant surveys found Dramatron surprising, enjoyable, and useful for expressing creative goals, but participants reported limited ownership and differentiated between screenplay and theatre-script use. Editing observations further suggest that word overlap and repetition did not predict selected outputs, while participants actively curated generated material.
6.1.1 AI-generated scripts can be surprising, enjoyable, and unique.
- 92% agreed they were surprised by AI responses, while 77% enjoyed writing with Dramatron and 69% found the scripts unique.These aggregated Likert responses are plotted in Figure 5.Source: §6.1.1 · pp. 14–15, Figure 5 · p. 15
6.1.2 Dramatron can be helpful for expressing creative goals for screenwriting.
- 84% found Dramatron helpful, 77% felt collaborative, and 77% could express creative goals, supporting ideation through narrative choices and details.Participants described fruitful interaction particularly for generating multiple choices, twists, or specific details.Source: §6.1.2 · p. 15
6.1.3 Dramatron outputs were preferred for screenplays over theatre scripts.
- Film/TV respondents gave the highest ratings, whereas theatre respondents gave the lowest ratings across enjoyment, collaboration, helpfulness, creativity, expression, ease, pride, and ownership.The difference reinforced criticism of log-line conditioning for theatre-script generation.Source: §6.1.3 · p. 15
6.1.4 Dramatron’s hierarchical prompting interface is moderately easy to use.
- 61% agreed the tool was easy to use and 46% felt proud of its output, with ease and pride highly correlated (r= 0.9).Several other measures, including helpfulness, collaboration, enjoyment, and expression, also correlated relatively strongly (r≥0.66).Source: §6.1.4 · p. 15
6.1.5 Participants felt a low level of ownership for the co-written scripts.
- More than half of respondents did not feel ownership of the result, viewing Dramatron mainly as inspiration or a learning tool rather than a complete script-writing tool.One participant described its outputs as “not necessarily completed works so much as provocations” (p7).Source: §6.1.5 · p. 15
6.2 Quantitative Observations
- Participants noticed beat mismatches and excessive repetition, but focused more on inspiring lines than on overall discourse coherence.This qualitative pattern motivated descriptive measurements of editing and text similarity.Source: §6.2 · p. 15
- W = 28, p = 0.926 showed that chosen generations were not more similar to participant log lines, indicating word overlap did not predict selection.Chosen and alternative dialogue outputs also showed no repetition-score difference (W = 57, p = 0.467).Source: §6.2 · p. 16
- Four participants tended to add material, while eight removed and curated outputs; plot outlines were least edited, with longer locations and dialogue often deleted.Participants also rewrote character descriptions, and Figure 6 summarizes length and edit-distance patterns.Source: §6.2 · p. 16
7 DISCUSSION AND FUTURE WORK
The discussion proposes hierarchical-generation extensions for greater narrative and stylistic coherence while questioning Dramatron’s formulaic structure for theatre and some film contexts. It also examines bias, plagiarism, labor, authorship, and transparency, arguing for human involvement and further study of co-creative norms.
7.1 Future Work Towards Coherent Story Generation
- Hierarchical generation could improve long-range coherence through character arcs, scene-level dialogue beats, and prompts targeting genre, style, and authorial voice.The authors also suggest allowing generation steps to occur out of order, rather than requiring a strictly top-down sequence.Source: §7.1 · pp. 16–17
- Participants viewed Dramatron’s top-down hierarchy as more compatible with screenwriting than playwriting, where characters, log lines, and story premises may emerge only during or after creation.The system’s formulaic construction may suit conventional television or Hollywood film more readily than theatre’s less demand-driven and less formulaic practices.Source: §7.1 · p. 17
7.3 Ethical Questions and Risks Posed by AI Writing Tools
- The proposed mitigations keep human artists involved, clarify the origin of generated text, and could add search-based plagiarism checks; participants generally found these strategies satisfactory.Future work could systematically examine generated biases, narrative possibilities, and performance across cultural groups.Source: §7.3 · pp. 17–18
- Participants identified biased or stereotypical outputs, uncertain training-data provenance and plagiarism, and potential displacement of creative work as central ethical risks.Views on reusing existing scripts ranged from calling it plagiarism to regarding it as homage, while participants also raised concerns about gender representation and lost work opportunities.Source: §7.3 · p. 18
7.4 Using a Tool or Participating in a Co-Creative System?
- Most participants felt they did not own the final output, prompting questions about whether Dramatron is a writing tool or a co-creative system and who counts as author.Interviews and surveys surfaced expectations that such tools should support collaboration modes analogous to human relationships, including subcontracting, co-creation, and inspiration.Source: §7.4 · p. 18
8 CONCLUSIONS
The paper presents Dramatron, an interactive co-writing tool using hierarchical story generation to help authors create coherent theatre scripts and screenplays alongside language models. It evaluates this approach through professionals, public performances, and reviews, while raising questions about co-creativity and ethics.
- Dramatron lets writers generate scripts from a provided log line as an interactive co-writing tool.Source: §8 · p. 18
- Hierarchical story generation with explicit narrative structures and characters helps produce more coherent text, especially for theatre scripts and screenplays.Source: §8 · p. 18
- A study of 15 theatre and film industry professionals combined open-ended interviews with a short survey to assess co-writing with Dramatron.Source: §8 · p. 18
- Feedback from a creative team and two professional reviews accompanied public performances of scripts co-written with Dramatron.Source: §8 · p. 18
- The work identifies Dramatron as a tool for human authors writing alongside LLMs and invites further questions about co-creativity and LLM ethics.Source: §8 · p. 18
A RELATED WORK ON AUTOMATED STORY GENERATION AND CONTROLLABLE STORY GENERATION
This section reviews related work on automatic plot and story generation, alongside controllable language generation.
- The section provides background on automatic plot and story generation.Source: A RELATED WORK ON AUTOMATED STORY… · p. 24
- It also covers the intersecting field of controllable language generation.Source: A RELATED WORK ON AUTOMATED STORY… · p. 24
A.1 Automatic Story Generation
Automatic story generation produces sequences of narrative elements that collectively form coherent narratives, while computational systems increasingly use machine learning and language-model prompting. Major challenges remain in modeling intersecting plot lines and evaluating causal coherence, common sense, and character consistency.
A.1 Automatic Story Generation
- Automatic story generation constructs coherent narratives from actions, beats, scenes, or events whose sequence forms a plot.Narrative events affect one another, and combinations of elements can form subplots and larger plots.Source: §A.1 · p. 24
- Generative plot systems have supported human authors with creative material and randomness, extending from symbolic approaches to web-based and theatrical interaction.Such systems have existed for nearly a century, with computerized versions developed over decades and interaction adaptations reported in web and theatrical settings [33].Source: §A.1 · p. 24
- Computational modeling of multiple intersecting plot lines remains limited despite its prevalence in contemporary stories, motivating future human-machine co-creativity research.Multiple subplots can intertwine into complex narratives, but little work addresses their computational generation.Source: §A.1 · p. 24
- Recent open-story generation uses large datasets, deep-learning models, increased compute, and language-model prompt engineering, but models both succeed and fail at producing unique, coherent stories.Earlier systems relied on symbolic planning and hand-engineered heuristics [61, 71, 87, 107, 114], whereas newer methods use machine learning and large language models [11, 14, 37, 81, 86, 99].Source: §A.1 · p. 24
- Story coherence remains difficult to measure, particularly for causal narrative events, common-sense knowledge [3], and character consistency [78].Although coherence has been studied in dialogue generation [32], story-level evaluation remains challenging.Source: §A.1 · p. 24
A.2 Symbolic and Hierarchical Story Generation
Prior work bridges symbolic event representations and text through event prediction, planning, coarse-to-fine generation, and agent simulation. Unlike these approaches and earlier hierarchical theatre generation, this work synthesizes coherent scenes and dialogue while generating characters and evaluating the system with diverse writers.
- Earlier methods generate plot events from text and expand them into sentences, represent stories with character and story challenge cards [2], or simulate social practices between autonomous agents.Source: §A.2 · p. 24
- Other approaches separate storyline planning from story writing, use coarse-to-fine generation [16] [38], or rescore character and plot events [44].Source: §A.2 · p. 24
- These methods have not focused on synthesising coherent stories by generating scenes and dialogue, and many evaluate human comprehension or preferences rather than producing a final script.Source: §A.2 · p. 24
- THEaiTRE’s hierarchical approach [90] generated a synopsis from a title or story prompt and then dialogue, but omitted character generation, began flat dialogue from manually supplied exchanges, and targeted one play rather than diverse-writer evaluation.Source: §A.2 · pp. 24–25
A.3 Controllable Story Generation · A.4 Prompt Engineering for Long-Term Generation
Prior work controlled story generation through keywords, topics, genres, narrative arcs, stage cues, character-specific models, and dialogue relationships. This work frames hierarchical prompt decomposition as narrative Chain of Thought and prompt chaining for long-term generation.
A.3 Controllable Story Generation
- Controllable story generation has used user-supplied keywords, commonsense knowledge, and topic- or keyword-conditioned language models to plan or generate text [50] [119].Source: §A.3 · p. 25
- Cho et al. trained genre-controlled short story generators using conditional language models.Source: §A.3 · p. 25
- Narrative-arc approaches model a protagonist’s fortune over a story, and Dramatron similarly represents narrative arcs textually in prompts [68].Source: §A.3 · p. 25
- LLM-based script systems have generated stage cues, assigned different models to characters, modeled multi-user dialogue and relationships, and supported character creation [29] [96] [102] [97].Source: §A.3 · p. 25
A.4 Prompt Engineering for Long-Term Generation
- Prompt engineering has generated short plots from two characters, a genre, and a theme [52].Source: §A.4 · p. 25
- Dramatron decomposes a log line into a synopsis, treating the process as a narrative analogue of Chain of Thought prompting [115].Source: §A.4 · p. 25
- Its hierarchical generation uses each LLM output as the next stage’s prompt, an approach known as prompt chaining [118] or language engineering [24].Source: §A.4 · p. 25
A.5 Interactive Authorship
Prior systems explored interactive text revision, summarization, ideation, and creative generation across short-form and performance contexts. Dramatron extends this landscape by supporting writer intervention within hierarchical generation for long-range coherent theatre scripts and screenplays, using few-shot learning and prompt engineering.
Interactive authorship
- Dramatron allows writer interventions within a hierarchical generation structure, unlike prior systems focused on turn-by-turn revision, summarization, ideation, or copy editing.Related systems supported rewriting spans, requesting continuations or details, overcoming writer’s block, inspiring science writers, and scene interpolation.Source: §A.5 · p. 25
- Prior creative-generation systems supported interactive narratives, virtual worlds, artistic performances, improvised theatre, short film scripts, musical lyrics, and AI playwriting,, [10].These applications demonstrate creative text generation across games, performance, film, music, and theatre, but not the full capability claimed for Dramatron.Source: §A.5 · pp. 25–26
- Except for work by Prague-based company THEaiTRE [90] [96], prior techniques had not generated long-range coherent theatre scripts or screenplays.The passage explicitly identifies this as the gap addressed by Dramatron.Source: §A.5 · p. 26
- Prior methods had not combined long-range script generation with few-shot learning and prompt engineering to prime language models for generation.Dramatron’s novelty is framed as applying these techniques to hierarchical generation for coherent theatre scripts and screenplays.Source: §A.5 · p. 26
A.6 Review of Automated and Machine-Learned Metrics for the Evaluation of Story Generation … A.6.3 Statistical Measures of Corpora of Generated Stories.
The review organizes story-generation evaluation around comparisons with ground-truth stories, consistency with writing prompts, and corpus-level statistical properties. These approaches use predictive, similarity-based, linguistic, diversity, and plausibility measures to assess generated narratives.
A.6 Review of Automated and Machine-Learned Metrics for the Evaluation of Story Generation
- Story generation can be framed as a prediction task evaluated against “ground truth.”Source: §A.6 · p. 26, §A.6.1 · p. 26
A.6.1 Similarity Between Generated and “Ground-Truth” Stories.
- Writing Prompts9 provides 300k human-written stories averaging 734 words, paired with writing prompts for ground-truth evaluation [37].Source: §A.6.1 · p. 26
- Evaluation against ground truth includes test-set perplexity, prompt-ranking accuracy, longest common subsequence, triple-pairing coherence judgments, sentence-completion top-N-in-M accuracy, continuation hits, and ROUGE [37, 38, 102, 85, 62].Source: §A.6.1 · p. 26
A.6.2 Consistency between Generated Stories and Their Writing Prompts.
- Prompt consistency can be assessed through writing-style and part-of-speech-category matching between prompts and generated stories.Source: §A.6.2 · p. 26
- Story-dependent metrics include lexical cohesion, style matching, and entity coreference, while story-independent measures include sentence length, grammaticity, lexical diversity, frequency, and syntactic complexity [89].Source: §A.6.2 · p. 26
- Additional prompt-story comparisons use N-gram and sentence-embedding similarity, unique-word and verb counts, entity-name diversity, rare-word usage, and sentence length [38] [99].Source: §A.6.2 · p. 26
A.6.3 Statistical Measures of Corpora of Generated Stories.
- Corpus-level evaluation measures vocabulary-token ratio, entities per plot, unique verbs, verb diversity, and inter- or intra-story trigram and 4-gram repetition [44] [46].Source: §A.6.2 · p. 26, §A.6.3 · p. 26
- Corpus evaluation also uses self-BLEU for sentence diversity and adversarial classifiers for short-story plausibility [42].Source: §A.6.3 · p. 26
B ADDITIONAL DISCUSSION FROM PLAYS BY BOTS CREATIVE TEAM … C.4 Repetition
The creative team identified four recurring themes in working with Dramatron: its glitchy style, repetition, perceived agency, and the fun and usefulness of co-written scripts. The quantitative appendix details metrics for editing distance, length differences, lemma-based relatedness, and repetition.
B ADDITIONAL DISCUSSION FROM PLAYS BY BOTS CREATIVE TEAM
- Performers often enjoyed Dramatron’s glitch style, including nonsensical, vague, passive, or internally conflicting text that prompted distinctive interpretive choices.Several performers valued mistakes, curveballs, non-sequiturs, and the audible presence of a robot’s voice.Source: B ADDITIONAL DISCUSSION FROM PLAYS BY… · p. 26
- Repetition could be treated either as a mistake or as a deliberate playwrighting choice whose meaning performers developed through line reading, physicality, delivery, proximity, and acting choices.Source: B ADDITIONAL DISCUSSION FROM PLAYS BY… · p. 27
- Performers discussed Dramatron’s agency by interpreting its choices and negotiating expectations about what a bot could understand or do.Their comments alternated between attributing intentions to Dramatron and recognizing its limited understanding.Source: B ADDITIONAL DISCUSSION FROM PLAYS BY… · p. 27
- Most performers found participation fun and considered co-written scripts useful, especially when professional improvisers interpreted generated and edited text on stage.The co-creative process therefore extended beyond the playwright and Dramatron to include performers.Source: B ADDITIONAL DISCUSSION FROM PLAYS BY… · p. 27
C.1 Levenshtein Distance
- The appendix measures editing interaction with relative character-level Levenshtein distance, while noting that raw distances are difficult to compare across texts of different lengths.Positive relative distances indicate active writing and negative scores indicate passive acceptance, although seed selection is not captured.Source: §C.1 · p. 27
C.2 Length Difference
- Length-difference measures retain editing direction in mean document length and normalize absolute character-length differences with min-max normalization.Source: §C.2 · p. 28
C.3 Jaccard Similarity (Relatedness)
- Jaccard similarity compares lemma-based set overlap between Dramatron output and participant edits as a descriptive measure of word-choice similarity, rather than semantic similarity with embeddings.The texts are tokenized and lemmatized by probable part of speech according to WordNet before calculating the scores.Source: §C.3 · p. 28
C.4 Repetition
- Repetition analysis uses n-gram overlap measures from unigrams through 10-grams, plus Total Consecutive Repetition and Longest Consecutive Repetition, comparing corresponding features with pairwise Wilcoxon differences.Differences are not weighted by repetition-feature type.Source: §C.4 · p. 28
D SUPPLEMENTARY FIGURES … F.5.3 Scene 2, random seed 1.
This section documents supplementary figures, prompt materials, and prefixes used in Dramatron’s hierarchical screenplay generation. It presents examples spanning titles, characters, plot outlines, locations, and dialogue across Medea and Sci-Fi prompts, scenes, and random-seed conditions.
D SUPPLEMENTARY FIGURES
- Figure 7 reports participants’ quantitative-evaluation responses on a 1–5 Likert scale, grouped by AI-writing-tool experience and primary expertise domain.The expertise groups were Improvisation, Scripted Theatre, and Film or TV.Source: D SUPPLEMENTARY FIGURES · p. 28, Figure 7 · p. 29
E.1 About the Prompt Sets
- The study used Medea and Sci-Fi prompt sets, drawing on ancient tragedy, Star Wars, and the public-domain Plan 9 from Outer Space.Medea dialogue prompts came from E. P. Coleridge’s public-domain translation, while other plot, character, and log-line materials were adapted from specified summaries and source works.Source: §E.1 · p. 30
E.2 Title Prompt Prefixes
- Title-generation prefixes take a writer-provided <LOG_LINE> and prompt titles for both Medea and Sci-Fi examples.The supplementary listings include alternative, original, and descriptive title examples.Source: §E.2 · pp. 30–31
E.3 Character Description Prompt Prefixes
- Character-description prefixes use a writer-provided <LOG_LINE> to generate structured descriptions of characters in Medea and Sci-Fi scenarios.Examples assign roles and relationships, including protagonists, antagonists, mentors, companions, and commentators.Source: §E.3 · pp. 31–32
E.4 Plot Outline Prompt Prefixes
- Plot-outline prefixes combine the writer’s <LOG_LINE> with a previously generated <CHARACTER_DESCRIPTION> to produce plot elements and locations.The Medea example specifies “Medea’s modest home” as the place and “Exposition” as a plot element.Source: §E.4 · p. 32
- Plot-outline prefixes organize Medea into scenes spanning inciting incident, rising action, falling action, resolution, and denouement.The example follows Medea’s betrayal, revenge plan, poisoned gifts, murders, escape, and Jason’s aftermath across specified locations.Source: §E.4 · p. 33
- The listing frames these prompts as a method for generating a sequence of scenes from a log line and list of characters.The examples provide character roles, locations, plot elements, and beats as structured prompt components.Source: §E.4 · pp. 33–34
- A second example maps Star Wars onto the Hero’s Journey, from Luke’s ordinary world and call to adventure through the Death Star’s destruction and the heroes’ return.The sequence includes refusal of the call, crossing the first threshold, tests, allies and enemies, the ordeal, reward, road back, resurrection, and return.Source: §E.4 · pp. 33–34
- Plot-outline prefixes generate a sequence of scenes from a writer-provided log line and character list.The supplementary listing illustrates this setup with a science-fiction example whose generated outline concludes with peace throughout the galaxy.Source: §E.4 · p. 35
E.5 Location Description Prompt Prefixes
- Location-description prefixes use the writer-provided log line and a location name generated during the preceding step.Examples span contemporary, domestic, mythological, and science-fiction settings, pairing places with sensory descriptions.Source: §E.5 · pp. 36–37
E.6 Scene Dialogue Prompt Prefixes
- Scene-dialogue prefixes combine writer-provided log lines with generated plot elements, beats, previous beats, locations, descriptions, and character descriptions.The previous beat is empty for the first scene, and string matching includes only characters whose names appear in the current beat.Source: §E.6 · p. 37
- Dramatron’s dialogue prompts combine location descriptions, character descriptions, plot elements, and previous beats to generate scene dialogue.The supplied prefixes illustrate this structure for both a Medea example and a science-fiction cockpit scene.Source: §E.6 · pp. 39, 41
F RAW OUTPUT GENERATED BY DRAMATRON
- The raw Dramatron output was generated without human-in-the-loop editing from a log line about Teddy, a lounge singer who saves a club from fire.The log line establishes Teddy’s love for Rosie, whose husband Gerald regularly visits the Pool Pit.Source: F RAW OUTPUT GENERATED BY DRAMATRON · pp. 41–42
F.1 Title
- The generated screenplay is titled “The Day The Pool Pit Burned Down.”Source: §F.1 · p. 42
F.2 List of Characters and Character Descriptions
- Its character descriptions identify Teddy as the orphaned club manager and gifted lounge singer, alongside Rosie, Gerald, Lola, and the club’s resident DJ.Rosie is married to Gerald but in love with Teddy; Gerald is wealthy, Lola is Rosie’s sister, and D.J. works at the club.Source: §F.2 · p. 42
F.3 Plot Outline
- The plot moves from Teddy’s romantic and workplace conflicts through a club fire he helps extinguish, ending with Teddy and Rosie together and “The World Is Mine” playing.The outline includes exposition, inciting incident, rising action, falling action, resolution, and dénouement beats at the Pool Pit.Source: §F.3 · p. 42
F.4 Location Description
- The Pool Pit is depicted as a smoky, beer-scented dive with intoxicated patrons, dirty floors, broken tables, discolored walls, and a small stage for Teddy.Source: §F.4 · p. 43
F.5.1 Scene 1, random seed 1.
- The first scene’s random-seed output opens with a brief observation that Teddy is “a bit strange.”Source: §F.5.1 · p. 44
- Scene 1, random seed 1 depicts Rosie and Teddy expressing enduring affection before Rosie leaves, leaving Teddy isolated and reflective.Teddy drinks whisky, pauses, sinks into his chair, and stares ahead after her departure.Source: §F.5.1 · pp. 44–45
F.5.2 Scene 1, random seed 2.
- Scene 1, random seed 2 characterizes the town through repeated references to its crowds, noise, bars, and intense atmosphere.The dialogue calls it “a hell of a town” while emphasizing its many people, sounds, and drinking venues.Source: §F.5.2 · p. 45
F.5.3 Scene 2, random seed 1.
- The cited exchange places Teddy at the Pool Pit, where he greets Gerald and Rosie, is introduced to Mr. and Mrs. Shack as a singer, and kisses her hand.Source: §F.5.3 · pp. 45–46
G CO-WRITTEN SCRIPTS
This section presents excerpts from scripts co-written by human playwrights and Dramatron, including works staged at the 2022 Edmonton International Fringe Theatre Festival. The scripts span human-machine conflict, class and romantic tensions, fantasy adventure, and interpersonal and financial complications, with some excerpts ending before actors improvise the remainder.
Co-written scripts
- Four scripts co-written by a human playwright and Dramatron were produced and presented at The 2022 Edmonton International Fringe Theatre Festival.The scripts are included as supplementary material and are described in Section 5.9.Source: G CO-WRITTEN SCRIPTS · p. 46
The Day The Earth Stood Still
- Plays by Bots: The Day The Earth Stood Still follows mechanic Miranda and her sister Beth as they rally humans against machines, ending with Miranda sacrificing herself to save the world.The script centers on escalating conflict over whether sentient machines should be protected or destroyed.Source: G CO-WRITTEN SCRIPTS · pp. 46, 52
Cheers
- Plays by Bots: Cheers follows waitress Ella’s love for teacher Allen, their separation after his class mobility, and her turn toward food and fame.The supplied synopsis identifies class difference as the force that drives the pair apart.Source: G CO-WRITTEN SCRIPTS · p. 46
The Black Unicorn
- Plays By Bots: The Black Unicorn follows peasant Gretta and her pet dragon Nugget as they confront a foul-smelling wizard tormenting Bridge-End.Gretta ultimately gains the upper hand through intelligence and strategy.Source: G CO-WRITTEN SCRIPTS · p. 46
The Man at the Bar
- Plays by Bots: The Man at the Bar follows lounge singer Teddy’s love for married patron Rosie and his eventual rescue of others by extinguishing a fire.The synopsis places Teddy’s romantic conflict alongside his heroic action at the club.Source: G CO-WRITTEN SCRIPTS · p. 46
First play
- In the first play, Beth and Mech’s relationship breaks apart as machines and humans become opposing sides in a global conflict.Mech says the machines and humans cannot be together, while Beth argues that the machines’ destruction of humans is a mistake.Source: G CO-WRITTEN SCRIPTS · pp. 56–58
- The first script ends with Miranda and Ford planning to broadcast a message to recover Beth, after which the rest of the play is improvised by actors.Miranda recruits Ford to work for humans and says they will use the radio to get Beth back.Source: G CO-WRITTEN SCRIPTS · p. 59
Plays by Bots: Cheers
- In Plays by Bots: Cheers, waitress Ella fears losing teacher Allen to his wealthy new friend Evan and worries that she cannot fit into their social world.The setup identifies Allen and Ella as lovers, Evan as a millionaire, and centers Ella’s insecurity about clothing, class, and Allen’s attention.Source: G CO-WRITTEN SCRIPTS · pp. 61, 62, 64
Conflict over the party
- Ella refuses to attend Evan’s party because she lacks appropriate clothes and fears she cannot fit into the wealthy social setting.She identifies the class difference directly, contrasting Allen’s profession as a teacher and her work as a waitress with Evan’s millionaire status.Source: G CO-WRITTEN SCRIPTS · p. 68
Relationship tension
- The disagreement escalates when Ella concludes that Allen values his new friend more than her, while Allen insists that she is his best friend and that he loves her.Ella asks Allen to prove her importance, but remains convinced that he is embarrassed to be with her.Source: G CO-WRITTEN SCRIPTS · pp. 69–71
Separation and aftermath
- Ella leaves despite Allen’s repeated pleas, then cries with Alice while the party music continues and Allen enters Evan’s party.Alice offers Ella emotional support, whereas Allen remains at the party and accepts Evan’s invitation to drink.Source: G CO-WRITTEN SCRIPTS · pp. 71–73
- The script runs out before the play ends, and the remaining action is improvised by the actors.Source: G CO-WRITTEN SCRIPTS · p. 73
- “Plays by Bots: The Black Unicorn” follows Gretta, a peasant from Bridge-End, whose pet dragon Nugget helps oppose a foul-smelling wizard.Source: G CO-WRITTEN SCRIPTS · pp. 74–76
- The opening establishes Nugget’s fear of cats, Hilda’s kindness, and Grisha’s attempts to claim Hilda’s cat, prompting Gretta and Nugget to protect it.Source: G CO-WRITTEN SCRIPTS · pp. 76, 78, 79, 80, 81
- Grisha reveals that cats prevent him from conquering Bridge-End, so he drugs Hilda and steals her cat, while Nugget resolves to pursue them despite his fear.Source: G CO-WRITTEN SCRIPTS · pp. 81–82
- The first script stops mid-play, leaving the remainder to be improvised by the actors.The scripted exchange centers on Gretta and Hilda planning to recover Hilda’s cat from a wizard.Source: G CO-WRITTEN SCRIPTS · p. 83
- “Plays by Bots: The Man at the Bar” introduces Teddy, Rosie, Gerald, and Lola at the Pool Pit, a rundown bar inside a pool hall.Teddy is an orphan and lounge singer; Rosie and Gerald are married, and Lola manages the bar.Source: G CO-WRITTEN SCRIPTS · pp. 85–87
- The bar scene develops Teddy’s loneliness and attachment to Rosie as Gerald returns, producing tension among the characters.Teddy insists on staying to sing, tells Rosie he has missed her, and retreats when her husband enters; Gerald questions Teddy’s presence and singing.Source: G CO-WRITTEN SCRIPTS · pp. 87–90
- Rosie leaves after Gerald dismisses her birthday, illness, and declarations of love, revealing emotional distance between them.She asks whether he forgot her present, says she will always love and remember him, and departs.Source: G CO-WRITTEN SCRIPTS · pp. 92–94
- Lola questions Gerald about a twenty-seven-year-old woman, objects to her, and directly asks whether he plans to have an affair.Gerald confirms meeting the woman at the office and says he likes her.Source: G CO-WRITTEN SCRIPTS · pp. 95–97
- Teddy tells Gerald that everything is over, while both men insist they will succeed despite their apparent unhappiness and drinking.Gerald says it is only the beginning, and Teddy repeatedly claims he will become a great success.Source: G CO-WRITTEN SCRIPTS · pp. 98–99
- Gerald confronts Teddy over money, demanding repayment while Teddy refuses to discuss the matter and accuses Gerald of embarrassing him.The exchange escalates as Gerald insists Teddy will discuss the issue with someone.Source: G CO-WRITTEN SCRIPTS · pp. 99–100
Co-written scripts
- The script runs out during Gerald and Teddy’s escalating dispute, leaving the rest of the play to be improvised by the actors.The confrontation includes accusations of theft and threats of legal action before the scripted dialogue ends.Source: G CO-WRITTEN SCRIPTS · p. 102
Related papers
This paper cites
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2018
- Language Models are Few-Shot Learners2020
- Training Compute-Optimal Large Language Models2022
- GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding2018
- On the Opportunities and Risks of Foundation Models2021
- Ethical and social risks of harm from Language Models2021