DramaAgent: Agentic Storytelling Video Generation

1 Introduction

• Long-form generation remains difficult because narrative drift, identity instability, weak temporal continuity, and limited audio-visual consistency accumulate across scenes.One-shot prompting and clip stitching can also lose semantic accuracy, character consistency, scene transitions, and story state.Source: §1 · pp. 1–2
• The framework treats long-form storytelling as structured control over heterogeneous generators, rather than independent short-clip synthesis.It combines story planning, persistent character conditioning, scene-wise synthesis, and reflection-guided targeted repair.Source: §1 · p. 2
• Story planning defines ordered scene targets with characters, environments, events, emotional states, and transition cues; persistent references carry identity across generation and repair.Reflection scores clips on identity, semantics, temporal continuity, and audio-visual alignment, then regenerates low-quality scenes using failure-specific constraints.Source: §1 · p. 2
• DramaAgent improves long-horizon coherence, character consistency, narrative fidelity, and scene-level audio-visual consistency over direct generation and strong baselines across multiple video backbones.The evaluation includes automatic and human-centered protocols.Source: §1 · p. 2

Long-form and agentic story video generation

• Long-form video research centers narrative coherence, character consistency, and temporal continuity; agentic systems decompose stories into scripts, storyboards, or scene prompts to improve controllability.DramaAgent focuses on text-to-video-and-audio control over heterogeneous generators, with reusable story and character states and failure-specific repair.Source: §2 · pp. 2–3

Figure 2

• Figure 2 depicts prompt decomposition into scene-level targets, persistent character stills, and coordinated image, video, and audio generation with reflection-guided repair.Compare the scene-wise synthesis stages with the repair stage that follows them.Source: Figure 2 · p. 3

Character-consistent video generation

• Character-consistent methods use references, personalized prompts, keyframes, or visual anchors; DramaAgent propagates fixed character references across generation, selection, and repair.Magic-Me (Ma et al., 2024) shows that external visual references can improve subject consistency in generated videos.Source: §2 · p. 3

General video generation backbones

• Diffusion, autoregressive, and hybrid backbones strengthen visual realism, motion, text alignment, and temporal modeling, but long-form stories still require cross-scene state and audio-visual coordination.DramaAgent is presented as complementary: it coordinates heterogeneous video and audio generators toward a long-form storytelling objective.Source: §2 · p. 3

3.1 Task Formulation

• For each scene, DramaAgent generates candidates, evaluates them against scene requirements and accepted context, and selectively repairs low-quality outputs before composing accepted audiovisual segments.The task is to produce a long-form video and audio that preserve story coherence, character consistency, cross-scene continuity, and scene-level audio-visual consistency.Source: §3.1 · p. 4

3.2 Story Analysis and Plot Breakdown

• An instruction-tuned language model expands the prompt into a story draft, then decomposes it into temporally ordered scenes corresponding to events, transitions, or emotional beats.Each scene target includes a description, participating characters, environment, narrative role or emotional state, and optional transition cue from the preceding scene.Source: §3.2 · p. 4

3.3 Persistent Character Conditioning

• DramaAgent creates canonical character references from identity-defining attributes and reuses them across scene generation, candidate selection, and repair.When image conditioning is unavailable, the references are converted into textual identity constraints.Source: §3.3 · p. 4

3.4 Reflection-guided Candidate Selection and Repair

• Candidates are scored for identity, semantic fidelity, temporal continuity, and audio-visual compatibility; the weakest dimension determines the targeted prompt repair.Identity failures strengthen character-reference constraints; semantic failures add missing content; temporal failures add transition cues; and audio-visual failures adjust dialogue timing, sound-event constraints, or visual pacing. Refinement is limited to two rounds by default, stopping earlier if score improvement falls below 0.02.Source: §3.4 · pp. 4–5

3.5 Audio Generation and Composition

• Scene-conditioned dialogue, character-specific speech, ambient sound, and background music are assembled with accepted clips, while transitions are smoothed and dialogue timing is aligned as closely as possible to clip duration.Fine-grained lip synchronization and precise sound-effect timing still depend on the underlying video and audio generators.Source: §3.5 · p. 5

4 Experiments

• The evaluation covers overall generation quality, backbone robustness, control-component contributions, and human-perceived long-form coherence.It focuses on preserving character identity, narrative progression, cross-scene continuity, and scene-level audiovisual consistency.Source: §4 · p. 5

4.1 Experimental Setup

• The evaluation uses 50 narrative scripts spanning genres, with each script comprising 10–20 sentences and yielding approximately 30–60 seconds of video after composition.Generation uses heterogeneous backbones, 3 candidates per scene, 5-second clips at 1280×720, and at most 2 reflection-guided repair rounds.Source: §4.1 · pp. 5–6

4.2 Main Results

• For full-movie generation, DramaAgent reaches CLIP 24.34 and Inception 9.67 and leads compared methods on Subject Consistency, Background Consistency, Motion Smoothness, and Aesthetic quality.In storyboard generation, it obtains CLIP 23.02 and Inception 7.64, outperforming StoryDiffusion, AutoStory, and MovieAgent.Source: Table 1 · p. 6, §4.2 · p. 6

4.6 Qualitative Analysis

• Qualitative examples show recurring characters and coherent scene progression across three narrative scenes and different story contexts.Backbones show complementary strengths, with some stronger in motion continuity and others in character appearance or visual style.Source: Figure 3 · p. 7, §4.6 · p. 8, Figure 4 · p. 8

4.3 Controlled Analysis

• Fixed-backbone ablations attribute progressive gains to story planning, character stills, and repair, while performance varies by generator under the shared framework.The analysis also compares controlled variants with and without reflection to separate targeted repair from generic additional sampling.Source: Table 2 · p. 6, §4.3 · p. 7, §4.4 · p. 7

4.4 Ablation Study

• Reflection raises storyboard CLIP from 22.05 to 23.02 and full-movie CLIP/Inception from 24.12/9.41 to 24.34/9.67, with gains across VBench metrics.On a fixed backbone, story planning improves semantic alignment, character stills strengthen identity preservation, and reflective repair adds further gains.Source: Table 4 · p. 7, §4.4 · p. 7

4.5 Audio-Visual and Human Evaluation

• Human evaluation prefers DramaAgent over baseline pipelines on temporal coherence, character consistency, narrative fidelity, and scene-level audiovisual consistency.The evaluation targets scene-level consistency rather than perfect frame-level lip synchronization, which remains partly constrained by the underlying generators.Source: §4.5 · pp. 7–8

• DramaAgent combines story planning, persistent character conditioning, scene-wise generation, reflection-guided repair, and audio composition as structured control over heterogeneous generators.Source: §5 · p. 8
• Across multiple backbones, DramaAgent improves long-horizon coherence, character consistency, narrative fidelity, and scene-level audiovisual consistency over direct generation and strong baselines.Source: §5 · p. 8
• The results suggest hierarchical agentic control is a practical direction for controllable long-form audiovisual storytelling.Source: §5 · p. 8

Limitations

• Third-party API changes can alter outputs, latency, pricing, and conditioning interfaces, limiting the stability of the framework’s operating conditions.The paper also notes inherited artifacts, biases, and safety risks, and recommends disclosing generated videos as synthetic content in real-world use.Source: Limitations · p. 8
• As an orchestration framework rather than a unified world-consistent model, DramaAgent may still struggle with complex interactions, physical contact, large pose or camera changes, and residual identity drift.It improves scene-level audiovisual consistency but does not fully solve frame-level lip-sync or precise sound-effect timing, which depend on the underlying generators.Source: Limitations · p. 8

A Detailed Generation Settings

• Experiments use 50 narrative scripts of 10–20 sentences, corresponding to approximately 30–60 seconds of video after scene decomposition and composition.Table 5 summarizes the generation settings used for reproducibility.Source: A Detailed Generation Settings · p. 11
• Across backbones, DramaAgent keeps its upper-level logic and scene-level budget fixed within comparison groups, but conditioning interfaces and API variants can differ.Some APIs may evolve, so exact outputs can vary across access dates; the paper records API variant and access date for reproducibility.Source: A Detailed Generation Settings · p. 11

A.1 Runtime and Cost Statistics

• 18–32 minutes is the average end-to-end generation time for a 30–60 second script, depending on scene count, repair rounds, and selected backbone APIs.Initial scene generation averages 75 seconds, and each repaired scene averages 28 seconds.Source: §A.1 · p. 11
• 4.1 API calls per scene is the average usage, including candidate generation and optional regeneration calls.The default setting uses K = 3 candidates per scene and at most R = 2 repair rounds.Source: §A.1 · p. 11
• $7.8 per minute of final video is the approximate generation cost under the default setup, with latency and pricing varying across providers and access dates.The default configuration uses 5-second clips at 1280×720 resolution.Source: §A.1 · p. 11

B Reflection Settings and Repair Policy

• The repair policy specifies that, for each scene, DramaAgent first generates K = 3 candidate clips; Table 7 summarizes the default reflection hyperparameters.Source: B Reflection Settings and Repair Policy · p. 11
• A best-candidate score below τ = 0.80 triggers repair, which targets the weakest identity, semantic, temporal, or audiovisual sub-score.The corresponding repair strengthens character references, adds missing prompt content, injects transition cues, or adjusts dialogue timing and temporal constraints.Source: B Reflection Settings and Repair Policy · p. 11
• Repair is limited to R = 2 rounds per scene and stops early when score improvement falls below ϵ = 0.02.The repair policy thus combines a maximum-round limit with an improvement-based stopping rule.Source: B Reflection Settings and Repair Policy · p. 11

C Human Evaluation Protocol

• Fifteen blind evaluators with relevant assessment experience compare paired outputs under the same prompt, with each sample judged independently by at least three annotators.The protocol evaluates temporal coherence, character consistency, narrative fidelity, and audio-visual alignment; inter-rater agreement is measured by Fleiss’ kappa.Source: C Human Evaluation Protocol · p. 11
• DramaAgent is preferred over baseline pipelines across all four human-evaluation dimensions: temporal coherence, character consistency, narrative fidelity, and audio-visual alignment.Table 8 reports preference rates, while the supplied passage gives no numeric rates.Source: Table 8 · p. 12, C Human Evaluation Protocol · p. 12

D.1 Story Analysis and Plot Breakdown Prompt

• An instruction-tuned language model expands the narrative prompt and decomposes it into ordered scene descriptions specifying characters, environment, emotional or narrative role, and transition cues.The resulting scene sequence is denoted S = {S1, S2, . . . , SN}.Source: §D.1 · p. 12

D.2 Character Stills Prompt

• Character Still references are generated from extracted character attributes and reused during scene generation and reflective repair, with qualitative examples in family-adventure and action-crime narratives.The prompt aims to preserve identity-defining traits such as facial structure, hairstyle, costume, and artistic style.Source: §D.2 · pp. 12–13, Figure 5 · p. 13, Figure 6 · p. 13

D.3 Reflective Rewrite Prompt

• Reflective rewriting uses the current scene, detected failure, accepted-scene summary, and character references to revise only the problematic scene prompt while preserving story intent.Rewrites target identity inconsistency, semantic omissions, temporal discontinuity, or audio-visual mismatch.Source: §D.3 · p. 13

D.4 Reflection Thresholds and Repair Settings

• Reflection evaluates candidates on identity, semantic fidelity, temporal continuity, and audio-visual compatibility; repair stops at the round limit or when improvement falls below ϵ.The stated settings are 3 candidate clips per scene and at most 2 repair rounds, intended to balance output quality and computational cost.Source: §D.4 · pp. 13–14

E Additional Qualitative Results

• Cases span animated family adventure, action-crime ensemble, fantasy mystery, and mythic fantasy, rather than a single story template or visual style.The examples highlight recurring identity, multi-character coordination, atmospheric consistency, and emotional continuity.Source: E Additional Qualitative Results · pp. 14–15
• Across scene transitions, characters remain recognizable while stories develop from setup through conflict toward confrontation or resolution.The examples span cinematic live-action prompts, stylized animation, fantasy environments, and mythology-inspired compositions.Source: E Additional Qualitative Results · pp. 14–15
• The examples depict comedic family interactions, tactical urban progression, clue-driven magical exploration, and mythic conflict and farewell.These cases illustrate scene progression and recurring character identity across distinct narratives.Source: E Additional Qualitative Results · p. 15
• Across diverse prompts and visual styles, the qualitative cases support DramaAgent’s claim of improved long-form story coherence.The authors attribute this to structured planning, persistent character conditioning, and reflective refinement.Source: E Additional Qualitative Results · p. 15

F Discussion

• DramaAgent studies whether structured planning, persistent character conditioning, and reflective repair improve story-level coherence without introducing a new backbone.The discussion presents the agentic control layer as practical for more controllable and robust generation across heterogeneous video generators.Source: F Discussion · p. 15
• Current automatic metrics may not capture story-level coherence, long-horizon character persistence, or audio-visual alignment across scenes.Even improved automatic scores may not reflect whether viewers perceive a story as globally coherent.Source: F Discussion · p. 15
• Future work calls for richer long-form multimodal benchmarks and evaluators that reason jointly over text, video, and audio.The stated goal is to better measure coherent long-form generation and support feedback loops for automatic repair and refinement.Source: F Discussion · p. 15