Source-linked AI summary
CineAGI: Character-Consistent Movie Creation through LLM-Orchestrated Multi-Modal Generation and Cross-Scene Integration
Tianyidan Xie, Zhentao Huang, Mingjie Wang, Xin Huang, Jun Zhou, Minglun Gong, Zili Yi
TL;DR
Automated movie creation must coordinate narrative, characters, and audio-visual elements across extended sequences, where existing approaches struggle with coherence and synchronization. CineAGI combines multi-agent narrative synthesis, decoupled character processing, and hierarchical audiovisual coordination; experiments report gains across consistency and quality measures.
Problem
Automated movie creation requires coordinating characters, modalities, and narrative elements over extended sequences, while existing end-to-end approaches struggle to do so effectively.
Source: §I · p. 1Method
CineAGI hierarchically coordinates specialized LLM agents, a decoupled character-centric pipeline, and audio-visual synchronization.
Source: §I · p. 1Results
40% overall-consistency improvement, 4.4% subject-consistency gain, 5.4% aesthetic-quality enhancement, and 28.7% higher character consistency were reported.
Source: §I · pp. 1–2Takeaways & Limitations
CineAGI provides a modular foundation for AI-driven filmmaking that coordinates narrative synthesis, character processing, and audiovisual synchronization.
Source: §V · p. 6
Abstract
from arXiv · showhide
Automated movie creation requires coordinating multiple characters, modalities, and narrative elements across extended sequences -- a challenge that existing end-to-end approaches struggle to address effectively. We present \textbf{CineAGI}, a hierarchical movie generation framework that decomposes this complex task through specialized multi-agent orchestration. Our framework employs three key innovations: (1) a multi-agent narrative synthesis module where specialized LLM agents collaboratively generate comprehensive cinematic blueprints with character profiles, scene descriptions, and cross-modal specifications; (2) a decoupled character-centric pipeline that maintains identity consistency through instance-level tracking and integration while enabling flexible multi-character composition; and (3) a hierarchical audio-visual synchronization mechanism ensuring frame-level alignment of dialogue, expressions, and music. Extensive experiments demonstrate that CineAGI achieves 40\% improvement in overall consistency, 4.4\% gain in subject consistency, 5.4\% enhancement in aesthetic quality, and 28.7\% higher character consistency compared to baselines. Our work establishes a principled foundation for automated multi-scene video generation that preserves narrative coherence and character authenticity.
I. INTRODUCTION
CineAGI targets identity preservation, scene coherence, and audiovisual synchronization in long-form movie generation. It coordinates specialized agents and decoupled pipelines, with reported gains across consistency and quality measures.
I. INTRODUCTION
- Existing methods face identity loss across scenes, memory-limited scene coherence, and missing frame-level synchronization of lip movements, expressions, and music.Identity preservation spans global attributes such as facial structure and physique through fine details such as expressions and lighting adaptation.Source: §I · p. 1
- CineAGI decomposes movie creation through specialized LLM-agent coordination and structured cross-modal information flow.Character Designer, Script Writer, Storyteller, Composer, and Quality Inspector collaboratively generate and validate cinematic blueprints.Source: §I · p. 1
- A decoupled character-centric pipeline combines segmentation, identity-preserving face integration, and talking-face synthesis for multi-character processing across scene contexts.The pipeline is described as preserving global consistency while retaining creative flexibility.Source: §I · p. 1
- Hierarchical audio-visual coordination aligns lips at frame level, generates emotion-aware music, and integrates scenes to address cross-modal misalignment.The approach coordinates multiple levels of audiovisual production rather than relying on end-to-end synchronization.Source: §I · p. 1
- 40% overall-consistency improvement, 4.4% subject-consistency gain, 5.4% aesthetic-quality enhancement, and 28.7% higher character consistency were reported.The cited passages report these gains across the evaluation results and compare character consistency to state-of-the-art baselines.Source: §I · pp. 1–2
A. Text-to-Video Generation
Prior text-to-video and agent-based systems advance duration, fidelity, consistency, and planning, but long-form multi-character coherence and synchronization remain challenging. CineAGI uses hierarchical agent coordination and decoupled character generation to address these gaps.
A. Text-to-Video Generation
- Text-to-video systems advance duration and visual fidelity but retain localized processing windows and limited explicit narrative-structure modeling.The passage discusses Sora, Lumiere, Stable Video Diffusion, CogVideoX, and VideoCrafter as examples of recent systems.Source: A. Text-to-Video Generation · p. 2
- Long-form generation, character animation, and LLM-guided planning have advanced, yet multi-character consistency, cross-scene coherence, and multimodal coordination remain challenging.Examples include ID-Animator, ConsisID, StoryDiffusion, Animate Anyone, StreamingT2V, FIFO-Diffusion, VideoDirectorGPT, VideoPoet, and MovieGen.Source: A. Text-to-Video Generation · p. 2
- CineAGI decouples character generation from scene synthesis through hierarchical multi-agent orchestration to support character consistency and narrative flexibility across extended sequences.This is the stated distinction from existing approaches.Source: A. Text-to-Video Generation · p. 2
B. Multi-Agent Systems
- CineAGI coordinates specialized production-role agents through structured information flow and validation to support creative consistency and temporal coherence.The agents include Character Designer, Script Writer, Storyteller, Composer, and Quality Inspector.Source: B. Multi-Agent Systems · p. 2
III. METHOD
CineAGI divides movie creation into coordinated modules that optimize components while maintaining global coherence. Its narrative synthesis agents establish character profiles, scripts, scene flows, dialogue timing, music direction, and validated outputs.
A. Overview
- The framework separates narrative planning, character asset creation, and cinematographic synthesis, enabling targeted component optimization while maintaining global coherence.The three modules coordinate story specifications, audiovisual assets, video generation, character integration, and synchronization.Source: A. Overview · pp. 2–3, Figure 2 · p. 3
B. Narrative Synthesis Module
- Narrative synthesis uses coordinated LLM agents and structured information flow to maintain cross-modal consistency, unlike approaches that treat narrative elements independently.The cited passage compares with previous approaches [1],,.Source: B. Narrative Synthesis Module · p. 3
- Character Designer defines hierarchical appearance, personality, and behavioral attributes, while Script Writer turns those profiles into scene-specific technical descriptions.Script details include visual composition, character positioning, camera movements, and detection keywords for downstream synthesis.Source: B. Narrative Synthesis Module · p. 3
- Storyteller organizes coherent scenes around character relationships and personalities, with dialogue timed at frame level for pacing and emotional progression.The passage describes preserving dramatic arcs alongside character-aligned actions.Source: B. Narrative Synthesis Module · p. 3
- Composer specifies cohesive background music from character profiles, scenes, and dialogue, while Quality Inspector validates agent-output connections and standardizes downstream inputs.The inspector checks consistency and relevance to the original input and outputs structured JSON.Source: B. Narrative Synthesis Module · p. 3
C. Character Generation Module
The character generation module converts narrative-stage profiles and dialogue scripts into visual and voice assets using specialized components.
C. Character Generation Module
- The module receives character profiles and dialogue scripts from narrative synthesis and produces concrete audiovisual assets.It uses portrait and voice-generation components.Source: C. Character Generation Module · p. 3
- Portrait Artist uses RealVisXL 3.0 to generate multi-feature character reference portraits for visual consistency during later face swapping.The portraits are generated from detailed character profiles.Source: C. Character Generation Module · p. 3
- Sound Generator uses ChatTTS to synthesize character-specific voice assets with identity preservation and dynamic emotional modulation.The voice profiles are derived from comprehensive character specifications.Source: C. Character Generation Module · p. 3
D. Cinematographic Synthesis Module
CineAGI’s cinematographic synthesis module combines text-driven scene creation with decoupled character processing and synchronized audiovisual assembly.
Scene Creator
- Scene Creator uses HunyuanVideo-13B and rich textual character descriptions to support flexible multi-character scene composition.Character specifications are encoded in text rather than reference images, providing a basis for subsequent character-specific processing.Source: D. Cinematographic Synthesis Module · p. 4
Decoupled Character Integration
- Decoupled Character Integration divides multi-character scenes into independently processed segments while preserving spatial-temporal relationships and scene context.The pipeline uses text-guided Grounded-SAM2 segmentation, SimSwap identity integration, and Wav2Lip talking-face synthesis.Source: D. Cinematographic Synthesis Module · p. 4
Music Virtuoso
- Music Virtuoso uses MusicGen to produce scene-specific background music aligned with emotional context and dramatic progression without interfering with dialogue clarity.Composer-provided musical directions guide music generation, which adapts to scene duration.Source: D. Cinematographic Synthesis Module · p. 4
Cinematographer
- Cinematographer reintegrates characters, dialogue audio, subtitles, and music, then concatenates processed scenes in narrative order.The multistage assembly is intended to synchronize audiovisual elements in the final output.Source: D. Cinematographic Synthesis Module · p. 4
IV. EXPERIMENTS
The experiments evaluate CineAGI on a 100-prompt, five-genre benchmark using standardized video settings, multiple metrics, and comparisons with three baselines.
A. Experimental Setup
- The benchmark contains 100 diverse story prompts across five genres, with character descriptions, plot elements, and emotional arcs.The genres are romantic comedies, action sequences, dramatic narratives, family dramas, and suspense thrillers.Source: A. Experimental Setup · p. 4
- Videos are sampled at 24 FPS, 5.375 seconds per scene, and 512×512 resolution; baseline scores average two strategies for equivalent-length generation.The strategies use multiple random seeds to match total video length or Script Writer scene descriptions to generate equivalent-length videos per scene.Source: A. Experimental Setup · p. 4
- VBench measures Overall Consistency, Subject Consistency, Aesthetic Quality, and Motion Smoothness.The metrics assess text-to-video alignment and story flow, character identity across scenes, visual fidelity and composition, and motion fluidity, respectively.Source: A. Experimental Setup · p. 4, Table I · p. 4
- CineAGI is compared with CogVideoX, VideoCrafter2, and Hunyuan.The cited baseline markers are [8],, and [27].Source: A. Experimental Setup · p. 4
- The full pipeline takes approximately 11.3 minutes per 5.375-second scene on one NVIDIA A100 GPU.A per-component computational-cost breakdown is provided in Appendix E.Source: A. Experimental Setup · p. 4
B. Quantitative Evaluation
CineAGI records the highest scores across the four reported quantitative metrics, with gains over Hunyuan in consistency, subject consistency, and aesthetic quality.
B. Quantitative Evaluation
- 40% improvement in Overall Consistency over Hunyuan, 4.4% in Subject Consistency, 5.4% in Aesthetic Quality, and 1.1% in Motion Smoothness.CineAGI’s scores are 0.259, 0.949, 0.600, and 0.987, respectively, and are reported as the highest for each metric.Source: B. Quantitative Evaluation · p. 4
C. Human Evaluation
Human ratings report CineAGI’s performance across visual, narrative, character, and audio dimensions; ablations report changes associated with removing three components.
C. Human Evaluation
- Qualitative comparisons describe more coherent narratives, consistent character portrayals across varied scenes, superior visual quality, and natural character movements.The figure caption identifies comparisons with state-of-the-art methods.Source: Figure 3 · p. 5
- 28.7% higher Character Consistency (3.14), with Visual Quality 3.83, Narrative Coherence 3.57, and Audio Coherence 3.26 in the human evaluation.Twenty participants, including 10 with professional multimedia experience, rated videos on a five-point Likert scale.Source: C. Human Evaluation · pp. 4–5, Table II · p. 5
D. Ablation Study
- Removing Narrative Synthesis lowered Overall Consistency to 0.232 and Subject Consistency to 0.924; removing Quality Inspector yielded 0.245 and 0.938, respectively.The ablation passage describes the Quality Inspector changes as moderate.Source: D. Ablation Study · p. 5
- Without Decoupled Character Integration, Aesthetic Quality decreased to 0.583 and Motion Smoothness to 0.971.Figure 4 depicts the DCI sequence from original scene video through segmented regions and face-swapped results to synchronized talking-head output.Source: D. Ablation Study · p. 5, Figure 4 · p. 5
E. Qualitative Results
CineAGI is presented as a hierarchical framework, with experiments reporting gains across automated metrics and human evaluation. Qualitative results describe coherent movies with consistent character portrayals and appropriate cinematography.
E. Qualitative Results
- Figure 3 demonstrates coherent movies with consistent character portrayals and appropriate cinematography.Source: E. Qualitative Results · p. 5
- Table III reports an ablation study covering the narrative synthesis module (NSM), Quality Inspector (QI), and decoupled character integration (DCI).The supplied caption identifies the study and abbreviations but does not state its results.Source: Table III · p. 6
V. CONCLUSION
- Experiments validate significant gains across automated metrics and human evaluation, establishing a modular foundation for AI-driven filmmaking.Source: §V · p. 6