Source-linked AI summary
SkyScript-100M: 1,000,000,000 Pairs of Scripts and Shooting Scripts for Short Drama
Jing Tang, Quanlu Jia, Yuqiang Xie, Zeyu Gong, Xiang Wen, Jiayi Zhang, Yalong Guo, Guibin Chen, Jiangping Yang
TL;DR
Short-drama video generation needs high-quality shooting scripts that capture scene, shot, and world information, while existing scripts omit important dramatic and relational annotations. The paper constructs SkyScript-100M by annotating short-drama keyframes and generating paired scripts, then uses the dataset to develop a richer generation paradigm and explore downstream applications. The dataset contains 1,000,000,000 script–shooting-script pairs and supports investigations into highlight detection, world-layout understanding, and implicit character relationships.
Problem
Existing shooting scripts lack annotations for dramatic highlights, promotional relationships, and detailed short-drama world information needed for automated production.
Method
The authors collect short dramas, extract and annotate keyframes, refine shooting-script structure, and generate paired scripts with SkyReels.
Results
SkyScript-100M contains 1,000,000,000 paired scripts and shooting scripts and is used to study short-drama generation, highlight detection, world-layout understanding, and implicit character relationships.
Takeaways & Limitations
The dataset provides a short-drama-specific foundation for richer script optimization and video-generation research.
Takeaways & Limitations
Bioelectric-signal-based highlight detection is expensive and difficult to scale or update in real time for popular short dramas.
Abstract
from arXiv · showhide
Generating high-quality shooting scripts containing information such as scene and shot language is essential for short drama script generation. We collect 6,660 popular short drama episodes from the Internet, each with an average of 100 short episodes, and the total number of short episodes is about 80,000, with a total duration of about 2,000 hours and totaling 10 terabytes (TB). We perform keyframe extraction and annotation on each episode to obtain about 10,000,000 shooting scripts. We perform 100 script restorations on the extracted shooting scripts based on our self-developed large short drama generation model SkyReels. This leads to a dataset containing 1,000,000,000 pairs of scripts and shooting scripts for short dramas, called SkyScript-100M. We compare SkyScript-100M with the existing dataset in detail and demonstrate some deeper insights that can be achieved based on SkyScript-100M. Based on SkyScript-100M, researchers can achieve several deeper and more far-reaching script optimization goals, which may drive a paradigm shift in the entire field of text-to-video and significantly advance the field of short drama video generation. The data and code are available at https://github.com/vaew/SkyScript-100M.
1 INTRODUCTION
Short drama production is shifting toward AI, but existing shooting scripts omit information needed for fully automated generation. SkyScript-100M addresses this gap with richly annotated shooting scripts and 1,000,000,000 paired scripts and shooting scripts.
- Motivation: Short drama production has become a major application target as video generation advances and companies pursue AI-powered drama production.The paper focuses on short drama because of its rapid growth and production needs.
- Motivation: Existing shooting scripts lack annotations for dramatic highlights and promotional information such as character-pairing compatibility.These omissions make fully automated, AI-powered drama production difficult.
- Dataset contribution: The dataset redefines shooting-script structure to include detailed information about the short-drama world for AI-powered production.The refined structure is presented in Table 1.
2 RELATED WORKS
Prior work spans broad video-text datasets and language-model-based story or script generation, but short-drama generation requires domain-specific world knowledge and structured data.
- Video-text datasets: Existing video datasets cover multiple domains, making data relatively sparse for a particular target direction.This dispersion can degrade training performance for specialized multimodal tasks.
- Story generation: Earlier story-generation research developed large paired datasets and hierarchical planning methods for structured or open-domain story generation.Examples include WP and ROC-Stories-based approaches.
- Script generation: Large language models support coherent script generation by providing broader world consensus than earlier WP- and ROC-based approaches.Dramatron and subsequent work extend this direction to long-text coherence, modular story management, and multimodal scripts.
- Gap: Short-drama worlds contain domain-specific common-sense assumptions that may not align with language-model internal knowledge.The authors motivate specialized paired script and shooting-script data for this setting.
3 THE SKYSCRIPT-100M DATATASET
SkyScript-100M combines professional-screenwriter-designed shooting scripts with multimodal pre-extraction, cleaning, geometric and character post-processing, and quality calibration. The pipeline produces structured frame descriptions paired with generated short-drama scripts.
- Dataset construction: SkyScript-100M uses large-scale frame descriptions of shots, characters, and objects paired with generated short-drama scripts.Professional screenwriters helped design the shooting-script format for short-drama video generation.
- MLLM pre-extraction: InternVL2-Llama3-76B performs preliminary multimodal key-information extraction to reduce manual annotation cost.The authors compared several multimodal models before selecting it for pre-extraction.
- Information processing: The extracted information is cleaned and converted into standard JSON for downstream object detection, character detection, and protagonist processing.Prompt optimization aligns extraction with the short-drama world, while Tables 3–5 show prompts and sample descriptions.
- Privacy and cleaning: Face pixelation and character-name masking are applied during cleaning to support ethical safety before later calibration and information extraction.The processed images are used with subsequent JSON post-processing.
- Protagonist post-processing: Deepface detects and annotates faces for 2D character-position understanding, while additional processing enriches depth, posture, age, gender, emotion, and ethnicity information.Test-time augmentation resolves conflicts between pre-extracted information and Deepface detections.
- Quality calibration: A 12-annotator calibration team replenishes omissions, corrects mislabels, and removes redundant annotations from sampled final results.The calibration process aligns annotations with professional short-drama evaluation criteria.
4 NEW SHORT DRAMA GENERATION PARADIGM
The paper proposes a new shooting-script paradigm that supplies video-generation models with richer information about the short-drama world, improving consistency and story understanding. SkyReels evaluations report strong performance across multiple script-quality dimensions.
- 4 NEW SHORT DRAMA GENERATION PARADIGM: The traditional drama-generation pipeline proceeds through story generation, script generation, character design, and shooting-script generation, with the final stage determining generated-video quality.The supplied pipeline is identified as Table 7.
- 4 NEW SHORT DRAMA GENERATION PARADIGM: The new shooting-script paradigm feeds detailed short-drama world information to video generation, preserving consistency across characters, objects, and world concepts.It expands beyond scene and camera parameters to support multi-frame generation with a consistent world representation.
- 4 NEW SHORT DRAMA GENERATION PARADIGM: SkyReels shows outstanding performance across theme expression, character development, dialogue quality, emotional impact, pacing, conflict resolution, plot coherence, and narrative structure compared with other advanced LLMs.The paper also reports better character and painting-style consistency, alongside deeper understanding of character relationships and plot trends.
5 BROAD IMPACT ON SHORT DRAMA
SkyScript-100M supports exploratory improvements across short-drama generation, including continuous highlight annotation, 3D world-layout understanding, anomalous-occlusion handling, and time-varying implicit relationship mining.
- 5 BROAD IMPACT ON SHORT DRAMA: SkyScript-100M enables broad exploratory research on short-drama generation and optimization of the SkyReels large short-drama generation model.The section uses the paired Script-Shooting Script data as the basis for multiple downstream studies.
- 5 BROAD IMPACT ON SHORT DRAMA: The section presents a new short-drama video-generation paradigm and related counterfactual script-generation directions enabled by the dataset.The supplied passages identify the paradigm and counterfactual generation as broader applications of SkyScript-100M.
- 5.1 VIDEO HIGHLIGHT DETECTION: Short-drama highlight detection is difficult because users rarely post barrages and likes usually apply to whole episodes rather than individual episodes.The section also notes that biosignal-based approaches are expensive and difficult to scale or update rapidly.
- 5.1 VIDEO HIGHLIGHT DETECTION: The paper estimates continuous per-episode highlight scores from character emotions using Plutchik theory and Valence, Arousal, and Dominance dimensions.This differs from previous single-frame discrete highlight annotations.
- 5.2 WORLD LAYOUT UNDERSTANDING: World-layout understanding combines rough 2D character positions, Multiple View Geometry, 3D positions, and a 3D-2D transformation matrix to regenerate new 2D views.Consistency constraints enforce physically plausible screen positions and can help circumvent anomalous occlusion.
- 5.3 IMPLICIT CHARACTER RELATIONSHIP MINING: The paired data supports mining time-varying implicit character relationships from images and annotated text using multimodal and unimodal large language models.Annotated character locations and continuous emotions support high-accuracy text-based mining, which can inform script generation about relational complexity and uncertainty.
6 CONCLUSION
The paper constructs SkyScript-100M as a multimodal short-drama dataset with one billion Script-Shooting Script pairs and explores its uses for generation and analysis.
- 6 CONCLUSION: SkyScript-100M contains 1,000,000,000 Script-Shooting Script pairs derived from a multimodal short-drama dataset.The paper describes the dataset construction, proposes a new short-drama generation paradigm, and studies applications in highlight detection, world-layout understanding, and implicit relationship mining.