Source-linked AI summary
COMIC: Agentic Sketch Comedy Generation
Susung Hong, Brian Curless, Ira Kemelmacher-Shlizerman, Steve Seitz
TL;DR
Humor is difficult to generate reliably, especially when producing coherent long-form comedic video. COMIC uses a multi-agent, multi-island pipeline with human-aligned critics and iterative competition, and reports results approaching professionally produced sketches while establishing state-of-the-art agentic video generation. Its scope is constrained by computational costs and by using normalized YouTube view counts as a noisy proxy for humor quality.
Problem
Reliable automated generation of funny, coherent comedic videos remains difficult because humor is subjective and current video models produce short clips with limited inter-clip consistency.
Method
COMIC uses production-role agents, multi-island script competition, iterative rendering refinement, and LLM humor critics aligned with engagement patterns from YouTube comedy videos.
Results
COMIC produces results approaching professionally produced comedy sketches and establishes a new state of the art for automated long-form video production.
Takeaways & Limitations
Diverse human-aligned critics and competitive iterative refinement support automated exploration and coherent improvement in open-ended sketch-comedy generation.
Takeaways & Limitations
The framework incurs computational costs, and normalized YouTube view counts may be noisy proxies for humor quality because of clickbait and algorithmic promotion.
Abstract
from arXiv · showhide
We propose a fully automated AI system that produces short comedic videos similar to sketch shows such as Saturday Night Live. Starting with character references, the system employs a population of agents loosely based on real production studio roles, structured to optimize the quality and diversity of ideas and outputs through iterative competition, evaluation, and improvement. A key contribution is the introduction of LLM critics aligned with real viewer preferences through the analysis of a corpus of comedy videos on YouTube to automatically evaluate humor. Our experiments show that our framework produces results approaching the quality of professionally produced sketches while demonstrating state-of-the-art performance in video generation.
1 Introduction
COMIC addresses the difficulty of reliably generating funny, coherent sketch-comedy videos through a fully automated, multi-agent production framework. It combines iterative competition, diverse critics, and viewer-aligned humor evaluation across scriptwriting and rendering.
- Humor remains difficult for AI because reliably producing content that makes audiences laugh is challenging despite occasional successful outputs.
- COMIC generates short comedic videos from character and background references while automating scenario design, scriptwriting, and high-quality video production.
- LLM-based humor critics are aligned with human preferences by analyzing YouTube comedy videos and their viewer engagement.
- Multiple script islands use critic committees, tournaments, and winner-guided refinement to promote diverse ideas and improve scripts.
- Scene directors decompose refined scripts into shots, render them consecutively for continuity, and refine results using script-conditioned rendering critics.
- COMIC is presented as the first fully automated agentic system for comedic video generation, with evaluations approaching professionally produced sketch quality.
2 Background
Prior work applies evolutionary computation, LLM multi-agent optimization, and controllable or agentic video generation to creative and video-production tasks. However, existing systems remain limited for the narrative complexity and quality demands of sketch comedy.
- Evolutionary methods support creative optimization through genetic algorithms, quality-diversity techniques, and distributed populations that balance exploration and exploitation.
- LLM research spans prompt optimization, heuristic discovery, mathematical reasoning, simulated development ecosystems, multi-agent debate, and iterative optimization of text and agent behavior.
- Video-generation models demonstrate strong text-to-video capabilities and increasingly incorporate controls such as audio conditioning.
- Agentic video systems use LLMs for direction, layout planning, self-improvement, or storyboarding, but remain limited in handling sketch comedy’s narrative complexity and quality demands.
3 Content Optimization via Multi-Agent Iterative Competition (COMIC)
COMIC frames sketch-comedy video generation as a coupled problem of writing a satisfying script and rendering a coherent, consistent shot sequence, addressed through human-aligned critics and iterative competition. Its pipeline uses viewer-calibrated critic selection, island-based script evolution, and critic-guided audiovisual refinement.
- Problem: COMIC seeks to generate a narratively coherent, visually consistent, genuinely funny video from character specifications and background assets.Character specifications include portrait images, voice samples, and text descriptions.
- Problem: The system decomposes generation into script synthesis and visual realization, with the latter translating the script into a continuous shot sequence.Scripts establish a comedic premise, develop character interaction, and deliver a payoff; visual realization preserves narrative, identity, and continuity.
- Pipeline: COMIC models studio roles as agents that generate concepts, write and revise scripts, storyboard scenes, render shots, synthesize voices, and refine videos.The design uses multiple rounds because a single pass is insufficient for high-quality results.
- Iterative competition: Script populations evolve on isolated islands through pairwise competition, while video rendering uses test-time scaling and tournaments across refinement histories and diverse realizations.Island-specific committees shape distinct evaluative standards, and additional inference compute can be directed toward rendering quality without retraining.
- Viewer-aligned evaluation: Critics are generated from engagement-labeled YouTube sketch data and selected for task- and channel-specific accuracy rather than relying on a pooled or single-best critic.The corpus contains 4,940 data points from five sketch-comedy channels, with view counts used as a popularity proxy and age-normalized through per-channel logistic growth models.
- Viewer-aligned evaluation: Task-specific critic selection outperforms pooled and single-best baselines across channels and engagement tiers and retains superior discrimination on held-out scripts.The reported results indicate that the selection procedure generalizes beyond the validation set.
4 Experiments
COMIC improves automated comedic video generation through iterative, critic-guided competition and refinement. Experiments show strong visual and narrative coherence, competitive human judgments, and gains from diversity-preserving evolution and increased test-time compute.
- Video Results: COMIC generates visually coherent and narratively purposeful sketch comedy videos from minimal character and background specifications.Generated sketches include complete comedic arcs, consistent character identities, stable backgrounds, and continuous scene transitions.
- Baseline Comparison: COMIC consistently outperforms agentic video baselines across human-rated Funniness, Watch More, Script, Narrative, Realism, and Consistency.The agentic baselines score between Definitely Not and Probably Not on Watch More, while COMIC scores between Unlikely and Neutral.
- Baseline Comparison: COMIC reaches a human-comparison level between Slightly Less Funny and Comparable, which neither frontier video models nor agentic baselines achieve.Frontier models score higher on Realism and Consistency, but COMIC outperforms both on Watch More despite its longer outputs.
- Automated Evaluation: The automated ranking COMIC > Sora > Veo > MA ≈ VGoT aligns with human results, while COMIC achieves the highest overall inter- and intra-diversity.The benchmark compares generated videos with middle-tier test videos using selected critics and reports win rate and diversity across channels.
- Island-Based Evolution: Win rate rises sharply through generation 4, while inter-diversity first decreases and then recovers as critic committees promote distinct niches.Rendering critics also correct issues including character mismatches and framing errors.
- Multi-Island: Multi-island evolution produces higher overall win rate and intra-diversity than a single-island configuration.The comparison controls for the number of iterations per script to maintain a fair evaluation.
- Scale: Increasing the numbers of islands, scripts, and critics improves win rate, showing that COMIC trades test-time compute for enhanced performance.The large configuration outperforms the small and base baselines on top-script win rate.
- Ablation Study: Human raters overwhelmingly prefer the full COMIC pipeline over the critic-free ablation across Script, Narrative, Realism, Consistency, and Funniness.This ablation supports the contribution of iterative multiagent critic refinement to output quality.
5 Conclusion
COMIC is presented as a multi-agent framework for open-ended comedy generation, with iterative competition and human-aligned critics. The conclusion highlights strong performance while acknowledging computational and evaluation-proxy limitations.
- 5 Conclusion: COMIC uses multi-agent competition and diverse critics to address open-ended sketch comedy generation.The framework is described as a state-of-the-art approach for automated video production.
- 5 Conclusion: The system’s evaluation uses normalized YouTube view counts as a proxy for humor quality.The authors caution that clickbait and algorithmic promotion may introduce noise into this proxy.
- 5 Conclusion: Iterative refinement incurs computational costs despite possible time savings from parallelizing local structures.This is identified as a limitation of the framework.
- 5 Conclusion: COMIC’s improvements occur without parameter updates, gradient-based optimization, or a fixed reward signal.The paper connects this iterative improvement mechanism to the Red Queen hypothesis.
A Video Results
The supplementary material contains MP4 files of the generated videos. Readers are encouraged to watch these files as part of evaluating the video results.
- A Video Results: The generated videos are provided as MP4 files in separate supplementary material.The files contain the videos associated with the paper’s results.
- A Video Results: The video files are distributed separately from the main paper.The passage directs readers to the supplementary material for access.
- A Video Results: Readers are strongly encouraged to watch the supplementary videos.Direct viewing is presented as part of engaging with the reported video results.
B Critic Selection
The critic-selection analysis evaluates in-context sample sizes and engagement-tier ranking performance. The data-processing pipeline uses logistic growth fits to model cumulative views across five comedy channels.
- B Critic Selection: 0, 15, and 45 tier-labeled in-context samples are compared for selecting high-performing script critics.The study measures selection performance across different engagement tiers.
- B Critic Selection: Task-wise selection achieves the highest validation accuracy and benefits most from calibration examples.This comparison is reported in Table 5 as a function of in-context sample size.
- B Critic Selection: 4,940 data points from five YouTube sketch comedy channels support the engagement analysis.Videos failing length and format criteria are excluded.
- B Critic Selection: A logistic growth function models each sketch’s cumulative view trajectory.The model reflects the observed S-curve behavior of online video view counts and supports projected carrying-capacity estimates.
C Storyboard Output Structure
The scene director produces storyboard outputs with a defined structure for organizing generated scenes.
- C Storyboard Output Structure: The scene director produces a storyboard output structure.The figure documents the structure generated during the storyboard stage.
- C Storyboard Output Structure: Storyboard structure is an explicit output of the scene-director component.The figure associates the output with the scene director rather than another production role.
- C Storyboard Output Structure: The storyboard output organizes information produced for the video-generation process.Its documented role is to represent the scene director’s structured output.
D Scale Configurations
COMIC scales comedy generation through island-based script competitions, rendering refinement, and modular model assignments across production stages. Its local evaluation reduces computational cost while supporting parallel execution and quality control before rendering.
- Computational complexity: Island-local evaluation is a computational necessity that makes iterative refinement at scale tractable, not only a diversity mechanism.
- Computational complexity: Island-local script tournaments require O(G · K · |S_k|^2 · |C_k|) evaluations across K islands and G generations.This is lower than globally pooling all K|S_k| scripts, which requires O(G · K^2 · |S_k|^2 · |C_k|) evaluations.
- Computational complexity: Rendering D scene directions across N shots and |C_render| critic iterations has dominant cost O(D · N · |C_render|^2) render calls per script.Shot-level tournament selection adds O(D·N ·|C_render|^2) comparisons, while final video-level selection costs O(D · |C_render|) comparisons.
- Scale configurations: Approximately one day on a single H200 GPU and an API budget of around $5 are required for the base configuration.The system can parallelize API or model calls across multiple islands and storyboards.
- Pipeline scaling: Scripts undergo inspection for formatting, character, dialogue, and structural issues before rendering, then compete across islands for selection.The cross-island league uses the best specialized critic, and the top four scripts proceed to production after final evaluation.
- Modular production models: COMIC assigns different foundational models to concept generation, critics, inspection, meta-critic instruction generation, scene direction, visual synthesis, voice consistency, and video rendering.The listed models include Claude 3.5 Sonnet, Claude 3.5 Haiku, Gemini 3 Flash Preview, Claude 3.5 Opus, FLUX.2, TAG, ElevenLabs, Chatterbox-TTS, and Wan 2.1.
G Human Evaluation Protocol
The human evaluation compares COMIC with four baseline video-generation methods using internationally recruited participants and multiple rating dimensions. A separate paired A/B study compares COMIC with a critic-free baseline across humor, narrative, and visual qualities.
- Baseline comparison: Five methods were evaluated with 22 responses per method, 110 responses total, using four videos per method.Participants came from the US, Europe, and Asia; each participant viewed two videos per method.
- Baseline comparison: COMIC’s four evaluation videos were automatically selected from top-performing scripts without manual curation, while each baseline was run four times.
- Rating dimensions: Participants rated each video on a 7-point Likert scale for funniness, re-watch intent, and comparison to human comedy.The supplied rating definitions span values from 1 to 7 for each dimension.
- Ablation study: The ablation paired COMIC against a critic-free baseline across funniness, script quality, narrative quality, visual realism, and visual consistency.Neutral “About the same” responses were excluded from the analysis.