PaperlayerView exact revision on arXiv

Source-linked AI summary

How Do Professional Editors Evaluate the Editing Quality of AI-Generated Cinematic Video Ads?

Po-Ming Law, Weizhi Li, Arpit Narechania

arXiv:2608.24329v1cs.HC

TL;DR

AI-generated cinematic ads are increasingly common, but evaluating their editing quality requires more precise vocabulary than vague impressions. This paper analyzes cinematic ads, uses a two-step generation pipeline, and derives six editing-quality dimensions from professional editors’ critiques to support future evaluation and generation research.

  • Problem

    AI tools can automatically produce cinematic video ads, but existing evaluation lacks a fine-grained framework for identifying specific editing-quality issues.

  • Method

    The paper characterizes social-media cinematic ads, analyzes their structure and techniques, codes editing features with an iteratively refined codebook, and uses a two-step generation pipeline followed by professional-editor critiques.

  • Results

    Professional editors’ critiques of AI-generated cinematic ads yielded a framework of six editing-quality dimensions.

  • Takeaways & Limitations

    The six dimensions can guide editing-aware generation, structured human-evaluation rubrics, and automated evaluators aligned with professional judgment.

  • Takeaways & Limitations

    The findings are scoped to ads generated by an LLM-based shot-planning pipeline followed by rendering with a selected video-generation model; other pipelines may produce different strengths and weaknesses.

Abstract

from arXiv · show

On social media, we often encounter short-form video ads that employ cinematic editing techniques to evoke an emotional response. While AI tools are beginning to generate such cinematic ads automatically, we lack a fine-grained framework for evaluating these ads. In this paper, we first characterize social media video ad formats and identify cinematic ads as a recurring format in our corpus. We then analyze the duration, shot structure, audio and text elements, and editing techniques of cinematic ads to inform a two-stage generation pipeline in which an LLM first generates a shot plan and a video generation model renders the video. Using this pipeline, we generated 70 cinematic ads for 35 real brands and recruited professional video editors to critique their editing choices. From their critiques, we derive six dimensions of editing quality: narrative progression, audiovisual coordination and sound design, visual composition and graphics, shot-to-shot continuity, message and brand coherence, and temporal rhythm and pacing. We discuss how these dimensions can guide editing-aware generation, human evaluation, and automated evaluation of AI-generated cinematic ads.

1 Introduction

The paper addresses the lack of fine-grained vocabulary for evaluating AI-generated cinematic ads by characterizing ad formats, modeling cinematic ads, and deriving six professional editing-quality dimensions.

  • Motivation: Cinematic ads use cinematic editing techniques and music or narration to evoke emotion without a traditional narrative arc.The paper defines them as emotion-arousing ads built from cinematic editing techniques and lacking a problem-solving story structure.
  • Motivation: AI-generated ads can include captions, transitions, and coherent shot arrangements, yet their editing may still feel subtly wrong and difficult to articulate.The paper identifies the need for an appropriate vocabulary to move beyond vague impressions.
  • Motivation: Fine-grained editing dimensions help evaluators identify concrete problems such as frequent music changes, unsmooth motion transitions, and incoherent narratives.These examples illustrate how the framework turns general dissatisfaction into specific editorial diagnoses.
  • Study 1: 124 ads from 15 brands across five product categories yielded eight recurring formats, with cinematic ads among the most prevalent.The formats included cinematic ads, UGC, product demos, slideshows, problem-solving narratives, visual hooks, monologues, and interviews.
  • Study 1: 99 cinematic ads were analyzed for duration, shot count, audio and text elements, and editing techniques, suggesting that social-media cinematic ads differ from television ads and movies.They tend to be shorter than television ads, have shorter shot durations than movies, and use some editing techniques less frequently used in movies.
  • Study 2: A two-step pipeline uses an LLM-generated shot plan followed by video rendering, producing 70 cinematic ads for 35 brands evaluated by six professional editors.Editors provided open-ended critiques online and discussed them in one-hour interviews.
  • Study 2: The critiques produced six dimensions: narrative progression; audiovisual coordination and sound design; visual composition and graphics; shot-to-shot continuity; message and brand coherence; and temporal rhythm and pacing.The framework is intended to support editing-quality evaluation of AI-generated cinematic ads.
  • Implications: The framework supports future research on generation shortcomings, structured human-evaluation rubrics, and automated evaluators aligned with professional judgment.These uses connect the dimensions to both assessment and improvement of AI-generated cinematic ads.

2 Related Work

The related work spans ad-format analysis, film-editing techniques, editing workflows, reviewer feedback, AI-assisted creation, and video-quality benchmarks, while positioning this paper as ad-specific.

  • Video Ad Formats: Prior television-ad research links narrative structures with comprehensibility, likability, and persuasiveness, including effects of how visual metaphors are explained.Television-ad studies also distinguish classical dramas from vignette dramas and often identify problem-solving structures.
  • Video Ad Formats: The paper maps problem-solving narratives to classical dramas and cinematic ads to vignette dramas, because cinematic ads montage related imagery without a problem-solving arc.Its social-media analysis also identifies formats without televised counterparts, such as visual hooks.
  • Video Editing Techniques: Film-editing scholarship supplies vocabularies for seamless cuts, eyeline matches, match cuts, and temporal ellipses, but no consensus exists on a complete technique inventory.Accordingly, the paper uses literature and observations to analyze a non-exhaustive set of techniques.
  • Video Editing Practices: HCI research has developed interfaces for organizing clips, matching sound effects, shortening videos, and expressing editing intent through language and sketches.These systems address editing activities rather than the professional evaluation framework developed here.
  • Video Editing Practices: Research on reviewer feedback and AI-assisted video creation includes collaborative feedback tools and systems that generate multiple variations for editor review.This paper complements those studies by deriving evaluation dimensions from editors’ critiques of AI-generated ads.
  • Video Generation Benchmarks: Existing video benchmarks evaluate realism, plausibility, consistency, controllability, continuity, and aesthetic quality across generated videos and films.Film-oriented benchmarks do not address the commercial requirements specific to cinematic ads.
  • Video Generation Benchmarks: Message and brand coherence is an ad-specific dimension because it assesses whether shots convey a central message and accurately portray the brand image.The paper notes that this dimension is generally inapplicable to traditional films.

3 Study 1a: Characterizing Social Media Ad Formats

Study 1a asks which video-ad formats brands use on social media and answers through qualitative analysis of ads collected from the Meta and TikTok ad libraries.

  • Study Design: The study collected 124 social-media ads from the Meta and TikTok ad libraries for qualitative analysis of their formats.The research question concerns the formats brands run on social media.

3.1 Methodology

The study sampled ads across five consumer-facing product categories and selected high-impression English-language videos from 15 brands, removing duplicates before coding.

  • Sampling: The sample covered apparel and footwear, beauty and personal care, consumer technology, direct-to-consumer e-commerce, and food and beverage.Three brands were selected from each category to capture visual variation and diversity.
  • Sampling: Ads were retrieved from Meta and TikTok libraries, ranked by impressions, limited to five per brand per library, filtered to English, and deduplicated.The procedure yielded 124 unique videos.
  • Coding: An author open-coded all 124 ads, after which two coders independently coded random 25% subsets and refined the codebook through disagreement resolution.The codebook specified definitions and inclusion and exclusion criteria for each ad format.

3.2 Social Media Ad Formats

The corpus contains eight recurring social-media ad formats, distinguished by their narrative structure, presentation style, and editing conventions. Cinematic ads use imagery, music, narration, and cinematic techniques without a traditional problem-and-resolution arc.

  • 3.2 Social Media Ad Formats: Eight recurring formats emerged: cinematic, UGC, product demo, slideshow, problem-solving narrative, visual hook, monologue, and interview.The formats are described in descending order of corpus frequency.
  • 3.2.1 Cinematic: Cinematic ads evoke emotion through imagery, music, narration, and cinematic editing while lacking an explicit problem-and-resolution arc.A Coca-Cola example uses a commentator, fan montage, motion match cut, and goal-related audiovisual progression.
  • 3.2.2 UGC: UGC-style ads use raw, casual footage to present products or personal experiences through direct address, voiceover, or first-person captions.The Rare Beauty example combines vertical selfie framing with first-person narration during a brow-gel application.
  • 3.2.3 Product Demo: Product demo ads objectively explain product features or functionality, often combining demonstrations with instructional graphics.A DJI ad teaches three drone movements using aerial footage and controller graphics showing joystick inputs.
  • 3.2.4 Slideshow: Slideshow ads communicate through text-heavy visual cards connected by animated transitions.The Apple Watch example progresses from a scrolling problem to hypertension checking and a concluding product card.
  • 3.2.5 Problem-Solving Narrative: Problem-solving narratives establish a problem, frustration, or situation that the advertised product or service resolves.A Sephora example moves from a father’s uncertainty about setting a bun to expert advice and the daughter’s performance.
  • 3.2.6 Visual Hook: Visual hooks are short, single-shot ads designed to be interpreted together with the accompanying social-media post.A Warby Parker product shot alone makes the brand and announcement difficult to identify, while the caption supplies that context.
  • 3.2.7 Monologue; 3.2.8 Interview: Monologue ads feature a single person speaking with higher production quality, whereas interview ads use question-and-answer exchanges between people.The Dove example intercuts direct-to-camera advocacy with cinematic B-roll; the Sephora example alternates beauty instruction and basketball instruction.

4 Study 1b: Characterizing Cinematic Ads

Study 1b analyzed 99 cinematic ads to characterize their duration, shot structure, audio and text, and editing techniques. The resulting measurements informed the generation pipeline and showed that social-media cinematic ads are shorter and faster-paced than television ads and films.

  • 4 Study 1b: Characterizing Cinematic Ads: Study 1b analyzed duration, shot count, audio and text elements, and editing techniques in a corpus of 99 cinematic ads.The corpus combined 36 ads from Study 1a with 63 newly collected cinematic ads.
  • 4.1.2 Analysis: A 22-item codebook covered shots, five audio and text elements, and 16 editing techniques, with reliability checks using intraclass correlation and Cohen’s Kappa.Coders independently assessed sampled data, discussed disagreements, refined definitions, and then completed the corpus coding.
  • 4.2 Findings: 17.9 seconds, 15.2 shots, and 1.2 seconds were the average total duration, shot count, and shot duration, respectively.These averages describe the cinematic ads in the corpus.
  • 4.2.1 Duration, Shot Count, and Shot Duration: Compared with approximately 30-second television ads and approximately four-second film shots, cinematic ads averaged 17.9 seconds overall and 1.2 seconds per shot.The comparison indicates a faster pace in the social-media cinematic-ad corpus.
  • 4.2.2 Editing Techniques: The most common editing techniques were cut on action at 52.2%, montage at 37.4%, cut to beat at 22.2%, movement match cut at 18.2%, and overlay-imagery alignment at 16.2%.These percentages report technique presence in the corpus.
  • 4.2.2 Editing Techniques: 44.4% of ads contained text overlays, and 36.4% of those ads used overlay-imagery alignment to match text with concurrent imagery.This technique reflects advertising-specific coordination between written messaging and visuals.
  • 4.2.2 Editing Techniques: The Aperol example pairs “JOIN THE MUSIC,” “JOIN THE SUNSET,” and “JOIN THE SUMMER” with festival, sunset, and toast imagery over upbeat audio.The sequence creates a vibrant summer atmosphere through coordinated text, imagery, montage, and sound.
  • 4.2.2 Editing Techniques: Using a conservative definition, montage required at least three consecutive shots sharing an underlying concept; 37.4% of ads contained one.The Aperol example strings together crowd, sunset, and glasses shots to evoke a cohesive summer atmosphere.

5 Generating Cinematic Ads

The paper generates cinematic ads through a two-stage pipeline: users provide brand inputs, an LLM creates a structured shot plan, and a video-generation model renders the ad. The plan encodes cinematic conventions, editing techniques, pacing, music, shot sequence, and transitions.

  • 5 Generating Cinematic Ads: The pipeline accepts brand description, color palette, logo, and style-reference images before generating a shot plan and rendered cinematic ad.The LLM planner creates the plan, while the video-generation model produces the video from it.
  • 5 Generating Cinematic Ads: The two-step workflow mirrors professional production by separating shot-list or storyboard planning from footage production.The structure is also used in recent AI video-generation systems.
  • 5.1.2 System Prompt: The LLM is prompted to act as a creative director and describe cinematic-ad characteristics, shot types, camera movements, editing techniques, and pacing.The prompt explicitly distinguishes cinematic ads from problem-solving narratives, product demos, UGC, and slideshows.
  • 5.1.2 System Prompt: The editing-technique list comes from Study 1b and includes montage, match cut, overlay-imagery alignment, cut on action, and cut to beat.A technique was included if it appeared in any Study 1b video.
  • 5.1.2 System Prompt: The prompt specifies an average shot duration of 1.2 seconds and per-shot durations between 0.5 and 7.5 seconds.These pacing parameters were observed in Study 1b.
  • 5.1.3 Shot Plan: Each shot plan specifies music, shot sequence, and transitions, illustrated by an Under Armour-inspired plan with a percussive heartbeat-like beat and three opening shots.The example coordinates narration, captions, camera framing, athlete imagery, blinking, and a beat-synchronized cut.
  • 5.2 Generating Video: The video-generation model receives the shot plan and brand logo but not style-reference images, because those images caused overly close reproduction of existing advertisements.The stated goal was to preserve original cinematic ads with original editing.
  • 5.2 Generating Video: The resulting ads could appear realistic, coherent, and cinematic, but those high-level impressions did not identify which editing choices worked or needed improvement.This motivated Study 2’s professional-editor evaluation.

6 Study 2: Evaluating AI-Generated Cinematic Ads

Study 2 asks how professional video editors evaluate the editing quality of AI-generated cinematic ads. It uses professional critique as the basis for examining editing quality.

  • 6 Study 2: Evaluating AI-Generated Cinematic Ads: Study 2 recruited professional video editors to critique AI-generated cinematic ads.The study frames professional critique as the approach to evaluating editing quality.

6.1 Methodology

The study generated cinematic ads for real brands with a two-stage LLM–video-generation pipeline, then collected and coded professional editors’ critiques of their editing quality.

  • Data Collection: 35 real brands supplied descriptions, logos, color palettes, and style-reference images for ad generation.The real-brand setting allowed editors to assess whether ads fit recognizable brand images.
  • Editor Evaluation: Six professional editors with five to thirteen years of experience evaluated assigned ads through four open-ended questions.They discussed effective choices, weaknesses, alternative edits, and additional comments beyond editing.
  • Editor Evaluation: A separate fourth question helped distinguish editing-quality critiques from comments about AI-generation defects.The design preserved generation-defect observations when those defects affected the viewing experience.
  • Interview Procedure: Researchers conducted one-hour interviews after the written critiques to elaborate and disambiguate editors’ responses.The interviews revisited ads individually, including audio effects or visual moments that were difficult to interpret from text alone.
  • Analysis: The analysis segmented 870 sentences, developed a codebook through open coding, and refined it until Cohen’s Kappa exceeded 0.7.Two coders independently coded randomly selected 25% subsets during codebook refinement before one coder coded the remainder.

6.2 Dimensions of Editing Quality in Cinematic Ads

Professional editors evaluated cinematic ad quality across six dimensions, emphasizing coherent narrative development, coordinated audiovisual design, polished visuals, continuity, and effective pacing.

  • Narrative Progression: Narrative progression concerns whether the story develops coherently from beginning to end, including ordering, openings, and closings.Five participants discussed it in 139 sentences across 46 videos.
  • Narrative Progression: Narrative order should make actions understandable, as Toyota and Aperol examples required reversing shots to produce a coherent timeline.Editors preferred showing the woman approach before the key turns and preparing drinks before serving them.
  • Narrative Progression: Editors valued establishing shots and attention-grabbing openings, such as a push-in paired with voiceover and on-screen text.A Warby Parker-inspired ad used a showroom-to-product sequence, while a Dove-inspired opening built curiosity and pulled attention immediately.
  • Audiovisual Coordination and Sound Design: Audiovisual coordination and sound design concern whether sound, music, silence, and visual editing support rhythm, meaning, and emotional effect.All six participants discussed this dimension in 135 sentences across 51 videos.
  • Audiovisual Coordination and Sound Design: Effective coordination included beat-synchronized cuts, sound-action matching, sound bridges, musical progression, variation, and purposeful silence.Editors praised a bass-drop cut, product and movement sound effects, overlapping breath with dissolves, and silence that focused attention.
  • Audiovisual Coordination and Sound Design: Sound choices could also weaken an ad when music stayed flat, stopped abruptly, or omitted an expected effect.Editors cited missing crescendos, an ill-fitting music cutoff, absent basketball swoosh effects, and confusingly silent breathing.
  • Visual Composition and Graphics: Visual composition and graphics concern whether framing, captions, color, and scene variation guide attention and make ads feel polished.All six participants discussed this dimension in 114 sentences across 41 videos.
  • Visual Composition and Graphics: Editors recommended keeping important subjects within device-safe margins, maintaining eye trace, and preserving focal clarity across shots.Examples included widening a drone shot, repositioning subjects and a bag, and matching pupil positions between eye close-ups.

7 Discussion

The discussion connects the six-dimensional framework to editing-aware generation, systematic human evaluation, automated evaluation, and established film and marketing theories. It also cautions that the findings reflect one specific generation pipeline, while the dimensions provide an initial evaluation framework.

  • Improving AI-Generated Cinematic Ads: Editing-aware planning could critique and revise shot plans across the six dimensions before video rendering.The plan could preserve motion continuity, establish a core product message, and assess whether a montage supports that message.
  • Human Evaluation: Rubrics based on the six dimensions could make human evaluation more fine-grained and diagnostic than a single overall score.Two videos with similar overall scores could fail for different reasons.
  • Automated Evaluation: Operational definitions and examples from the framework could train human raters and help autoraters align their ratings with human judgments.Researchers can compare autorater and human scores through correlations, while system prompts can encode the framework’s definitions.
  • Connections to Existing Literature: Several dimensions correspond to established film-editing principles, but the framework transfers those principles to cinematic advertisements.The authors identify continuity, rhythm, and audiovisual coordination as connections to film theories.
  • Connections to Existing Literature: The framework extends film-oriented evaluation by adding message and brand coherence and treating montage coherence as a form of narrative progression.These additions reflect cinematic ads’ commercial purpose and lack of a problem–resolution structure.
  • Limitations: The findings are scoped to ads generated through LLM-based shot planning followed by rendering with a selected video-generation model.Other pipelines, including direct generation from a single prompt, may produce different strengths and weaknesses.

8 Conclusion

The conclusion summarizes the paper’s studies of social-media ad formats and cinematic-ad characteristics, then frames the six-dimensional framework as the basis for evaluating AI-generated cinematic ads.

  • 8 Conclusion: Study 1a found cinematic ads to be a common social-media advertising format, while Study 1b analyzed their duration, shot structure, audio, text, and editing techniques.The paper reports that cinematic ads tended to be shorter than typical television ads and had shorter shot durations than movies.

Third-Party Content and Trademark Notice

The notices clarify that third-party advertisements and brand materials remain the property of their rights holders and are used for scholarly purposes.

  • Third-Party Content and Trademark Notice: Referenced advertisements, brands, trademarks, logos, and selected frames are used for identification, scholarly analysis, and criticism without claimed ownership or affiliation.The authors state that the discussed brands neither sponsor nor endorse the paper.
  • Third-Party Content and Trademark Notice: Study 2’s AI-generated videos are research stimuli, not official advertisements, and were created solely for noncommercial research and evaluation.Brand names and logos in these materials do not imply affiliation or endorsement.
Loading 2608.24329v1…