PaperlayerView exact revision on arXiv

Source-linked AI summary

VideoGen Model 1: Prompt Expansion Versus Formatting in Audiovisual Generation

David Grossman, Anton Koenig

arXiv:2610.29407v1cs.CV

TL;DR

Video prompts can leave important creative choices unspecified, including scene progression and dialogue. This study compares instruction-based expansion with formatting alone, finding overall gains over formatting under VA-Judger while the Gemini change remains uncertain.

  • Problem

    Video prompts can leave important creative choices unspecified, including what happens, how a scene progresses, and what characters say.

    Source: §2 · p. 1

  • Method

    The study compares raw, format-only, and expanded prompts, with format-only organizing inputs into H3 fields and expansion adding creative details.

    Source: §1 · p. 1, §3 · p. 4

  • Results

    Expansion increased overall scores over format-only prompts by 0.63 points for VA-Judger (95% interval: [+0.11, +1.15]) and 0.15 for Gemini (95% interval: [−0.03, +0.35]).

    Source: §4 · p. 5

  • Takeaways & Limitations

    Prompt-alignment scores increased by 0.97 points under VA-Judger and 0.50 under Gemini, while VA-Judger also showed an audio-quality gain of 1.00 point.

    Source: §5 · p. 6

    The controlled pilot used 30 authored prompts and one generation seed, leaving generalization to new prompts, seeds, and longer videos untested.

    Source: §5 · p. 6

Abstract

from arXiv · show

Creative prompt expansion improved overall audiovisual quality scores over raw prompts in a controlled pilot of VideoGen Model 1, our post-trained version of MiniMax H3. We compared the raw, reformatted, and expanded versions of 30 prompts, holding the generator, seed, and sampling settings fixed. Each clip was rated separately on five dimensions. Their mean formed an overall score on a 1-10 scale. Compared with formatting alone, overall scores increased by 0.63 points (95% interval: [+0.11, +1.15]) under VA-Judger and 0.15 points (95% interval: [-0.03, +0.35]) under Gemini 3.8 Flash, whose overall change remained uncertain. Prompt-alignment scores increased by 0.97 points under VA-Judger and 0.50 under Gemini. VA-Judger also showed higher audio quality; visual-quality ratings changed little under either judge.

1 Introduction

This study tests whether creative prompt expansion adds value beyond formatting prompts for H3. It distinguishes that inference-time approach from earlier learned prompt rewriting.

1 Introduction

  • Formatting organizes an original prompt into H3’s visual and audio fields, while expansion adds creative details about setting, action, camera movement, performance, and sound.Source: §1 · p. 1
  • The study tests the added value of creative expansion over a formatting control using the same VideoGen Model 1 checkpoint in all three conditions.MiniMaxAI’s H3 documentation recommends refining inputs before generation.Source: §1 · p. 1
  • Unlike VPO’s supervised and preference-trained rewriting, this study evaluates an instruction-based rewriter at inference time and rates resulting clips with audio.VPO was introduced by Cheng et al. [2].Source: §1 · p. 1

2 Writing architecture

The writing architecture separates story development from the translation of material into H3-ready audiovisual prompts. Its workflow adapts to prompt detail and duration, with checks for clarity, structure, and continuity.

2 Writing architecture

  • The writers’ room develops and revises scripts, while a separate clip rewriter translates material into H3’s audiovisual prompt format.The room assigns writing, criticism, and reader roles to language models; Wu et al. [3] used hierarchical scene and shot planning in MovieAgent.Source: §2 · p. 1

2.1 Writers’ room

  • Open-ended requests can proceed through concept selection, story-arc development for videos over 15 seconds, and timed screenplay drafting, while detailed one-clip prompts skip story development.The workflow preserves requested shots, actions, lines, sounds, style, and order; reader and code checks address dialogue clarity and screenplay structure.Source: §2.1 · p. 2

2.2 Clip-level expansion

  • The clip rewriter optionally researches named real-world subjects, then writes ordered shots, action, dialogue, soundscape, and music in H3’s three prompt fields.Its instructions preserve supplied dialogue and on-screen text, prohibit invented dialogue and unrequested real brands, and retain requested framing and shot order.Source: §2.2 · pp. 2–3

2.3 Continuity across renders

  • For writers’ room screenplays, a director plans shots and a writer compiles clips into the same format, with checks for planned content and exact text.If two writing attempts fail, code assembles prompts from the plan; background music is composed after the clips are cut together.Source: §2.3 · pp. 3–4

3 Methods

The pilot compares raw, format-only, and expanded prompts across 30 authored prompts under fixed generation settings. Two judges rate individual clips, and paired prompt-level bootstrap intervals quantify changes.

3 Methods

  • Thirty prompts across ten scene categories produced 90 clips using one assigned size per prompt, seed 1001, 28 sampling steps, and the same rendering backend.Each clip lasted approximately 5.2 seconds, or 124 frames at 24 fps.Source: §3 · p. 4
  • Format-only prompts reorganized source text and added audio defaults, while expanded prompts used the rewriter and averaged 237 words versus 27 for format-only and 15 for raw prompts.Research was requested for four expanded prompts; expansion cost an average of $0.02 per prompt in model charges, excluding rendering and evaluation.Source: §3 · p. 4
  • VA-Judger and Gemini 3.8 Flash rated all 90 clips separately on five 1–10 dimensions, whose equal-weight mean formed the overall score.Judges saw the original prompt but not the rewritten prompt or condition label; VA-Judger used 2 fps sampling, while Gemini used default video processing.Source: §3 · p. 4
  • Paired mean changes used all 30 prompts and descriptive 95% bootstrap intervals, which were unadjusted for multiple comparisons and did not capture variation across seeds, writers, or evaluators.Because both judges rated the same clips, their results were not independent replications.Source: §3 · pp. 4–5

4 Results

Expansion increased overall scores over both formatting alone and raw prompts, though the Gemini interval for the primary comparison included zero. Formatting alone also increased scores over raw prompts under Gemini.

4 Results

  • Compared with raw prompts, expansion increased overall scores by 0.70 points for VA-Judger (95% interval: [+0.22, +1.18]) and 0.47 for Gemini (95% interval: [+0.21, +0.74]).Source: §4 · p. 5
  • Scores increased by 0.07 for VA-Judger (95% interval: [−0.51, +0.62]) and 0.32 for Gemini (95% interval: [+0.09, +0.56]); Figure 2 shows these paired mean changes.Source: §4 · p. 5
  • Table 1 reports paired mean differences between expanded and format-only prompts for each 1–10 dimension, using all 30 prompts.Positive values favor expansion; brackets give descriptive 95% paired-bootstrap intervals.Source: Table 1 · p. 5
  • Expansion raised overall scores over format-only prompts by 0.63 points for VA-Judger (95% interval: [+0.11, +1.15]) and 0.15 for Gemini (95% interval: [−0.03, +0.35]).Figure 2 shows paired mean changes across all 30 prompts; Gemini’s interval included zero.Source: Figure 2 · p. 5, §4 · p. 5

5 Discussion

Beyond formatting, expanded prompts improved prompt alignment under both judges and audio quality under VA-Judger, while visual-quality ratings changed little. Audio gains have multiple plausible explanations, and the pilot does not establish generalization beyond its tested prompts, seed, and short clips.

5 Discussion

  • +0.97 points for VA-Judger and +0.50 for Gemini in prompt alignment; VA-Judger audio quality rose 1.00 point, versus a 0.07-point video-quality change.Visual-quality ratings changed little under either judge.Source: §5 · p. 6
  • Expansion specified background music in 22 prompts versus one in the formatting control, so the experiment cannot isolate music’s contribution from sound design or coordination with visible action.Audiovisual-consistency and completeness/coherence intervals included or touched zero under both judges, so improved synchronization or story coherence is not established as the cause of score changes.Source: §5 · p. 6
  • The pilot used 30 authored prompts and one generation seed on short clips; generalization to new prompts, seeds, and longer videos remains untested.The experiment evaluated the combined effect of added content, greater prompt length, and occasional research; evaluator behavior may also contribute to different overall changes.Source: §5 · p. 6

AI assistance disclosure

OpenAI Codex assisted with manuscript drafting and revision, analysis code, and figure preparation; the authors take responsibility for the final content.

AI assistance disclosure

Loading 2610.29407v1…