Building a Precise Video Language with Human-AI Oversight

Precise specification

• The specification formalizes subjects, scenes, motion, spatial framing, and camera dynamics with hundreds of visual and motion primitives developed alongside professional video creators.Training guidelines improve caption consistency and coverage in human evaluations compared with datasets lacking structured annotation policies.Source: §1 · p. 1

Oversight framework

• CHAI shifts human effort from writing to verification: models produce high-recall pre-captions, while experts critique and revise them into improved post-captions.A user study reports improved annotation accuracy, writing quality, and efficiency over manual caption editing.Source: §1 · p. 2

Post-training strategies

• The resulting preference pairs and (pre-caption, critique, post-caption) triplets improve SFT, DPO, and RLHF-V, while critique quality in precision, recall, and constructiveness governs post-training success.The oversight framework enforces these critique properties by design and produces more accurate, complete, and constructive feedback than prior collections such as OpenAI’s GDC [53] [88].Source: §1 · pp. 2–3, Figure 1 · p. 2

Benchmarks and findings

• Off-the-shelf video-language models capture subject appearance and scene context but struggle with fine-grained subject motion and camera dynamics.These limitations motivate a unified benchmark for precise video-language understanding covering caption, reward-modeling, and critique-generation capabilities.Source: §1 · p. 2
• Using standard SFT and RL methods, open-source Qwen3-VL models outperform closed-source Gemini-3.1 with modest expert supervision.The benchmark uses approximately 4,000 videos spanning films, games, commercials, and user-generated clips, jointly evaluating caption generation, reward modeling, and critique generation.Source: §1 · pp. 2–3

Application: improving text-to-video generation

• Re-captioning professional films, commercials, music videos, and games enables Wan [60] to follow prompts up to 400 words with finer control over motion, composition, and cinematography.The application targets professional-grade text-to-video generation through precise language control.Source: §1 · p. 3

Video–text datasets

• Video–text datasets have progressed from short, loosely aligned descriptions [26] to longer, detailed captions, but often contain incomplete coverage, visual hallucinations, and writing problems.Human evaluations attribute these issues largely to inadequate specification and oversight, while many foundation models rely on unverifiable closed-source datasets [3] [11] [76].Source: §2 · p. 3

Scalable oversight

• Scalable oversight [5] [53] studies human–AI collaboration, using models to detect errors and help humans supervise complex tasks such as question answering, summarization, and code review [43].Most video–text datasets instead use human-written [26] [61] or model-generated captions [12] [73] without cooperation.Source: §2 · p. 3
• The proposed adaptation lets models generate fluent video captions while humans focus on visual details, assigning each collaborator the aspects they handle best.This addresses the absence of cooperation in prevailing video-caption curation practices.Source: §2 · p. 3

Post-training strategies

• Post-training methods align models with preferences through supervised fine-tuning, reinforcement learning, or inference-time scaling [6] [10] [22] [34] [37] [45] [50] [51] [80], while multimodal work often uses synthetic or image-only feedback [1] [38] [83].The paper claims to be the first to show that high-quality feedback effectively supervises video understanding.Source: §2 · p. 3

Motivation

• Prior datasets commonly lack detailed annotation policies, causing imprecise terminology, missing visual information, and subjective descriptions that distract from observable content.Examples include confusing camera translation with zoom, omitting camera shake or focus changes, and using emotional judgments such as “inspiring.”Source: §3 · p. 3

Building specification with content creators

• Over a year, more than 100 professional creators helped develop an objective framework covering subjects, scenes, motion, spatial relations, and camera dynamics.The framework captures both visible elements and how they change over time while excluding subjective impressions and emotional interpretations.Source: §3 · p. 4, Figure 2 · p. 4

Primitives for precise specification

• CameraBench-Pro defines over 200 visual and motion primitives with rigorous definitions, examples, corner cases, and decision rules for precise and temporally aware labeling.The taxonomy includes approximately 50 camera-motion, 100 camera-setup, and 70 cinematography primitives, with static properties labeled at shot start and end.Source: §3 · pp. 4–5

Structured captioning policy

• Structured captioning guidelines connect primitives into coherent descriptions of temporal changes, transitions, focus shifts, and cases where subjects are absent or evolve over time.Examples include describing a bird’s-eye-to-level transition through camera tilt and a foreground-to-midground rack focus.Source: §3 · p. 5

Motivation and limitations

• Prior human- and model-written captions suffer from poor writing, temporal or reference errors, visual hallucinations, and missed spatial or motion details.Examples include confusing left versus right and incorrectly describing a moving camera as static.Source: §4 · p. 5, Figure 3 · p. 6

Human–AI collaboration for data curation

• The workflow improves caption accuracy, completeness, and fluency while reducing the cognitive burden of writing detailed 200–400-word descriptions from scratch.Models follow comprehensive guidelines and labeled primitives, while humans concentrate on verification rather than generation.Source: §4 · p. 5
• CHAI has humans label visual primitives, models draft captions, humans critique errors and omissions, and models refine captions until experts confirm accuracy.The workflow separates generation from verification so models handle fluent writing while humans focus on visual fact-checking.Source: §4 · p. 5, Figure 3 · p. 6

Screening, training, and incentives

• Reviewers check every critique and post-caption, while bonuses reward error-free annotations and valid corrections to motivate precise quality control.Promoted reviewers correct mistakes, annotators can inspect corrections, and disagreements can be appealed to managers.Source: Figure 3 · p. 6, §4 · p. 6
• Only the top 3% of more than 600 applicants are selected after six primitive-based exams, followed by paid training and shadowing on 100 gold-standard captions.Applicants are recruited from content-creation backgrounds, and high performers advance to captioning roles after developing primitive-level discrimination skills.Source: §4 · p. 6

Benchmarks and tasks

• The benchmark contains approximately 20k triplets, reserves 5k for evaluation, covers subject, scene, motion, spatial, and camera aspects, and uniquely evaluates captioning, reward modeling, and critique generation.Its scale substantially exceeds recent public benchmarks, which typically contain approximately 1k examples.Source: §5 · p. 6

Findings

• The evaluation reports BLEU-4 against a reference caption for captioning, binary accuracy with chance = 0.5 for reward modeling, and BLEU-4 against a reference critique for critique generation.ROUGE and a “critique-guided revision” proxy metric are included in Table 9 for critique generation.Source: §5 · p. 7

Post-training

• The triplets support SFT, DPO, and RLHF-V through preferred post-captions and rejected pre-captions, while explicit critique generation and preference-label training enable self-critique and reward scoring.At inference, the model’s probability of “Yes” serves as a reward score and is more reliable than prompted Likert-scale ratings.Source: §5 · p. 7

Findings

• Our fine-tuned model achieves state-of-the-art performance across captioning, reward modeling, and critique generation, outperforming open- and closed-source baselines including Gemini-2.5/3.1.Adding explicit preference and critique signals improves both supervised fine-tuning and reinforcement-learning methods.Source: Table 1 · p. 8

Critique quality

• Critiques must be accurate, complete, and constructive; weakening any property substantially reduces performance, whereas second-stage quality control improves results.Gemini-2.5-Pro critiques perform poorly even with video access, underscoring that critique quality—not merely quantity—is essential for scalable oversight.Source: Figure 4 · p. 7, §5 · p. 8

6. Improving Professional Video Generation

• The pipeline re-captioned about 150K manually collected videos spanning films, commercials, music videos, and game footage under standard non-commercial licenses.These captions were used to fine-tune Wan2.2 for better prompt following.Source: §6 · p. 8
• Weakening critique precision, recall, or constructiveness substantially reduced post-training performance, while second-stage quality control noticeably improved results.The analysis compared inaccurate, incomplete, and unhelpful critiques with critiques from Gemini-2.5-Pro, blind Gemini-2.5-Pro, and annotators before quality control.Source: Table 2 · p. 9
• Wan2.2 fine-tuned on Qwen3-VL re-captioned videos better followed prompts up to 400 words and controlled cinematography, including dolly zooms and isometric views.Figure 5 reports improvements over zero-shot Wan and Wan fine-tuned using zero-shot Qwen3-VL captions.Source: §6 · p. 8, Figure 5 · p. 9
• Evaluations on 200 samples significantly improved prompt following over Wan2.2 fine-tuned with zero-shot Qwen3-VL captions across camera, focus, speed, perspective, overlays, framing, and subject transitions.Examples include rolling, Dutch angles, rack focus, speed ramps, side views, watermarks, height transitions, shot-size changes, and revealing shots.Source: §6 · p. 9

Limitations and future work

• CHAI enables efficient high-quality data curation through human–AI collaboration, even with modest academic resources.Future work could scale the process using stronger critique models to assist human annotators [5] [53].Source: §7 · p. 9
• Building benchmarks for video generation is a natural next step beyond the paper’s focus on video understanding.Source: §7 · p. 9

Summary

• The work combines clear specification, scalable oversight, and effective post-training into an open recipe, dataset, and benchmark suite for precise video captioning.It aims to advance scalable human–AI collaboration for high-quality data curation in video understanding and generation.Source: §7 · p. 9

Outline

• Sections A–C analyze prior annotation errors and document the specification and complete human-AI oversight pipeline.Coverage includes creator demographics, captioning policy, annotator screening and training, platform design, and promotion criteria.Source: Outline · p. 14
• Section D provides additional pre-caption, critique, and post-caption triplets from the dataset.Source: Outline · p. 14
• Section E describes LLM-generated negative critiques that violate precision, recall, or constructiveness and examines non-constructive critiques in prior work.Source: Outline · p. 14
• Section F reports metric and reward-format ablations, SFT dataset preparation, adversarial negative critique generation, and training hyperparameters.Source: Outline · p. 14
• Section G presents inference-time scaling results, while Section H details the web-scraped video set and text-to-video fine-tuning experiments.Source: Outline · p. 14
• Section I provides complete captioning guidelines for human annotators.Source: Outline · p. 14
• Section J gives Python-style pseudocode for converting labeled primitives into model instructions for generating pre-captions.Source: Outline · p. 14

• Across eight datasets, missing specification and oversight produce captions that are inconsistent, incomplete, inaccurate, or poorly written.Observed failures include imprecise terminology, omitted motion or spatial details, subjective interpretations, temporal-order errors, hallucinations, and confusion of subtle visual details.Source: A. Error Analysis of Prior Datasets · p. 14
• The evaluation uses a common Likert-scale rubric across five caption aspects, but scores are illustrative rather than directly comparable because datasets differ in video distributions, detail levels, and covered aspects.The study manually evaluates more than 20 random samples per dataset and marks frequent issues occurring in over 15% of inspected samples.Source: A. Error Analysis of Prior Datasets · p. 14, Table 3 · p. 15
• Crowdsourced captions are typically about 100 words versus about 400 words under structured specification and oversight, and often omit key details or misuse cinematic and motion terminology.Examples include confusing a high viewpoint with a bird’s-eye view, a full shot with a close-up, or a dolly zoom with a warping effect.Source: A. Error Analysis of Prior Datasets · p. 15, Figure 6 · p. 15
• Human evaluation finds recurring weaknesses in prior datasets, including omitted camera motion and scene context, spatial and subject hallucinations, subjective language, inconsistent coverage, and grammar errors.These problems occur in both automatically generated captions and human-refined or crowdsourced captions, though their specific patterns vary by dataset.Source: A. Error Analysis of Prior Datasets · pp. 20–22
• TUNA-Bench provides the highest overall human caption quality among the evaluated benchmarks and is recommended for evaluation.Its quality is attributed to comprehensive internal specifications and quality control, although its guidelines are not public and minor writing errors remain.Source: A. Error Analysis of Prior Datasets · pp. 15, 22
• Captions describe subject appearance, scene context, and general motion or activity strongly, highlighting the value of detailed specification and quality control.Source: A. Error Analysis of Prior Datasets · p. 23
• Annotators frequently misinterpret camera movement, confuse hard cuts with zooming, and omit spatial depth or precise camera-axis and orbital distinctions.Examples include labeling a midground car as background and describing rotation without specifying whether it arcs around a subject or follows the camera’s own axis.Source: A. Error Analysis of Prior Datasets · p. 23
• Captions often omit overlay elements such as subtitles, while minor hallucinations occur relatively infrequently despite the dataset’s detailed descriptions.Reported hallucinations include incorrect subject position, texture, and person counts.Source: A. Error Analysis of Prior Datasets · p. 23

Creator collaboration

• A core team of 30 creators and over 100 additional trained creators refine the taxonomy, resolve ambiguities, verify definitions, and support annotation.The broader creator pool was selected from more than 600 applicants, and annotators were drawn from this trained group for visual literacy and specification familiarity.Source: B. Specification Details · p. 23

Specification overview

• The specification organizes descriptions into five major aspects and their corresponding subaspects.Figure 18 provides the high-level overview of this organization.Source: B. Specification Details · p. 23

From primitives to captioning

• Approximately 200 labeled primitives are converted into pre-captions with dynamically generated prompts tailored to each video’s specific primitives.Prompts vary by content, directing models to describe subjects and relationships in multi-person scenes or environments and absent main subjects in scenery shots.Source: B. Specification Details · p. 23
• The resulting pre-captions cover subject, scene, motion, spatial, and camera information, with the conversion procedure documented in Python-style pseudocode.The primitives are drawn from CameraBench-Pro.Source: B. Specification Details · p. 23

C. Oversight Details

• The oversight team comprises 30 core creators, roughly 100 screened annotators, ∼20 captioners, and ∼8 reviewers drawn from more than 600 applicants.Professionals are screened through primitive-labeling exams requiring scores above 90%, while captioning promotion follows more than 300 videos with a strong track record.Source: C. Oversight Details · p. 23
• The in-house platform supports pre-caption critique, post-caption polishing, expert shadowing, reviewer approval or rejection, and appeals through regrade requests.Figure 19 shows the interface and training workflow, while the human–AI team structure is summarized in Figure 20.Source: C. Oversight Details · pp. 23–24, Figure 19 · p. 25, Figure 20 · p. 26
• 95% of samples reach accurate post-captions after one critique-revision cycle, while spatial captions require 1.4 iterations and score 3.3/5 initially.Across five aspects, only 5% require a second reviewer pass; spatial difficulty reflects reference-frame disambiguation challenges.Source: C. Oversight Details · p. 25, Table 4 · p. 26
• CHAI matches write-from-scratch captioning at 15 minutes per video but produces ∼3.7× longer captions, averaging 405 versus 110 words, with substantially higher accuracy.The workflow reallocates effort from writing to verification rather than eliminating human labor.Source: C. Oversight Details · p. 25
• Upfront training is the main scaling bottleneck: new annotators typically need 1.5–3 months of paid training to reach ≥90% accuracy, limiting the initial release to ∼4k videos.The paper identifies stronger critique models and cross-task skill transfer as possible ways to reduce this overhead.Source: C. Oversight Details · p. 26

D. Examples of Pre-captions, Critiques, and Post-Captions

• Figures 21–26 provide examples of pre-captions, critiques, and post-captions spanning word clouds, subjects, scenes, motion, spatial relations, and camera descriptions.These examples cover each captioning aspect used by the oversight pipeline.Source: D. Examples of Pre-captions,… · p. 26, Figure 21 · p. 27, Figure 22 · p. 28

E. Critique Quality Matters

• CHAI critiques must be accurate, complete, and constructive: they must avoid hallucinations, identify every error, and explain how to fix each one.The workflow enforces these properties by requiring critiques to directly guide the model toward the final post-caption.Source: E. Critique Quality Matters · p. 33

E.1. Prompts for Controlled Critique Error Injection

• Controlled injection uses caption instructions, pre-captions, and applicable human feedback to generate three degraded critique variants with LLMs.The framework isolates each critique property by injecting one error type at a time.Source: §E.1 · p. 33

E.1.1. Inaccurate Critiques

• Inaccurate critiques introduce errors by inserting spurious details or replacing correct feedback with plausible but incorrect alternatives.Insertion errors add irrelevant or fabricated points, whereas replacement errors swap a correct detail for a wrong one while preserving the feedback’s structure and tone.Source: §E.1.1 · p. 33

E.1.2. Incomplete Critiques

• Incomplete critiques remove one necessary correction from the ground-truth feedback, while preserving the no-edit response when the original critique is too short or already accurate.Deletion targets one key detail, suggestion, or explanation and can prioritize elements conflicting with the caption.Source: §E.1.2 · p. 34

E.1.3. Non-Constructive Critiques

• Non-constructive critiques retain error identification but strip corrective guidance, converting suggestions into general statements about what the caption gets wrong or omits.If feedback only provides guidance or a corrected version, the prompt rewrites it as criticism without helpful solutions.Source: §E.1.3 · p. 34

E.2. Model-Generated Critiques

• Gemini-2.5-Pro critiques are evaluated in two settings: a video model receives the full video and pre-caption, while a blind model receives only the pre-caption.Unlike controlled error injection, neither model-generated critique prompt receives ground-truth feedback.Source: §E.2 · p. 34
• The video-conditioned prompt requires checking missing elements, inaccuracies, and unnecessary content against the instruction, then giving concise, specific, actionable improvements.It instructs the model to state when a caption is accurate and requires no edits, and to avoid feedback beyond the caption instruction.Source: §E.2 · pp. 34–35
• The blind prompt explicitly asks Gemini-2.5-Pro to imagine visual content and generate substantial critique points and improvement suggestions from that imagined evidence.This setting permits plausible visual hallucination because the model has no actual video access.Source: §E.2 · p. 35

• The proposed framework differs by requiring critiques to fix model outputs, naturally enforcing precision, recall, and constructiveness.Prior GDC and MM-RLHF annotators critique outputs without using their critiques to correct them.Source: §E.3 · p. 35
• More than half of critiques in prior datasets are non-constructive, exceeding 90% in MM-RLHF for responses requiring critique.Table 5 reports constructive versus non-constructive critique distributions across benchmark tasks.Source: §E.3 · p. 35, Table 5 · p. 36
• Non-constructive critiques comprise over 47% of GDC summarization critiques, 49% of GDC text-QA critiques, and 90% of MM-RLHF video-QA critiques.These estimates come from random test-set examples, including 1,728 GDC summarization samples, 1,851 GDC text-QA samples, and non-empty critiques from 58% of 8,751 MM-RLHF samples.Source: Table 6 · p. 37, Table 7 · p. 38, Table 8 · p. 39

F. Post-training details

• Captioning instructions separately target subjects, scenes, motion, spatial layout, and camera dynamics, specifying details such as chronology, framing, depth, camera movement, lens, and focus.These task-specific instructions are also used by reference-based evaluation to judge whether captions capture the relevant aspect.Source: F. Post-training details · p. 40
• Critique-Revision evaluates critique generation by using generated feedback to revise captions and comparing the revised captions with references, avoiding direct critique-style matching.The method accommodates critiques that differ widely in format and wording.Source: F. Post-training details · p. 41
• The SFT dataset uses eight formats spanning captioning, critique, reward scoring, revision, and critique-based revision, with newline stripping and balanced Yes/No reward examples.Negative responses for DPO and RLHF-V are constructed from pre-captions, adversarial critiques, flipped reward labels, or pre/post-caption pairs so each triplet supports comparable post-training methods.Source: F. Post-training details · p. 41
• Qwen3-VL-8B-Instruct uses full-parameter SFT with frozen vision components, while DPO and RLHF-V use LoRA under shared 3 × 10^-5 learning-rate, 2048-token, 8-FPS, and 128-frame settings.All methods train for three epochs; DPO uses β = 0.1, and RLHF-V follows defaults with β = 0.5 and γ = 5.Source: F. Post-training details · pp. 42–43
• 98.0% pairwise accuracy makes LLM-Judge-Instruct with GPT-4o the best-aligned metric, ahead of LLM-Judge-Direct at 95.3% and BLEU-4 at 95.4%.Task instructions improve alignment by weighting task-relevant details; BLEU-4 remains the reproducible classic metric, while LLM-Judge-Instruct is reported as a complementary signal.Source: F. Post-training details · p. 40, Figure 27 · p. 43
• Explicit preference and critique supervision improves standard SFT and RL methods, while the full-data post-training recipe outperforms all baselines across reference-based caption and critique metrics.Table 9 includes ROUGE-L, LLM-Judge-Instruct, and Critique-Revision scores.Source: F. Post-training details · p. 41, Table 9 · p. 43
• VQAScore substantially outperforms text-generated Likert scoring after SFT across all five aspects, so the main experiments use binary Yes/No prediction with a continuous P(Yes) score.The advantage is attributed to undiluted binary supervision and avoidance of imbalanced five-level human labels.Source: F. Post-training details · p. 42, Figure 28 · p. 44

G. Inference-Time Scaling

• Critique-Last gives the best overall reward-model performance among the tested inference-time scaling strategies, whereas Critique-First and Self-Critique-Conditioned underperform.The weaker methods may reflect insufficient Qwen3-VL training on their critique-augmented formats; Self-Consistency averages P(Yes) across multiple rollouts.Source: Table 10 · p. 45

G.1. Test-time Scaling for Reward Modeling

• The evaluation compares VQAScore, using the probability of Yes, with Self-Consistency, using the Yes-to-No ratio across five rollouts.The experiments assess both direct Yes/No generation and critique-augmented outputs.Source: §G.1 · p. 45
• Critique-Last marginally outperforms direct VQAScore, showing that inference-time critique can yield further reward-modeling gains.The comparison is reported in Table 10.Source: §G.1 · p. 45

G.2. Test-time Scaling for Caption Generation

• Generation cost counts caption, critique, and revision calls, with critique-based revisions costing 2× generation; reward scoring is counted separately and is cheaper through KV caching.For example, best-of-N caption costs N generation and N reward calls, while critique-based revision costs 1 + 2N generation and N reward calls.Source: §G.2 · pp. 45–46, Table 11 · p. 46
• Parallel methods generate independent candidates and use reward-model selection, whereas sequential methods repeatedly refine one dependent caption over N steps.Parallel computation scales with N because candidates can be generated simultaneously; sequential computation accumulates linearly across iterations.Source: §G.2 · pp. 45–46
• 25.1 BLEU-4 at N = 16 makes vanilla best-of-N caption the strongest strategy for N ≥4, outperforming sequential iterative approaches.Across N = 1 to 16, iterative critique-based revision improves only from 16.4 to 18.0, indicating diminishing returns from sequential refinement.Source: Table 11 · p. 46, §G.2 · p. 46
• The study excludes hybrid search, reward-guided early stopping, pruning, and beam-search-like candidate filtering.These extensions are left for future work rather than evaluated in the reported parallel and sequential comparisons.Source: §G.2 · pp. 46–47

H. Video Generation

• The curated corpus contains approximately 150K diverse, transition-filtered YouTube clips averaging 4.6 seconds, spanning professional, user-generated, documentary, gaming, and other genres.Annotators source high-quality channels, TransNetV2 filters potential hard cuts or soft transitions, and clips shorter than one second are discarded.Source: H. Video Generation · p. 47
• Five structured captions are merged into one temporally coherent caption because Wan2.2 accepts roughly 400 words, preserving unique details while removing redundancy and adding no unsupported information.The merge uses Spatial as the base, integrates Motion and Camera chronologically, and adds Subject and Scene details only when unique.Source: H. Video Generation · pp. 47–48
• SFT-Caption achieves the best prompt following among Base, ZS-Caption, and SFT-Caption Wan2.2 variants on 200 prompts, according to expert Likert ratings.The evaluation averages 1-to-5 scores from three expert annotators; qualitative examples cover diverse camera and cinematographic effects.Source: H. Video Generation · p. 48, Table 12 · p. 48

I.2. High-Level Captioning Policy

• Captioning policies require descriptions to reflect only visible content, remain concise and objective, and present important subjects and events in temporal order.They also require sufficient context for viewers who have not seen the video while ignoring audio information.Source: §I.2 · p. 55

I.3. Subject Description Policy

• Subject policies specify precise types, visual attributes, poses, relationships, prominence, and transitions, distinguishing primary subjects from secondary subjects and ordering them temporally or by importance.Complex scenarios require identifying focus shifts such as revealing, disappearing, or switching subjects and explaining whether subject or camera movement causes them.Source: §I.3 · pp. 55–56

I.4. Subject Motion and Dynamics Policy

• Motion and scene policies describe specific actions, interactions, group behavior, environments, overlays, viewpoints, and settings, using chronological order and precise visual terminology.Motion descriptions cover subject-object, subject-subject, and collective actions; scene descriptions establish context without excessive detail and distinguish overlays from the scene.Source: §I.4 · pp. 57–58, §I.5 · p. 58

I.5. Scene Composition and Dynamics Policy

• Scene descriptions prioritize setting, prominent elements, overlays, viewpoint, and environmental motion, using a decision flow that stops when transitions or relevant content make further description unnecessary.Descriptions distinguish natural, human-made, crowd, time-based, and movement-based changes, while ordering scene elements from overall setting to specific spatial details.Source: §I.5 · p. 59

I.6. Spatial Framing and Dynamics Policy

• Spatial framing records shot size, frame position, scene depth, camera-relative height, and transitions for subjects or scenery in temporal order.For complex scenes, annotators prioritize a clear main subject, otherwise describe subjects by prominence or provide a general overview without forcing an unmeaningful shot size.Source: §I.6 · pp. 59–61

I.7. Camera Framing and Dynamics Policy

• Camera descriptions cover playback speed, lens distortion, height, angle, Dutch angle, focus, movement, shaking, subject following, and precise motion types in temporal order.The policy favors concise descriptions when movement is fragmented or random, while distinguishing movement direction, speed, purpose, and changes in focus or camera configuration.Source: §I.7 · pp. 61–62

I.8. Policy for Grading the Caption and Feedback

• Caption feedback uses a five-point scale from completely correct to almost entirely wrong, with scores reflecting factual accuracy and the extent of required revisions.The rubric distinguishes minor corrections, substantial omissions or hallucinations, major rewrites, and complete irrelevance.Source: §I.8 · p. 63

I.9. Special Cases

• Special-case handling improves self-contained readability, regenerates captions with formatting errors, corrects mislabeled data, and separates ignored feedback from feedback that annotators themselves missed.Only factual mistakes are corrected for formatting issues, while Type-B feedback errors require restarting the feedback process.Source: §I.9 · p. 63

I.10. Common Mistakes to Avoid

• Annotators must keep descriptions visual, consistent across captioning tasks, accurate about depth and static shots, and free of unsupported overlay or commentary claims.Camera motion must align across subject, scene, spatial, and camera captions; a shot is static only when the camera, focus, and zoom remain unchanged.Source: §I.10 · p. 63

J. Pseudocode for Generating Pre-Captions from Labeled Primitives

• The pseudocode section implements caption-generation policies that dynamically construct prompts from labeled cinematography and camera-motion primitives.These implementations connect structured primitive labels to pre-caption generation for video annotation.Source: J. Pseudocode for Generating… · p. 64

J.1. Subject Description Policy

• The SubjectPolicy implementation uses existing shot-composition labels to prompt concise, informative structured captions for subjects.Its prompt generator exposes a subject_description caption field and derives annotation prompts from video shot-composition primitives.Source: §J.1 · p. 64
• J.1. Subject prompts require precise subject types, visual attributes, poses, orientations, relationships, and temporal ordering, while adapting detail to focus, transitions, ambiguity, and shot composition.They distinguish multiple subjects using appearance, location, action, or relationships; special cases handle shot transitions, scenery without a main subject, changing or atypical subject size, revealing or disappearing subjects, and varying focus.Source: §J.1 · pp. 64–66

J.2. Scene Description Policy

• J.2. Scene prompts describe viewpoint, indoor/outdoor setting, location, time, weather, architecture or nature, props, style, overlays, and visible natural or human-made motion without speculation.Captions integrate observations across the sequence into one fluent paragraph, focus on scenery rather than detailed subjects, and treat text, titles, subtitles, icons, watermarks, HUDs, and framing elements as overlays with content and placement.Source: §J.2 · p. 67
• The scene description policy uses overlay and point-of-view primitives, including objective, first-person, overhead, drone, broadcast, and game-view perspectives.Source: §J.2 · p. 68

J.3. Subject Motion & Dynamics Policy

• The subject-motion policy requests chronological, observable descriptions of individual, object-directed, interpersonal, and group actions while avoiding speculation and abstract judgments.It adapts instructions to scenery shots, multiple-subject configurations, subject transitions, unstable framing, atypical motion, and human- or non-human-focused shots.Source: §J.3 · pp. 68–69

J.4. Spatial Framing & Dynamics Policy

• The spatial-framing policy describes shot size, subjects’ 2D and 3D positions within the scene, and notable spatial movements or changes in shot size or spatial position.Source: §J.4 · p. 71
• J.4. Spatial Framing & Dynamics Policy branches prompts by subject and scene configuration, emphasizing salient subjects, scenery, transitions, focus, framing changes, and spatial relationships.The policy handles revealing, disappearing, switching, dynamic or atypical shot sizes, multiple-focus conditions, scenery, complex scenarios, and existing descriptions without inferring stated framing information.Source: §J.4 · pp. 72–73
• The spatial policy encodes shot size, camera height relative to subjects, subject descriptions, and scene descriptions through a broad primitive and attribute vocabulary.Its listed primitives include shot-size categories and changes, camera-height relations, subject-status conditions, and non-primitive fields such as subject_description, scene_description, Shot_Size_Description, and Subject_Height_Description.Source: §J.4 · pp. 73–74

J.5. Camera Framing & Dynamics Policy

• J.5. Camera Framing & Dynamics Policy uses existing labels to prompt structured captions covering camera framing and dynamics.The policy exposes a camera_framing_dynamics caption field and is implemented as a dedicated CameraPolicy prompt generator.Source: §J.5 · pp. 74–75
• The camera policy maps playback-speed labels to natural-language descriptions, distinguishing time-lapse, fast or slow motion, stop motion, speed ramping, reverse playback, and regular speed.Regular playback is explicitly marked as requiring no mention, while unknown labels receive an unknown-speed fallback.Source: §J.5 · p. 75
• The camera policy also formats lens distortion and starting camera height, including barrel or fisheye distortion and levels from aerial or overhead to underwater.The supplied implementation includes explicit natural-language mappings for camera-height categories and lens-distortion descriptions.Source: §J.5 · p. 75
• The policy instructs descriptions to cover playback speed, lens distortion, camera height and angle, focus and depth of field, and camera movement attributes.Movement attributes include direction, steadiness, speed, complexity, purpose, and temporal ordering; normal or unremarkable properties may be omitted.Source: §J.5 · p. 77
• Camera height and angle formatting represents fixed values or transitions, including aerial, overhead, eye, hip, ground, water, bird’s-eye, high, level, low, worm’s-eye, and Dutch angles.The implementation explicitly describes height changes, angle changes caused by motion, and fixed or varying Dutch angles.Source: §J.5 · pp. 77–78
• Focus descriptions distinguish deep, shallow, and ultra-shallow depth of field, identify focus planes, and report rack focus or focus tracking when applicable.The implementation can describe focus transitions between foreground, midground, background, and out-of-focus states.Source: §J.5 · pp. 78–79
• Movement descriptions distinguish static or shaking cameras from intentional motion and verbalize simple movements, tracking perspectives, subject-size changes, steadiness, and speed.Supported movements include panning, tilting, rolling, zooming, dollying, trucking, pedestaling, arcing, craning, and directional translation.Source: §J.5 · pp. 78–79
• The policy directly uses human-labeled ground truth and forbids inferring details that are not provided, while keeping descriptions brief.Regular playback speed and other normal or insignificant properties need not be mentioned.Source: §J.5 · p. 78
• The policy describes camera motion using movement patterns, tracking information, steadiness, and non-regular motion speed when applicable.It can distinguish minor from clear movement, append specific movement descriptions, and format steadiness and speed attributes.Source: §J.5 · pp. 79–80
• The policy requires camera descriptions to identify the focused or tracked subject and specify the concrete tracking type rather than using generic references.For example, it favors identifying the man in the foreground and describing the specific tracking operation.Source: §J.5 · p. 80
• The policy draws on primitives covering camera setup, motion direction and type, focus, angle, height, speed, distortion, and tracking, plus several non-primitive descriptions.The listed vocabulary includes focus changes, rack pull focus, tracking variants, pans, tilts, zooms, arcs, cranes, and height or angle changes.Source: §J.5 · p. 80