CineSubBench: Evaluating LLMs on Long-Form Narrative and Cultural Understanding from Multilingual Movie Subtitles

1 Introduction

• Film’s long stories unfold through dialogue and temporally separated events, so relationships, motivations, and the significance of early exchanges may become clear only later.Models must integrate evidence across the film to reconstruct an unfolding narrative.Source: §1 · p. 1
• Subtitles provide temporally ordered dialogue but omit speakers, scene boundaries, events, motivations, and causal relations, requiring models to reconstruct narrative structure from distributed evidence.Unlike screenplays, subtitle timestamps provide temporal order without identifying narrative units.Source: §1 · p. 2
• Evaluating the same films across subtitle languages and national classification systems provides a setting for examining linguistic and cultural consistency together.Subtitle-based evaluation also gives closed- and open-weight models a shared text interface without full-length video processing choices.Source: §1 · p. 2
• CineSubBench evaluates 1,012 films through six subtitle languages and ten national rating systems across seven narrative and cultural tasks.The matched corpus contains 6,072 tracks and 8.13M timestamped subtitle entries.Source: §1 · pp. 2–3
• Across nine LLMs, broad plot premises are recovered more reliably than event-complete synopses, while cross-lingual consistency, national calibration, and safety grounding vary.Strong profanity is grounded far better than mild obscenity; together, the patterns distinguish narrative integration, multilingual consistency, cultural prediction, and evidence grounding.Source: §1 · p. 3

Domain-grounded LLM evaluation

• Film remains less explored as an LLM evaluation domain than law and medicine (Guha et al., 2023; Singhal et al., 2023), finance (Cao et al., 2024; Guo & Yang, 2024), and other specialized domains.Those other domains include software engineering, science, education, journalism, cybersecurity, and geospatial reasoning.Source: §2 · p. 3

Long-context narrative and temporal understanding

• Long-context benchmarks study books and stories, but subtitle timestamps provide temporal order without explicit event boundaries, causal structure, or narrative importance.CineSubBench holds the underlying films fixed across tasks, subtitle languages, and national classification systems.Source: §2 · p. 3

Film, subtitle, and audiovisual understanding

• Prior film and television work addresses individual tasks such as screenplay summarization (Chen et al., 2022; Saxena & Keller, 2024) and US age-rating, rather than a broad matched LLM benchmark.CineSubBench spans narrative generation, genre, age suitability, multiple national ratings, multilingual subtitles, and subtitle-indexed safety evidence.Source: §2 · pp. 3–4
• Audiovisual benchmarks target visual grounding, multimodal reasoning, event understanding, scene interpretation, or cinematographic analysis rather than matched subtitle-based MultiX comparison.Publicly accessible subtitle tracks support matched comparisons across tasks, languages, national systems, and model families.Source: §2 · p. 4

3 CineSubBench

• The benchmark covers 1,012 films with 6,072 subtitle tracks and 8.13M timestamped entries, matched across six languages and ten national classification systems.Mean formatted tracks contain roughly 40K–45K tokens, and the longest exceed 100K; each entry retains its index and time interval for film-level understanding and line-level grounding.Source: §3 · p. 4

3.1 Narrative Understanding and Generation

• Plot generation targets a concise premise, synopsis generation requires spoiler-aware event selection and chronology, and key-message generation requires thematic abstraction.Genre prediction maps story, character, setting, tone, conflict, and themes to one or more labels in a normalized 22-label space.Source: §3.1 · p. 4

3.2 Cultural Prediction and Assessment

• Cultural tasks distinguish family-oriented age suitability from formal national ratings, which retain ten distinct ordered label spaces; safety assessment predicts lexical-category counts and supporting subtitle indices.Safety categories are strong profanity, crude bodily language, mild obscenity, and religious profanity or exclamation; evidence indices make missed or unsupported judgments inspectable.Source: §3.2 · pp. 4–5

3.3 Construction and Quality Assurance

• Complete-coverage optimization selects films shared across languages and countries, then subtitle checks and task-oriented curation produce the matched 1,012-film cohort.Subtitle candidates undergo structural, temporal, encoding, language-consistency, and runtime checks, with manual inspection or cross-provider recovery when needed.Source: §3.3 · pp. 5–6, Figure 2 · p. 5

4 Evaluation Setup

• Nine models are evaluated across closed frontier, closed baseline, and open-weight groups, with narrative and cultural predictions measured in six languages and narrative generations written in English.The same films are used for each subtitle language; language-safety assessment is English-only because its gold evidence is localized to the English track.Source: §4 · p. 6
• GPT-5.6 Luna serves as a common reference-based judge for plot, synopsis, and key-message generations, with 98.4% overall within-one-point agreement against an independent narrative judge.The same references, task-specific rubrics, and structured scoring protocol are applied across models.Source: §4 · p. 6

4.1 Evaluation Metrics

• Task-specific metrics include reference-based 1–5 narrative scores, genre micro-F1, age MAE and accuracy, and country-rating exact match and ordinal MAE.Language-safety category-agnostic evidence F1 measures index recovery, while strict F1 also requires the correct lexical category; the composite is a compact summary and subsequent analyses retain original task metrics.Source: §4.1 · p. 6
• Narrative, genre, age, and country metrics are averaged equally across six subtitle languages, whereas strict language-safety F1 is English-only.The age-label span from 2+ to 18+ is 16.Source: §4.1 · p. 6

5.1 RQ1: Does a Single Rank Reflect Uniform Capability?

• Gemini 3.8 Flash leads the composite score at 78.04, but Sol leads synopsis at 3.40 while Gemini leads plot, key message, genre, country ratings, and strict safety F1.DeepSeek V4 Pro ranks third overall but exceeds Sol on genre and age accuracy while trailing on narrative, country ratings, and safety evidence F1.Source: Table 1 · p. 7, §5.1 · p. 7

5.2 RQ2: How Robust Is Understanding Across Subtitle Languages?

• Narrative scores fall from English to the five-language mean for all nine models, with declines ranging from 0.05 points for GPT-5.6 Sol to 0.28 for Llama 4 Scout; Persian is the lowest narrative setting for every model.Persian also yields the lowest genre score for eight of nine models and lowest country-rating exact match for every model; with films and references held constant, these comparisons measure end-to-end cross-language consistency.Source: §5.2 · pp. 7–8, Figure 3 · p. 7

5.3 RQ3: Where Does Cultural Calibration Fail?

• Sol overpredicts age more often than underpredicts it (48.3% versus 20.6%), whereas Gemini shows the reverse (15.2% versus 50.6%), despite similar within-one-year accuracy.The age-error comparison spans six subtitle languages.Source: Figure 4 · p. 8, §5.3 · p. 8
• France averages 31.6% country-rating exact match against a 72.0% majority baseline, while Sol reaches 56.1% in Brazil, 25.1 points above its largest-class baseline.Romanian subtitles raise country-rating exact match for some models and improve Brazil and Germany on average, while all five non-English settings reduce US performance.Source: Figure 4 · p. 8, §5.3 · pp. 8–9

5.2 RQ2: How Robust Is Understanding Across Subtitle Languages?

• English averages are 3.55 for plot, 2.67 for synopsis, and 2.86 for key message, while Persian averages are 3.20, 2.40, and 2.66, respectively.Evaluator annotations find that 79.7% of synopses omit a major event and 71.1% invent one.Source: §5.2 · p. 8

5.4 RQ4: Can Safety Judgments Be Evidence-Grounded?

• Gemini reaches 86.8% category-agnostic evidence F1 and 77.8% strict F1, showing that locating relevant dialogue and assigning its lexical category are distinct capabilities.Sol has higher recall than DeepSeek (96.3% versus 45.6%), while DeepSeek has higher precision (77.8% versus 70.9%).Source: Figure 5 · p. 9, §5.4 · p. 9
• Across nine models, strong profanity reaches 62.8% category-level F1 versus 20.3% for mild obscenity; 73.7% of strong-profanity gold evidence is missed.For Sol and Gemini, strong-profanity strict F1 is 99.6% and 99.1%, compared with 39.7% and 52.0% for mild obscenity.Source: Figure 5 · p. 9, §5.4 · p. 9

6 Discussion

• Strong models score around 3.4/5 on synopses, with major-event omissions and invented events, leaving headroom for film summarization, search, catalog enrichment, and narrative analysis.These applications require tracking characters, events, and causal progression.Source: §6 · p. 9
• Narrative performance declines from English to non-English settings for all nine models in point estimate, and Persian repeatedly yields the weakest results across several tasks.The discussion identifies this instability as a limitation for systems serving multilingual audiences.Source: §6 · p. 9
• Country-rating exact match is 31.6% in France against a 72.0% majority baseline, while Brazil shows substantial gains over its weaker baseline.The same films evaluated across ten national systems expose country-specific calibration differences that a generic rating task would obscure.Source: §6 · pp. 9–10
• Strong profanity approaches near-ceiling grounding for the best models, but mild obscenity reaches only 52.0% strict F1 for Gemini and 39.7% for Sol.Together, the benchmark findings show that current LLMs are not uniformly reliable across narrative understanding, multilingual consistency, cultural calibration, and evidence grounding.Source: §6 · p. 10

• Premise recovery is more reliable than complete narrative reconstruction, while cross-lingual consistency varies across models and languages.Source: §7 · p. 10
• National classification systems reveal distinct calibration behavior, and evidence-grounded language safety remains strongly category-dependent.Source: §7 · p. 10
• The findings indicate substantial headroom for current models across narrative, linguistic, cultural, and evidence-grounding dimensions of film understanding.Source: §7 · p. 10
• CineSubBench provides a unified benchmark for measuring progress across these capabilities through matched task, language, and cultural coverage.Source: §7 · p. 10

• The benchmark is intended for non-commercial research and uses public film metadata, audience-guidance information, national classification labels, and community-contributed subtitle files.Source: Ethics Statement · p. 10
• CineSubBench uses publicly accessible community-contributed subtitle tracks for 1,012 films without redistributing movie video.This representation supports matched evaluation across six languages and ten national classification systems.Source: Ethics Statement · p. 10
• 6,594 non-narrative artifacts are removed from 8,133,088 subtitle entries, while email addresses are replaced and surrounding narrative text retained.Source: Ethics Statement · p. 10
• Language-safety annotations preserve task-relevant sensitive-language evidence by subtitle index rather than reproducing offensive expressions in label fields.Source: Ethics Statement · p. 10

A.1 Source Roles and Construction Overview

• Construction starts from Kids-in-Mind and uses IMDb identifiers to link subtitle, review, and metadata sources before filtering for complete task fields.The initial inventory contained 6,186 films; 2,322 metadata-complete candidates remained for coverage selection, and title/year alone was not treated as sufficient evidence of a match.Source: §A.1 · p. 17, §A.2 · pp. 17–18

A.3 Subtitle Acquisition, Provenance, and Recovery

• Subtitle files are retrieved by IMDb identifier, with download count used only to choose an initial candidate before structural, language, timing, and manual checks.Suspect tracks can be replaced or repaired, but unresolved parsing or decoding failures prevent ingestion.Source: §A.3 · p. 18

A.4 Complete-Coverage Optimization over Languages and Countries

• 1,231 films across six selected subtitle languages yield 7,386 files, 444 more than the six-language frequency prefix; final rating coverage retains 1,012 films across ten countries.The selected languages are Arabic, English, Indonesian, Persian, Romanian, and Vietnamese; the ten rating systems are selected through complete-coverage search.Source: §A.4 · p. 19

A.5 Multi-Layer Subtitle Verification and Manual Audit

• Automated verification flags distinct subtitle failures for curator review, who decides whether to retain, repair, replace, or reject the track.Checks target provider error payloads, incomplete temporal coverage or gaps, and malformed timing blocks; unresolved structural failures stop ingestion.Source: §A.5 · p. 21

A.5.1 Structural and Encoding Safeguards

• Strict parsing and deterministic decoding reject unresolved malformed or undecodable files; sanitization removes 6,594 artifacts while preserving subtitle alignment and required dataset fields.Of 8,133,088 entries, non-narrative distribution artifacts are removed; in 65 narrative-relevant entries, only the email address is replaced.Source: §A.5.1 · p. 21, §A.5.2 · p. 22, §A.5.3 · p. 22

A.6 Subtitle-Grounded Language-Safety Gold Construction and Audit

• Subtitle-grounded safety labels count lexical occurrences separately from evidence lines, retain source disagreements, and limit categories to language reproducibly recoverable from subtitles.The workflow scans normalized English subtitle entries, audits discrepancies against review-level counts, and excludes non-lexical evidence such as obscene gestures.Source: §A.6 · pp. 22–23

A.7 Task-Oriented Label Curation and Normalization

• Task references are curated for complete coverage, narrative fields are cleaned to film-internal content, and age suitability remains distinct from country-specific ordinal rating systems.Country labels are normalized within each national system rather than treated as interchangeable scales or continuous pseudo-normalized scores.Source: §A.7 · pp. 23–24

A.8 Final Film-Entity Structure and Field Provenance

• The final JSON schema stores one film per record, combining metadata, task references, provenance, and six timestamped subtitle tracks.Source: §A.8 · p. 24
• CineSubBench’s film entity combines identity and cast metadata, narrative and classification references, subtitle-derived language-content assessments, provenance URLs, and six ordered subtitle tracks.The schema distinguishes subtitle-derived language-content labels and evidence indices from retained external review information; field provenance names the source or construction process.Source: §A.8 · p. 24, Table 7 · p. 25

A.9 Dataset Statistics and Coverage Audits

• The dataset statistics report 22 core genre labels and a mean/median of 4.08/4.0 raw genres per film.The accompanying tables cover all 1,012 benchmark films and include narrative-reference lengths and label distributions.Source: §A.9.1 · p. 25, §A.9 · p. 25

A.9.1 Subtitle-Duration Coverage Audit

• Arabic tracks had mean temporal span/runtime 0.931 and fifth percentile 0.868, with none below the short-runtime threshold; 27 fell below the token-ratio threshold and 120 below the entry-ratio threshold, while density indicators showed little association with narrative quality.The thresholds flag tracks for inspection rather than automatic deletion because subtitle activity need not cover every visual moment.Source: §A.9.1 · pp. 25, 27

B.1 Model Coverage and Generation Settings

• The main evaluation compares nine models on matched six-language films, while a separate Ministral 3B/8B/14B experiment tests English-only within-family scaling; language-safety assessment is English-only.API models use temperature 0 and local models greedy decoding, with shared task-specific output ceilings and schema validation.Source: §B.1 · pp. 27–28

B.2 Metric Definitions

• Evaluation uses a reference-based LLM judge for narrative generation, micro-F1 for genre prediction, and task-specific error or accuracy measures for age, country ratings, and subtitle-evidence safety.The cross-task composite averages narrative and cultural task metrics across six input languages, while language-safety strict F1 remains English-only.Source: §B.2 · pp. 29–30

B.3 Task and Evaluator Instructions

• Task prompts require subtitle-only English outputs; narrative instructions specify plot, synopsis, key-message, and genre constraints, while safety instructions require exhaustive English-line extraction with category-specific evidence indices.The narrative judge uses a two-stage reference-based procedure: it inventories story centrality, identifies concrete task-specific errors, then scores against task-matched references, penalizing omissions, inventions, weak causality, and mis-centering.Source: §B.3 · pp. 31–32
• Narrative outputs are scored in two stages with typed errors and 1–5 dimension and overall scores, anchored to fidelity and plot-specific preservation of premise, conflict, and emphasis.The rubric distinguishes strong, incomplete, materially mis-centered, substantially erroneous, and mostly wrong responses.Source: §B.3 · p. 32
• Cultural predictions must use subtitle-visible evidence only, be written in English, and assign age suitability and each country’s rating independently, with conservative judgments when visual evidence is absent.The instructions prohibit outside film knowledge and require culturally situated country-specific scales rather than a universal rating.Source: §B.3 · p. 33

B.4 Independent Narrative-Judge Agreement

• Pearson r = 0.808, Spearman ρ = 0.815, and quadratic weighted κ = 0.778; 98.4% of ratings differed by at most one point, while Claude Sonnet 5 scored 0.19 points lower on average than GPT-5.6 Luna.Exact agreement was 56.8%. Film-clustered bootstrap 95% confidence intervals were [0.780, 0.833] for Pearson correlation, [0.785, 0.839] for Spearman correlation, [0.749, 0.803] for quadratic weighted κ, and [−0.253, −0.122] for the mean score difference.Source: §B.4 · p. 34, Table 17 · p. 34

B.5 Response-Format Reliability

• Schema validation precedes scoring; formatting-only repair or one retry may address serialization defects, but unusable predictions, provider errors, and substantial truncation remain failures, with semantic content unchanged.The appendix separates response-format reliability from semantic task accuracy.Source: §B.5 · p. 34

B.6 Title/Year Recall and Character-Name Counterfactuals

• The probes compare title/year-only outputs with subtitle-only outputs on 50 films, and compare standard with transformed subtitles on 20 films.For the counterfactual, two manually screened central-character aliases are exchanged in subtitles and references; titles and years are withheld.Source: §B.6 · pp. 34–35
• Title/year-only genre prediction raises micro-F1 for all five models by 2.5–11.0 percentage points, while narrative changes range from −0.23 to +0.07.Narrative intervals exclude zero for Claude Haiku 4.5 and Llama 4 Scout, both favoring subtitle-conditioned generation.Source: §B.6 · pp. 35–36, Table 19 · p. 35, Table 21 · p. 36
• Film identity alone provides substantial information for genre recognition, whereas detailed narrative reconstruction benefits more unevenly from subtitle evidence across models.Source: §B.6 · pp. 35–36
• The character-name counterfactual lowers the paired narrative mean for all five models; GPT-5.6 Sol falls by 0.48 and Gemini 3.8 Flash by 0.35, with intervals excluding zero.Intervals for the other three models include zero; the intervention tests sensitivity to the name substitution as a whole, not to any single name property.Source: §B.6 · p. 36

C.1 English and Cross-Lingual Headline Results

• On English input, DeepSeek reaches 83.2% genre micro-F1 and 82.5% age accuracy within one year, but 58.5% country-rating exact match versus Gemini’s 78.8%.GPT-5.6 Sol and Gemini are nearly tied on English synopsis scores (3.42 and 3.43); across languages, task leaders also differ, with Sol highest on synopsis at 3.39/5 and Gemini highest on age exact match at 33.8%.Source: §C.1 · pp. 36–37, Table 22 · p. 37, Table 23 · p. 38

C.2 Paired Comparisons of Leading Composite Scores

• GPT-5.6 Sol’s composite score is 3.34 points below Gemini 3.8 Flash (95% CI [−3.86, −2.84]); DeepSeek V4 Pro 1.6T is 12.52 points below (95% CI [−13.61, −11.44]).Both Holm-adjusted randomization-test p-values are < 0.001; intervals capture film-level sampling uncertainty, not run-to-run generation variability.Source: §C.2 · p. 37, Table 25 · p. 39

C.3 Narrative Failure Patterns

• Missing major events occur in 79.7% and invented events in 71.1% of judged synopses; plot errors most often involve wrong emphasis (54.7%), while key-message errors most often involve semantic mismatch (80.3%).These are evaluator diagnostic incidences, not independent human labels; the error-profile figure summarizes annotation shares by subtask and type.Source: §C.3 · p. 39, Table 26 · p. 39, Figure 10 · p. 40

D.1 English-to-Cross-Lingual Overview

• Narrative overall decreases for all nine models from English to the equal-weight five-language average, while some model-language combinations improve on genre, age suitability, or country ratings.The cross-lingual overview compares the same films; it motivates the subsequent language-specific analyses.Source: §D.1 · p. 41, Table 27 · p. 41

D.2 Narrative Scores by Subtitle Language

• Across models, English means are 3.55 for plot and 2.67 for synopsis; Persian means are 3.20 and 2.40, respectively, so the plot–synopsis gap persists within languages.The six-language-average figure compares plot, synopsis, and key-message scores by model.Source: §D.2 · p. 41, Table 28 · p. 41, Table 29 · p. 42

D.3 Genre Stability and Label-Specific Failures

• Cross-lingual genre changes vary by model: Mistral 4 loses 3.8 percentage points of micro-F1 while Gemini changes by +0.3, and high micro-F1 does not ensure exact label-set recovery.Gemini reaches 84.9% six-language micro-F1 but 40.8% exact match; label errors are selective, including near-matches on some genres and severe Biography omissions by Qwen3 and Mistral 4.Source: §D.3 · p. 43, Table 36 · p. 45

D.4 Paired Cross-Lingual Uncertainty

• Paired film-level resampling finds that the small narrative declines for GPT-5.6 Sol and DeepSeek V4 Pro have intervals containing zero, while Llama 4 Scout’s narrative decline remains clearly negative.The analysis preserves six-language pairing; intervals containing zero do not indicate a consistently signed change.Source: §D.4 · p. 43, Table 37 · p. 46

E.1 Age Error Magnitude and Direction

• Across six languages, GPT-5.6 Sol overpredicts age more often than it underpredicts (48.3% versus 20.6%), while Gemini 3.8 Flash shows the reverse (15.2% versus 50.6%).Gemini matches the reference age exactly in 34.2% of predictions and within one year in 81.4%; Sol reaches 31.1% and 78.8%, respectively.Source: §E.1 · p. 46, Table 38 · p. 46, Table 43 · p. 48
• More-than-two-year age errors range from 5.4% for DeepSeek V4 Pro and Sol to 29.1% for Llama 4 Scout, despite models sometimes appearing similar on near-match metrics.Source: §E.1 · p. 46, Table 38 · p. 46, Table 43 · p. 48

E.2 Country-Level Calibration and Language Sensitivity

• France has 31.6% mean model exact match against a 72.0% majority baseline, with eight of nine models below baseline; in the United States, the best-model lift is 41.8 points over a 49.0% baseline.Country-level interpretation therefore requires both raw exact match and baseline-relative lift.Source: §E.2 · p. 46, Table 45 · p. 49, Table 44 · p. 49
• Sol’s nonzero errors are mostly stricter in France, Germany, and the Netherlands, yet its lift over majority baselines ranges from −21.1 points in France to +12.9 in the Netherlands.The same exact-match score can arise from different calibration regimes; DeepSeek’s 58.0% country exact match exceeds Claude’s 54.3% overall but trails Gemini in every country.Source: §E.2 · p. 49, Table 45 · p. 49
• Country-rating language effects are heterogeneous: Romanian subtitles raise exact match by 3.7 points for Llama 4 Scout and 3.4 for GPT-5 Nano, while every non-English input lowers United States accuracy.France is comparatively stable across languages, with mean model accuracy between 30.0% and 32.5%.Source: §E.2 · p. 49, Table 48 · p. 50, Figure 12 · p. 51

F.1 Evidence Errors by Category and Model

• Sol has the highest category-agnostic recall at 96.3% but 70.9% precision; Gemini is more balanced at 83.0% precision and 90.9% recall, yielding the highest agnostic F1 at 86.8%.DeepSeek shows the reverse profile, with 77.8% precision and 45.6% recall.Source: §F.1 · p. 52, Table 50 · p. 52
• Strict matching reduces Gemini’s category-agnostic F1 from 86.8% to 77.8% and Sol’s from 81.7% to 74.5%; Gemini has the lowest count MAE at 5.01 per film and category.Strict F1 requires the correct subtitle index and lexical category, while agnostic metrics assess evidence-index recovery regardless of category.Source: §F.1 · p. 52, Table 50 · p. 52, Table 52 · p. 53
• Mild obscenity reaches 20.3% category-level F1, with 26.3% recall and an 81.8% false-positive rate; strong profanity reaches 62.8% F1.The category profile figure likewise identifies strong profanity as easiest overall and mild obscenity as combining low recall with frequent false positives.Source: §F.1 · p. 52, Table 51 · p. 53, Figure 13 · p. 54

G Additional Analysis: Within-Family Scaling

• Narrative overall rises 2.25→2.55→2.78 and age MAE falls 2.42→1.92→1.41 years from 3B to 14B; genre micro-F1 and country-rating exact match peak at 8B.Genre micro-F1 is 56.7%, 66.7%, and 64.8%; country-rating exact match is 22.5%, 36.6%, and 35.6% across 3B, 8B, and 14B.Source: G Additional Analysis: Within-Family… · p. 54, Table 53 · p. 54, Figure 14 · p. 55

H Qualitative Case Studies

• Fluent synopsis excerpts can contain unsupported locations, events, relationships, or genres, and one example reverses a central relationship outcome.The examples are selected cases, not estimates of how often these errors occur.Source: H Qualitative Case Studies · p. 55, Figure 15 · p. 56, Figure 19 · p. 58
• For Alpha, Llama 4 Scout shifts the gold key message from interspecies empathy toward strength as a condition of survival.GPT-5.6 Sol instead emphasizes compassion, loyalty, and protecting others.Source: H Qualitative Case Studies · p. 57
• Other examples show a thematic reversal, a genre outside the gold set, a missed explicit evidence line, and a selected broad upward shift in age and national classifications.The examples illustrate mechanisms rather than error frequency or prevalence.Source: H Qualitative Case Studies · p. 55, Figure 16 · p. 56, Figure 17 · p. 57
• Figure 18 contrasts saved GPT-5.6 Sol outputs after changing subtitle-input language while keeping the film’s gold age and national-rating labels identical.The selected case illustrates a possible cultural benefit of multilingual input; aggregate language comparisons support population-level analysis.Source: Figure 18 · p. 58

I.1 Limitations

• Cross-lingual comparisons reflect translation, tokenization, subtitle density, and model-language proficiency together, despite structural, temporal, density, consistency, and manual checks.All six languages are evaluated over the same matched films.Source: §I.1 · p. 59
• The 1,012-film matched design supports controlled comparisons across tasks, languages, and cultural settings, not population-level characterization of global cinema.CineSubBench prioritizes complete matched coverage across six subtitle languages and ten national classification systems.Source: §I.1 · p. 59
• Age recommendations and national ratings retain source conventions, with national systems kept as separate ordered label spaces and variant labels normalized only as needed.Narrative references provide a common semantic anchor while allowing valid wording and emphasis variation.Source: §I.1 · p. 59
• Language-safety evaluation is limited to auditable lexical evidence in four categories from the English subtitle track; broader taxonomies and multilingual annotations remain future directions.Source: §I.1 · p. 59
• Titles, release years, cast, and reference answers are withheld from standard inputs, but public film information may appear in model pretraining data.Appendix B.6 separately measures title/year cue signal and sensitivity to character-name perturbations.Source: §I.1 · p. 59
• The subtitle interface measures narrative and cultural understanding from timed text, while visual action, cinematography, music, gestures, and other non-speech signals require complementary multimodal evaluation.Source: §I.1 · p. 59

I.2 Intended Use

• Intended applications include long-form film understanding, multilingual subtitle processing, culturally situated audience assessment, and human-reviewable language-safety evaluation rather than automated censorship.Country-specific ratings should remain associated with their originating national systems.Source: §I.2 · p. 59

I.3 Data Statement

• Records cover 1,012 films with source metadata, normalized labels, six timestamped subtitle tracks per film, indexed safety evidence, task definitions, and provenance fields.Construction and verification are documented in Appendix A; release and redistribution follow source-provider requirements.Source: §I.3 · p. 59