PaperlayerView exact revision on arXiv

Source-linked AI summary

MEV: A Multi-Event Video Dataset for Long-Take Generation

Peiyuan Zhu, Shaoan Xie, Yifan Shen, Wen Tian, Zijian Li, Guangyi Chen, Kun Zhang

arXiv:2610.38241v1cs.CVcs.AI

TL;DR

Existing video-text datasets often provide short, single-event clips or coarse captions, limiting supervision for long-take temporal structure. MEV introduces a publicly released multi-event dataset with temporal grounding, structured annotations, and event-centric evaluation. Fine-tuning with grounded event conditioning improves event transitions and content coherence.

  • Problem

    Many video-text datasets are dominated by short or action-limited clips and lack explicit temporal boundaries or fine-grained captions for long-take multi-event generation.

    Source: §1 · p. 2

  • Method

    MEV combines long-take videos segmented into temporally grounded events with structured global and event-level captions, then evaluates generation using temporally grounded event conditioning.

    Source: §1 · p. 2, §2 · p. 3, §3.1 · p. 8

  • Results

    Fine-tuning Wan with MEV and temporally grounded event conditioning improves long-take event transitions and coherence, and outperforms the counterpart dataset across evaluations.

    Source: §1 · p. 3, §4 · p. 9

  • Takeaways & Limitations

    The reported experiments indicate that MEV provides effective supervision for multi-event generation with more coherent long-take content.

    Source: §4 · p. 9

    Gemini can confuse camera-motion labels, requiring dedicated motion descriptions and manual verification of obvious inconsistencies.

    Source: §2.3 · p. 7

Abstract

from arXiv · show

Current datasets for text-to-video generation are largely limited to short clips depicting a single event or coarse-grained captions. As a result, they provide limited supervision for modeling long-take temporal dependencies, often leading to temporally inconsistent generations when describing multiple events. In this paper, we introduce MEV, a multi-event video dataset designed for long-take video generation. MEV provides high-quality real-world videos where subjects can perform a sequence of distinct, non-overlapping events, featuring precise temporal boundaries and fine-grained procedural annotations. This event-centric discretization reduces confounding factors and enables models to support temporally localized conditioning and build long-take videos from atomic event units. Furthermore, we introduce a comprehensive annotation pipeline that segments videos into temporally grounded events, and generates structured captions at both the global level and the event level. In addition, under an event-centric evaluation protocol, we conduct multi-event generation experiments by fine-tuning on MEV with temporally grounded event conditioning, leading to improved long-take event transitions and more coherent content. We believe this work provides a publicly available resource for multi-event fine-tuning and event-centric evaluation, together with practical insights into learning from temporally structured video-text data.

1 Introduction

Long-take multi-event generation needs training data with clear event boundaries and detailed temporal supervision. MEV supplies this structure, and fine-tuning with temporally grounded event conditioning improves event transitions and narrative coherence.

1 Introduction

  • Existing datasets often lack explicit temporal boundaries, rich action detail, and captions describing stylistic and temporal nuances needed for long-take multi-event generation.Many consist of trimmed clips with limited action richness, while some temporally annotated videos contain discontinuities or cluttered scenes.Source: §1 · p. 2
  • MEV contains more than 8k high-definition long videos, with 2–21 events per sequence and over 25k event-level segments.Videos have precise event timestamps, global and event-level captions, camera annotations, semantic attributes, and quality scores.Source: §1 · p. 2
  • MEV combines video-level captions at multiple granularities and structured global fields with temporally segmented events annotated with timestamps and event-level descriptions.The annotations also include semantic attributes for subset selection or conditioning and VBench-based quality scores.Source: §1 · p. 2, Figure 2 · p. 3
  • Fine-tuning Wan 2.1 on MEV with temporally grounded event conditioning improves long-take event transitions and narrative coherence, outperforming a counterpart dataset across evaluations.Baselines show disordered event structures or content drifting, while MEV-Wan better grounds event-level information.Source: §1 · p. 3

2 MEV Dataset

MEV combines long-take videos with precise event segmentation and global-to-event annotations. Its construction uses curated real-world footage, rule-guided segmentation, and structured captions, with dataset statistics and comparisons characterizing its coverage.

2 MEV Dataset

  • MEV is a long-take, multi-event text-video dataset with precise temporal grounding and global-event captions, constructed through collection, segmentation, annotation, and statistical analysis.Its videos are compared with representative text-video datasets.Source: §2 · p. 3

2.1 Data Collection and Curation

  • The collection prioritizes high-quality real-world footage with evolving activities, meaningful camera motion, adequate resolution, and diverse subjects.The initial pool contains 227K clips; statistical filtering reduces this to 87K, and model-based purification yields 41K candidates for event triage.Source: §2.1 · p. 4

2.2 Event Segmentation

  • MEV defines events as procedural action units sharing a micro-intent and interaction object, then uses rule-guided atomic grounding to identify reproducible events and localize their boundaries.Continuous sub-actions without a reproducible cut point may be merged.Source: §2.2 · p. 5
  • Three annotators label 52.9% of the curated videos, and a fine-tuned event-aware model predicts boundaries for the remaining 47.1%, followed by manual verification.Annotator timestamps agree when they differ by no more than 0.5 seconds, with at least two of three annotators meeting that threshold.Source: §2.2 · p. 6

2.3 Comprehensive Annotation

  • Annotations combine global context fields and short, middle, and long captions with event-level camera descriptions, focus shifts, semantic attributes, and VBench quality scores.The caption lengths are around 20, 80, and 200 words, respectively.Source: §2.3 · p. 6
  • Gemini-generated annotations can confuse camera-motion labels, so the pipeline adds natural-language motion descriptions and manually verifies obvious inconsistencies.The reported examples include confusing truck-left with pan-left.Source: §2.3 · p. 7

2.4 Statistical Comparison and Analysis

  • MEV emphasizes dense temporally grounded supervision, combining high-resolution long-take video with global-event annotations; its statistics cover durations, event counts, camera labels, captions, and attributes.The three global-caption granularities exhibit clear length separation.Source: §2.4 · p. 7

3 Experiments

The experiments evaluate MEV-Wan against Wan2.1, LongLive, and Ultra-Wan using event-centric and generic video-quality measures. MEV-Wan ranks first across the event-centric metrics and produces more coherent multi-event generations in the reported comparisons.

3.3 MEV Benefits Multi-Event Video Generation.

  • MEV-Wan generates all events with smooth transitions and consistent content, while the baselines show disordered structure, drifting, or misalignment.Ultra-Wan also exhibits video stagnation and increasing misalignment after the third event.Source: Figure 4 · p. 8, §3.3 · p. 9

3.1 Experimental Settings

  • MEV-Wan combines global-prompt semantics with temporally localized event conditioning, using ReRoPE and duration-aware event grouping for ordered control.The method fine-tunes Wan with LoRA; Ultra-Wan uses the same backbone and MinT-style adaptation but caption-inferred events and uniform boundaries.Source: §3.1 · pp. 7–8

3.2 MEV-Eval: Event-Centric Evaluation Protocol.

  • Human and Gemini evaluations both rank MEV-Wan first on all four event-centric metrics.The metrics assess event execution, ordering, temporal grounding, and boundary-transition coherence.Source: Table 2 · p. 8, §3.2 · p. 9

3.3 MEV Benefits Multi-Event Video Generation.

  • MEV-Wan maintains Wan-backbone fidelity and smoothness while improving imaging quality, without trading visual quality for event control.The event-centric comparison also reports better event order, temporal-window alignment, and reduced transition drift.Source: §3.3 · p. 9

4 Conclusion

MEV is a long-take multi-event text–video dataset with precise temporal grounding and global- and event-level captions. The paper reports that adapting Wan with temporally grounded event conditioning improves long-take event transitions and content coherence.

4 Conclusion

  • MEV combines precisely grounded event boundaries and global-event captions with an annotation pipeline, quality analyses, and event-centric evaluation.Adapting Wan with temporally grounded event conditioning improves long-take event transitions and content coherence.Source: §4 · p. 9

Outline

The outline maps the supplementary material from related work and annotation through data release, experiments, analysis, implementation, evaluation, and broader use.

Outline

  • Sections A–D cover related work, annotation taxonomy and guidelines, data sources, release format, licensing, and responsible-use considerations.The supplementary material includes these topics alongside experimental and analytical sections.Source: Outline · p. 16
  • Sections E–H present SFT and other VLM experiments, dataset statistics, MEV-Wan implementation, and the MEV-Eval protocol.Source: Outline · p. 16
  • Sections I–J provide additional qualitative results and failure cases, then discuss broader use, limitations, and future directions.Source: Outline · p. 16

A.1 Video-Text Datasets

The related-work discussion identifies limits in existing video-text datasets and generation methods for long-take multi-event modeling. MEV's taxonomy and annotation guidelines address these needs with structured captions, event attributes, and reproducible temporal boundaries.

A.1 Video-Text Datasets

  • Many video-text datasets use short or loosely aligned clips and lack explicit event boundaries and paired global- and event-level descriptions for multi-event generation.MEV instead provides subject-centric procedural descriptions at sequence and event granularity, including appearance and camera-motion changes.Source: §A.1 · pp. 16–17

A.2 Multi-Event Video Generation

  • Multi-event generation methods use temporal autoregression or divide-and-conquer, while multi-shot generation is orthogonal to event count.The cited approaches include prompt-based generation, latent anchoring, timestamped event control, and synchronized local and global optimization.Source: §A.2 · p. 17

B Annotation Taxonomy and Field Definitions

  • MEV pairs eight global fields and three caption granularities with event-level semantic and camera attributes for describing persistent context and local change.Its simplified camera labels prioritize common, visually reliable motions while leaving finer details to natural-language descriptions.Source: B Annotation Taxonomy and Field… · p. 17

C Annotation Guidelines

  • Annotators segment videos with LosslessCut, then label event attributes and camera motion through a question-based interface.Boundaries count as matching within 0.5 seconds; at least two of three annotators must agree, or segments are merged or samples filtered.Source: C Annotation Guidelines · pp. 18–20
  • MEV favors reproducible, action-driven atomic events over finer splits that annotators cannot consistently agree on.A consistent activity with one continuous camera motion remains one event, while a semantically meaningful camera change can mark a new event.Source: C Annotation Guidelines · pp. 18–20
  • Only retained reliable multi-event videos receive full event-level annotations, after extensive filtering of the curated pool.Manual inspection covers 52.9% of the pool; agreement statistics distinguish video-level triage from boundary reconciliation.Source: C Annotation Guidelines · p. 20

D Data Sources, Release, Licensing, and Responsible Use

MEV draws from selected public video sources and is released with annotations and metadata under stated licensing and responsible-use conditions. Its collection choices favor visually clear, action-rich, non-sensitive clips, limiting dataset coverage.

Data sources and scope

  • MEV selects public videos primarily from MixKit, supplemented by YouTube videos in predefined activity-oriented categories, then filters candidates for long-take multi-event requirements.Categories include daily activities, cooking, fitness or sports, tool use or DIY, animal interactions, and outdoor activities.Source: D Data Sources, Release, Licensing,… · p. 21

Release and licensing

  • The official public package includes videos and metadata; author-created annotations and evaluation materials use CC BY 4.0 unless a file states otherwise.Included video files remain subject to applicable source licenses or platform terms; optional restricted 4K variants are not part of the public package.Source: D Data Sources, Release, Licensing,… · p. 22

Responsible use

  • MEV excludes sensitive or private-context content and avoids identity or sensitive personal attributes, biasing coverage toward visually clear, action-rich, non-sensitive clips.Excluded categories include violence, accidents, medical scenes, political or religious events, surveillance footage, and news.Source: D Data Sources, Release, Licensing,… · p. 22
  • Materials may be removed or modified if rights holders request takedown or legal, ethical, privacy, or security concerns arise.Affected videos are removed from subsequent public packages when appropriate.Source: D Data Sources, Release, Licensing,… · p. 22

E Experiments on Annotation Model

The experiments evaluate supervised event segmentation against VLM baselines using human-verified references. SFT improves segmentation quality, while qualitative examples show that precise temporal grounding still requires human review.

SFT data and setup

  • Fine-tuning Qwen3-VL-32B uses about 5K videos in its temporally grounded multi-event portion, with human-annotated event boundaries and status-only, timestamp, and explanation templates for filtering, retained videos, and borderline cases.The language model is trainable while the vision tower and multimodal projector remain frozen; videos are sampled at 8 FPS.Source: E Experiments on Annotation Model · pp. 22–23

Evaluation metrics

  • Evaluation covers boundary precision, recall, and F1 at 0.5s and 1.0s tolerances, segment-level overlap, and absolute and relative event-count error.The 0.5s boundary setting is the main number; segment evaluation reports Segment F1 at tIoU=0.7 and length-weighted matched tIoU.Source: E Experiments on Annotation Model · pp. 23–24

Experimental results

Baselines and results

  • The comparison includes Qwen3-VL-8B, Qwen3-VL-32B, and Gemini 2.5 Pro; among non-SFT baselines, Gemini leans toward recall while Qwen3-VL-32B is stronger in precision and segment overlap.Qwen3-VL-8B is weaker than the larger models across counting and temporal-localization metrics.Source: E Experiments on Annotation Model · pp. 23–24

Qualitative comparison

  • SFT often recovers coarse event structure and event count better than non-SFT baselines, but its timestamp precision remains imperfect and human verification remains necessary.The qualitative comparison notes that native Qwen3-VL can merge camera-behavior changes and distinct actions into one broad event.Source: E Experiments on Annotation Model · p. 24, Figure 7 · p. 25

Event boundary interpretation

  • Human-defined windows distinguish nameable camera changes and subject disappearance from incidental motion, providing more focused event units for downstream conditioning.In the examples, a camera transition marks a meaningful event cue, and a boy disappearing from view forms an event centered on subject visibility.Source: E Experiments on Annotation Model · pp. 24–25

F Additional Analysis of MEV Dataset

MEV’s analyses show that fine-grained camera-motion factorization and subtle lighting reasoning remain difficult for current VLMs, while caption granularity affects generation and retrieval. Dataset-level variation and human correction also inform how annotations and quality scores should be interpreted.

Additional dataset statistics

  • 51.1% of videos contain moving-camera events and 59.7% static events; at event level, the shares are 42.9% and 57.1%, respectively.The video-level categories overlap because a video can include both event types. Event captions commonly ground actions in visible subjects and body parts, and frequent verbs include look, wear, hold, sit, smile, and stand.Source: F Additional Analysis of MEV Dataset · p. 25

Quality-score granularity

  • Subject and background consistency vary substantially within videos, whereas imaging and aesthetic quality vary mainly between videos, supporting event-level scoring versus global screening.Event-level imaging and aesthetic scores can still identify localized degradation.Source: F Additional Analysis of MEV Dataset · p. 26, Table 16 · p. 26

Multi-granular caption quality analysis

  • Finer global captions improve generated-video alignment and sequential-event coverage, while added global and event-level signals improve bidirectional retrieval over short-caption baselines.Even Kling 2.6 omits some events, such as turning off the lamp.Source: F Additional Analysis of MEV Dataset · p. 26, Figure 8 · p. 27

Annotation model evaluation and qualitative analysis

  • Camera understanding is weaker than semantic attributes overall; models often notice viewpoint changes but miss fine-grained motion labels, and lighting judgments remain unstable.In qualitative examples, models confuse translation with rotation or assign the wrong motion axis or direction; all tested models also miss a lighting change in one example.Source: F Additional Analysis of MEV Dataset · pp. 27–28

Human verification and correction

  • 83.2% of Gemini-generated camera-label sets required at least one human correction across 25,262 events.The rate counts events whose label sets changed, not individual label edits.Source: F Additional Analysis of MEV Dataset · p. 28

Qualitative analysis of global structured captions

  • Global-caption styles differ: Gemini is compact, Qwen supplies richer background detail, and both miss very fine visual attributes.Gemini was selected for MEV annotation based on its action-focused event descriptions and stronger overall camera-motion and semantic-attribute performance.Source: F Additional Analysis of MEV Dataset · pp. 28–29

Qualitative analyses

  • Camera-motion factorization and subtle lighting reasoning remain notably difficult for current VLMs, despite accessible coarse global understanding and many semantic attributes.The analyses also find style differences in fine-grained scene descriptions: Gemini is more compact, while Qwen provides denser background details.Source: F Additional Analysis of MEV Dataset · p. 29

Conclusion

  • MEV’s analyses connect recurring VLM weaknesses to downstream captioning, event understanding, and long-take video generation.The paper presents human-verified annotations as a resource for connecting those observations to downstream tasks.Source: F Additional Analysis of MEV Dataset · p. 29

G Implementation Details of MEV-Wan

MEV-Wan retains Wan’s global text conditioning and adds event-aware temporal conditioning, with duration-aware event groups used for training. Its implementation reuses Wan’s flow-matching objective and includes choices for efficient adaptation, split integrity, and comparison with related methods.

Relationship among compared methods

  • All four compared methods belong to the Wan2.1-1.3B lineage; MEV-Wan and Ultra-Wan use MinT-style adaptation, while the released Wan and LongLive checkpoints were not further fine-tuned by the authors.Ultra-Wan’s multi-event subset was inferred from temporal connectives and uniformly partitioned into segments before fine-tuning.Source: G Implementation Details of MEV-Wan · pp. 29–30, Table 20 · p. 33

Dual-path conditioning architecture

  • MEV-Wan preserves Wan’s global text pathway and adds temporal cross-attention for event captions aligned to frame regions, with a near-zero-initialized residual gate and soft temporal binding.The global branch supports scene layout, appearance consistency, and style; ReRoPE’s soft temporal bias accommodates smooth transitions and visual overlap near event boundaries.Source: G Implementation Details of MEV-Wan · pp. 29–31

Dataset split and overlap audit

  • Videos are split before event-group construction, and all events from an original video inherit its split; a frame-matching audit found no confirmed duplicate or near-duplicate pairs.The audit used DINOv2 features from frames sampled at 1 fps and manually reviewed flagged pairs.Source: G Implementation Details of MEV-Wan · pp. 31–32, Table 21 · p. 33

Training sample construction

  • Training samples use contiguous groups of 2–5 events, with frame allocations proportional to annotated event durations to encode when events occur and how long they persist.The construction also yields frame-wise event indices, normalized timestamps, validity masks, and aligned temporal spans.Source: G Implementation Details of MEV-Wan · pp. 32–33

Optimization and training procedure

  • MEV-Wan uses Wan’s standard flow-matching supervised fine-tuning objective while extending conditioning to include both global and temporal inputs.The training loop samples a clip and contiguous event group, constructs temporal tensors, encodes latents, and trains the denoiser to predict target velocity.Source: G Implementation Details of MEV-Wan · p. 33

Implementation choices

  • The default adaptation uses rank-32 LoRA on Wan attention and feedforward projections; the implementation also caps samples at five events and uses padding masks and memory-oriented settings.Near-zero temporal-gate initialization and LoRA adaptation are described as helping reduce catastrophic forgetting of Wan’s original generation ability.Source: G Implementation Details of MEV-Wan · p. 33

Compute resources

  • Reported experiments used AMD MI210 GPUs and approximately 2,800 GPU-hours across training, generation, and evaluation.Generation evaluation averages five random draws of 128 prompts from the MEV test split.Source: G Implementation Details of MEV-Wan · p. 33

H MEV-Eval Protocol for Multi-Event Video Generation

MEV-Eval complements general video-quality metrics by checking event execution, order, temporal grounding, and transition coherence; human and Gemini evaluations rank MEV-Wan first on all four metrics. Qualitative examples compare how methods organize three-event sequences.

H MEV-Eval Protocol for Multi-Event Video Generation

  • MEV-Eval assesses whether a generated video executes the intended events, preserves their order, grounds them in intended temporal regions, and transitions coherently between adjacent events.The protocol uses Gemini 2.5 Pro by default, with target temporal assignments derived from MEV’s human-verified annotations and methods evaluated under the same event list and rubric.Source: H MEV-Eval Protocol for Multi-Event… · pp. 33–34
  • EEA measures visible event execution, EOA pairwise event ordering, and TGA whether each duration-assigned region mainly depicts its corresponding event.TGA differs from EEA because a video may execute every event yet place them in the wrong temporal regions or mix them across regions.Source: H MEV-Eval Protocol for Multi-Event… · pp. 34–35
  • BTC rates adjacent-event transitions for semantic continuity, identity stability, object-state consistency, scene or camera continuity, and severe drift or stagnation.Ratings range from 1 for a broken or unrelated transition to 5 for a smooth, logically coherent transition, then are normalized to [0, 1].Source: H MEV-Eval Protocol for Multi-Event… · p. 35
  • Human validation uses 128 fixed test prompts, four methods, and three annotators to assess agreement with Gemini and compare generation methods.The four criteria are EEA, EOA, TGA, and BTC; BTC receives a 1–5 boundary-coherence score.Source: H MEV-Eval Protocol for Multi-Event… · p. 35
  • Both human and Gemini evaluations rank MEV-Wan first on all four event-centric metrics.Their relative ordering of LongLive and Ultra-Wan changes for EOA and TGA.Source: H MEV-Eval Protocol for Multi-Event… · pp. 35–36, Table 24 · p. 35

I Additional Qualitative Results for Multi-Event Generation

  • Flat text-only Wan2.1-1.3B examples do not consistently organize instructed actions into a stable temporal progression, even when prompts include temporal cues.The examples motivate event captions and temporal boundaries that make the intended sequence explicit during training.Source: I Additional Qualitative Results for… · p. 36, Figure 14 · p. 36
  • In a three-event qualitative comparison, MEV-Wan best aligns event order with visual evolution, while LongLive still shows severe drift.Both MEV-Wan and Ultra-Wan show clearer event structure than vanilla Wan2.1, but MEV-Wan has more consistent pose, orientation, and motorcycle interaction across the sequence.Source: I Additional Qualitative Results for… · p. 36

J Broader Use, Limitations, and Future Directions

MEV’s annotations support broader work on controllable generation, temporal grounding, event relations, and causal discovery. The paper also identifies limitations in scalable evaluation and annotation, alongside risks and mitigation measures for public-video data.

J Broader Use, Limitations, and Future Directions

  • Reappearance, mirror reasoning, and multi-view consistency are challenging long-take cases that MEV can support for future attribute-guided studies and event-level analysis.They involve maintaining identity and state across disappearance, consistent reasoning about reflections, and preserving subjects and scene state across viewpoint changes.Source: J Broader Use, Limitations, and Future… · pp. 36–37, Figure 16 · p. 38
  • MEV’s event-level camera descriptions, semantic attributes, captions, and temporal boundaries support controllable generation, language-guided temporal grounding, and studies of event relations and causal discovery.Compared with MECD, MEV is described as larger-scale and richer in generation-oriented conditioning signals and explicit temporal changes.Source: J Broader Use, Limitations, and Future… · p. 37
  • The authors flag misuse risks from misleading synthetic content or imitation of recognizable public scenes, and potential source and content biases in public-video datasets.Mitigations include filtering sensitive and private-context videos, avoiding sensitive personal-attribute annotation, documenting sources and licensing constraints, and supporting removal or modification when concerns arise.Source: J Broader Use, Limitations, and Future… · p. 37
  • Scalable event-centric evaluation still relies heavily on imperfect VLM judges, which can struggle with fine-grained localization, subtle state changes, camera motion, and reflection or multi-view reasoning.The authors identify improved human validation, judge calibration, and uncertainty estimates as future directions.Source: J Broader Use, Limitations, and Future… · p. 37
  • The annotation pipeline still requires manual verification and correction for fields including event captions, semantic attributes, and camera descriptions.The authors propose more accurate annotation models, uncertainty-aware filtering, model-ensemble checks, and task-specific annotators to reduce that burden while preserving reliability.Source: J Broader Use, Limitations, and Future… · pp. 37–38

Related papers

Loading 2610.38241v1…