Source-linked AI summary
CoinVE-200K: A Large-Scale High-Quality Dataset for Compositional Instruction-Guided Video Editing
Fuchen Long, Cong Wang, Zitao Gao, Wenhao Zhong, Yu Cheng, Xiaolu Hou, Yan Li, Xiao Cao, Xinlong Sun, Xi Chen, Yu Liu
TL;DR
Existing instruction-guided video editing datasets and models struggle with multiple editing operations, irrelevant-region modification, and incomplete instruction execution. This paper introduces CoinVE-200K, CoinVE-Bench, and CoinVE-Edit, achieving strong compositional instruction following, region-aware editing, non-target preservation, and temporal stability.
Problem
Existing video editing methods struggle with multi-operation instructions, modifying irrelevant regions or ignoring parts of the given instructions.
Method
The paper constructs CoinVE-200K and CoinVE-Bench, then develops CoinVE-Edit, a 22B model for compositional video editing.
Results
CoinVE-Edit achieves strong performance, with better compositional instruction following, region-aware editing, non-target preservation, and temporal stability than existing models.
Takeaways & Limitations
CoinVE-200K, CoinVE-Bench, and CoinVE-Edit together provide a foundation for compositional instruction-guided video editing research.
Takeaways & Limitations
CoinVE-200K does not cover reference-based editing, fine-grained motion editing, complex camera control, long videos, or highly interactive scenes.
Abstract
from arXiv · showhide
The quality and diversity of instruction-based video editing datasets are steadily improving, yet existing datasets mainly focus on single editing operations and fall short in supporting compositional instruction-guided video editing. In particular, multiple editing intents must be jointly understood and faithfully executed within the same video. To address this issue, we introduce CoinVE-200K, a large-scale, high-quality dataset for Compositional Instruction-Guided Video Editing. CoinVE-200K contains 1080p video-editing pairs of up to 201 frames, covering diverse compositional scenarios where each sample involves 2 to 5 atomic editing operations. The instructions target humans, objects, and backgrounds, and cover edit types such as addition, removal, modification, and stylization. All samples are built through a carefully designed generation and filtering pipeline to ensure instruction faithfulness, visual quality, temporal consistency, and compositional diversity. We also introduce CoinVE-Bench, a benchmark for compositional-instruction video editing across diverse subjects, operation types, and instruction complexities. Furthermore, we present CoinVE-Edit, a 22B compositional video editing model built upon Wan2.1-T2V-14B and Qwen3-VL-8B-Instruct. CoinVE-Edit disentangles region-aware attention for different editing instructions, enabling precise multi-region editing while preserving irrelevant content and temporal coherence. Experiments on CoinVE-Bench show that CoinVE-Edit achieves strong performance in instruction following, compositional editing accuracy, visual quality, and temporal consistency.
1 Introduction
Existing instruction-guided video editing datasets primarily support isolated operations, leaving models ill-equipped for requests combining multiple edits across spatial-temporal regions. The paper addresses this gap with CoinVE-200K, CoinVE-Bench, and CoinVE-Edit for compositional instruction-guided video editing.
- Motivation: Existing datasets mainly target single edits, while compositional requests require jointly understanding instructions, locating editing regions, executing operations faithfully, and preserving irrelevant content.Models trained on these datasets may modify irrelevant regions or ignore parts of complex instructions.
- Dataset: CoinVE-200K contains 200K 1080p video-editing pairs of up to 201 frames, with each sample combining 2 to 5 atomic editing operations.The dataset covers compositional scenarios involving humans, objects, and backgrounds, alongside addition, removal, modification, and stylization.
- Model: CoinVE-Edit is a 22B model built on Wan2.1-T2V-14B and Qwen3-VL-8B-Instruct that disentangles region-aware attention across editing instructions for precise multi-region editing.This design aims to preserve irrelevant content while handling multiple editing operations.
- Benchmark: CoinVE-Bench evaluates compositional video editing across diverse subject combinations, operation types, and instruction complexities.Its evaluation dimensions include instruction following, compositional editing accuracy, region-level precision, non-target preservation, visual quality, and temporal consistency.
- Results: Experiments show that CoinVE-Edit performs strongly on challenging cases involving multiple operations and target regions, with improved instruction following, region-aware editing, non-target preservation, and temporal stability.These results are reported relative to existing instruction-guided video editing models.
2 Related Work
Related work progresses from task-specific video understanding and diffusion-based generation toward instruction-guided video editing. Existing editing datasets and models largely target single operations, motivating compositional methods that jointly decompose, ground, and execute multiple intents while preserving video structure.
- Video Understanding: Video understanding spans spatial-temporal perception, description, and reasoning, evolving from task-specific representation learning toward large language model and multimodal large language model approaches.These capabilities support identifying editable subjects, parsing user intentions, and grounding operations to spatial-temporal regions.
- Video Generation: Video generation has progressed from image-diffusion extensions with temporal modules toward scalable Diffusion Transformer architectures that model videos as spatial-temporal tokens.Early methods introduced temporal layers, motion modules, or 3D U-Nets, while Latte validated latent spatial-temporal token modeling with transformer blocks.
- Instruction-Guided Video Editing: Instruction-guided video editing offers a more intuitive interface than mask-, scribble-, keyframe-, or manually controlled editing by applying natural-language modifications to input videos.This paradigm was inspired by instruction-guided image editing models such as InstructPix2Pix and MagicBrush.
- Instruction-Guided Video Editing: A central challenge is faithfully modifying videos while preserving their spatial-temporal structure and temporal consistency across frames.Early approaches adapted image diffusion models and used attention control or feature propagation, including TokenFlow’s cross-frame diffusion-feature propagation.
- Compositional Video Editing: Recent datasets and models advance large-scale instruction-based video editing but mainly focus on single operations, leaving compositional instructions with multiple intents, subjects, and edit types insufficiently addressed.Compositional editing requires models to decompose instructions and ground multiple operations within the same video.
3 CoinVE-200K
CoinVE-200K is constructed through a filtered, mask-guided pipeline that generates compositional video-editing samples from structured visual taxonomies and multiple atomic operations. The resulting 200K-sample dataset contains 1080p videos with 2–5 instructions, diverse edit types, and broad visual and linguistic coverage.
- Dataset Overview: CoinVE-200K is a large-scale, high-quality dataset for compositional instruction-guided video editing, built through a dedicated data-construction pipeline.The pipeline is designed to produce video-editing pairs supporting multiple editing intents within the same video.
- Source Video Selection: Source videos are filtered from OpenVid-HD to retain clips with at least 81 frames and a shorter side of at least 1080 pixels, then selected for high aesthetic quality and appropriate motion.Aesthetic scores and optical flow are used during source-video selection.
- Instruction Construction: Qwen3.6-27B parses each retained video into subjects, objects, and backgrounds, enabling grounded compositional instructions formed by combining operations such as replace, add, remove, and stylization.The taxonomy supplies concrete visual concepts for instantiating atomic operation templates and composing them into multi-instruction prompts.
- Mask-Guided Synthesis: The mask-guided synthesis pipeline segments edited regions with SAM3, propagates masks across frames using SAM2, edits a key frame, and synthesizes the final video with Wan2.2-Animate-14B or VACE.This design improves control over edited regions, instruction faithfulness, and temporal consistency.
- Quality Filtering: Samples are retained only when every atomic instruction is executed and semantic consistency, temporal consistency, and physical plausibility receive positive judgments.Gemini 2.5 Pro performs automatic assessment, and strict filtering removes incomplete, inconsistent, or unrealistic generations.
- Dataset Statistics: 200K samples contain 81–201 frames at 1080p, cover six relatively balanced atomic editing types, and average 2.55 instructions per video.Instruction counts are distributed across 2, 3, 4, and 5 instructions, while the dataset spans diverse objects, attributes, colors, and editing actions.
4 CoinVE-Edit
CoinVE-Edit is a 22B compositional video editing model that combines Qwen3-VL instruction-visual reasoning with region-aware control and Wan2.1-T2V-14B video generation. Its mask- and gate-guided Q-Blending mechanism enables localized multi-intent editing while preserving irrelevant content and avoiding added DiT parameters.
- MLLM with Learnable Video Queries: CoinVE-Edit uses Qwen3-VL-8B-Instruct with learnable video queries to jointly encode each atomic instruction and the source video context.For each instruction, it extracts visual tokens at video positions and query hidden states encoding the instruction with visual context.
- Architecture: A shared context tensor from all instructions feeds DiT cross-attention, the Mask Predictor, and GateNet through a Connector that produces editing-aware tokens.The Connector projects MLLM query features into DiT-compatible representations, while the shared context supports downstream region-aware control.
- Mask-based Conditions: The Mask Predictor estimates each instruction’s spatial-temporal editing region, while GateNet distinguishes local edits from global or style edits to determine mask strength.The gate approaches 1 for local edits requiring mask adherence and 0 for global edits that bypass the region constraint.
- Video Generation Backbone: The Wan2.1-T2V-14B backbone concatenates source and noised video latents before the DiT, providing structural information for preserving layout, identity, and motion.The denoised latent is decoded by the VAE to produce the edited video.
- Q-Blending Cross-Attention: Q-Blending Cross-Attention softly rescales instruction-specific DiT cross-attention outputs using predicted masks and gates, then sums them across instructions.It preserves the original DiT cross-attention and introduces no new trainable parameters into the DiT backbone.
- Training Objectives: Training combines the standard flow-matching objective with segmentation and gate-classification objectives, while stopping auxiliary-loss gradients before the MLLM and DiT backbones.The auxiliary losses update only the Mask Predictor and GateNet control modules.
5 CoinVE-Bench
CoinVE-Bench establishes a unified evaluation protocol for compositional instruction-guided video editing using 361 high-quality, training-disjoint source videos. Its evaluation combines MLLM-based checklist scoring with specialized evaluator-based perceptual video-fidelity assessment across four dimensions and eleven fine-grained criteria.
- Benchmark construction: CoinVE-Bench contains 361 high-quality source videos collected from open-source repositories and kept strictly disjoint from the training data.The benchmark targets compositional instruction-guided video editing with a unified evaluation protocol.
- Benchmark construction: The benchmark retains clips at least 1080p and 3–10 seconds long while removing severe motion blur, watermarks, subtitles, abrupt transitions, heavy occlusion, and ambiguous content.The resulting videos cover diverse objects, scenes, motions, and camera viewpoints.
- Evaluation protocol: The evaluation matrix combines MLLM-based checklist evaluation of compositional correctness with specialized evaluator-based assessment of perceptual video fidelity.The checklist group evaluates Editing Accuracy, Physical Naturalness, and Semantic Preservation, while the specialized group evaluates Video Quality.
- Evaluation protocol: The two evaluation groups cover four dimensions and eleven fine-grained criteria, using question-answer accuracy for checklist scores and model-specific scales for specialized evaluations.Video Quality includes Aesthetic Quality, Technical Quality, Comprehensive Quality, and Temporal Stability.
6 Experiments
Experiments evaluate CoinVE-Edit on single-instruction OpenVE-Bench and compositional CoinVE-Bench tasks, alongside visualization and ablation studies. The results indicate strong instruction following, localization, visual quality, and motion consistency, while the training setup combines image, video, and CoinVE-200K data in staged optimization.
- Implementation: CoinVE-Edit uses Qwen3-VL-8B-Instruct and Wan2.1-T2V-14B, with staged feature alignment, mixed image-video training, and final CoinVE-200K compositional tuning.The first stage finetunes the MLLM query, connector, and rank-256 LoRA; the second stage uses a 1:1 image-video sampling ratio and rank-128 video-DiT LoRA.
- Single-instruction evaluation: 3.41 overall score: CoinVE-Edit ranks highest among open-source approaches on OpenVE-Bench, including 3.11 for Background Change and 3.81 for Local Remove.The reported scores are judged by Gemini 2.5 Pro, and the two highlighted tasks require accurate localization of target editing regions.
- Single-instruction evaluation: CoinVE-Edit produces better instruction following, higher visual quality, and more natural motion consistency than existing baselines in qualitative OpenVE-Bench comparisons.The “Adding Sunglasses” example highlights failures by other MLLM-based methods to place the edit at the specified region.
- Compositional evaluation: Compositional evaluation on CoinVE-Bench compares Editing Accuracy, Physical Naturalness, Semantic Preservation, and Video Quality across open-source and closed-source models.The study includes Seedance 2.0 and Kling O3 in addition to open-source approaches.
- Visualization and ablation studies: Visualization studies show distinct instruction-associated editing masks, while ablations contrast instruction concatenation and hard Q-Bias guidance with soft Q-Blending cross-attention.Q-Bias can degrade physical naturalness, especially scale consistency and motion naturalness, whereas Q-Blending better preserves the pretrained backbone’s generative prior.
7 Conclusions
The paper introduces CoinVE-200K, a diverse, high-quality dataset for compositional instruction-guided video editing, with samples combining 2 to 5 atomic operations across subjects and edit types. It also identifies uncovered editing scenarios and challenges in long-video, interactive editing as priorities for future work.
- Contributions: CoinVE-200K provides diverse video-editing pairs with 2 to 5 atomic editing operations per sample, spanning humans, objects, backgrounds, and multiple edit types.Covered edit types include addition, removal, modification, and global stylization.
- Contributions: A robust generation and filtering pipeline emphasizes instruction faithfulness, visual quality, temporal consistency, and compositional diversity.
- Limitations and Future Work: Future work targets reference-based, fine-grained motion, and complex camera-control editing, alongside efficient long-video and highly interactive-scene editing.Planned directions include expanding the editing taxonomy, improving long-range temporal consistency, reducing model cost, and developing unified video editing agents.