Source-linked AI summary
InfinityEdit: Infinite Video Editing with a Lightweight Edit-Ignition Adapter
Yunze Tong, Mushui Liu, Canyu Zhao, Shiyi Zhang, Didi Zhu, Peng Zhang, Wanggui He, Jinlong Liu, Ying Chen, Hao Jiang, Pipei Huang, Bo Zheng
TL;DR
The paper addresses video editing for open-ended streams, where fixed-window in-place methods cannot continue edits into future frames. It introduces a data pipeline and InfinityEdit, a lightweight adapter for a frozen streaming generator, and reports faithful continuation with stable quality across unbounded edit sequences.
Problem
Existing instruction-based video editors assume fixed temporal windows, leaving open-ended streams without a way to apply edits as future frames arrive.
Method
The paper constructs training triplets and adds InfinityEdit, a three-attention-module adapter that activates when an edit arrives while a frozen streaming generator continues later chunks.
Results
Extensive experiments show that InfinityEdit faithfully continues the stream under each edit and remains stable over unbounded edit sequences.
Takeaways & Limitations
Infinite video editing is presented as a distinct setting requiring both continuation beyond aligned source frames and stability as edits accumulate.
Takeaways & Limitations
The current editor accepts only natural-language instructions, and edit-boundary transitions can remain abrupt.
Abstract
from arXiv · showhide
With large pretrained models, existing methods have effectively improved instruction-based video editing. However, most of them rely on an in-place editing assumption. They align the edited video with the given source clip frame by frame over a fixed time span. This pattern fails for open-ended streams, e.g., restyling a live game or applying a camera move to an ongoing shot. In such cases, edits must extend to future frames as they arrive, rather than be applied to a static input clip. In this paper, we study this setting and name it infinite video editing: given a preceding segment and an edit request, a model must generate the next segment that continues the stream while applying the requested edit. This process repeats as an unbounded sequence of edit instructions arrives. This task brings two challenges: the edit must be a faithful continuation rather than a frame-wise rewrite, and generation quality must remain stable as edits accumulate. To address them, we first design a data-collection pipeline for infinite video editing. Based on the collected data, we propose InfinityEdit, a lightweight edit adapter that equips a streaming video generator with unbounded editing ability. The adapter contains three attention modules. History cross-attention guides the denoising frames using the input frames. Temporal causal self-attention keeps temporal cues flowing only from earlier frames to later ones. Edit cross-attention injects the edit request into generation. During inference, the adapter is activated only in the chunk where an edit request arrives. Subsequent chunks are generated by the original model with a reset anchor frame. This scheme applies the edit while preserving the original model's infinite generation ability. Extensive experiments show that InfinityEdit faithfully continues the stream under each edit, and stays stable over unbounded edit sequences.
1 Introduction
The paper introduces infinite video editing for open-ended streams, where edits must continue into future segments rather than rewrite a fixed clip. InfinityEdit combines a dedicated data pipeline with a lightweight adapter, and experiments show effective handling of the task.
- Problem: Infinite video editing generates each next stream segment while applying newly arriving edit instructions.Unlike in-place editing, the output extends beyond the preceding segment rather than sharing its fixed temporal span.
- Challenges: The task requires faithful continuation beyond the input window and stable quality as generated segments become history for later edits.Unaffected entities should remain consistent, while accumulated errors must not destabilize generation.
- Approach: The paper builds a data-collection pipeline that synthesizes supervision for infinite video editing.The pipeline is designed specifically for repeatedly applying edits to ongoing streams.
- Approach: InfinityEdit is a lightweight edit adapter that equips a streaming video generator with unbounded editing ability.The adapter applies each edit when it arrives, while the frozen generator carries edited content forward across later chunks.
- Results: Extensive experiments show that InfinityEdit could effectively handle the infinite video editing task.The reported evaluation targets the task introduced in the paper.
2 Related Work
Related work spans video generation, in-place editing, long-video continuation, streaming editing, and multi-shot generation. Infinite video editing differs by applying edits to an ongoing stream while requiring continuation into future segments.
- Video Generation and Editing: Diffusion transformers and open foundational generators have advanced video quality, motion realism, and video length.The related generators include CogVideoX, Wan, HunyuanVideo, SkyReels-V4, Sora 2, and Seedance 2.0.
- Video Generation and Editing: Instruction- and reference-driven methods primarily perform in-place editing over existing video content.These methods modify a given clip rather than defining how an edit continues beyond its fixed temporal window.
- Long Video Generation and Streaming Video Generation: Long-video generation commonly performs autoregressive chunk continuation conditioned on previously generated frames.A central challenge is drift caused by conditioning on imperfect model outputs at inference.
- Streaming Editing and Multi-shot Video Generation: Streaming editing systems target live, frame-by-frame in-place editing, whereas multi-shot generators change prompts to introduce new content.The latter describes target scenes directly instead of specifying edits to existing content.
- Streaming Editing and Multi-shot Video Generation: Infinite video editing combines ongoing-stream continuation with edit requests that must affect future segments.The paper positions this target as related to, but distinct from, both streaming editing and multi-shot generation.
3 Preliminaries
The paper formalizes infinite video editing as generating a temporally subsequent segment that satisfies an edit instruction, then repeatedly using generated history for later edits. It identifies edit alignment, faithful continuation, and stability as core challenges.
- Task Definition: Traditional in-place editing rewrites a fixed-duration source clip, while infinite video editing generates a subsequent target segment.The target need not share the input duration or directly correspond to its frames.
- Task Definition: Each edited segment becomes the conditioning history for the next edit in the infinite regime.At round i, the most recent generated frames provide the history for the next generation step.
- Challenges: The task requires edit alignment, faithful continuation, and stable quality under repeated editing.These requirements cover realizing each instruction, continuing naturally without aligned source frames, and limiting error propagation.
- Challenges: Faithful continuation preserves edit-dependent content: style edits retain motion and camera movement, while camera edits retain appearance and subject identity.The preserved properties depend on the type of requested edit.
- Base Model: Helios-Distilled generates video chunk by chunk, keeps clean history as context, and compresses growing history into a constant-token multi-scale memory.The paper adopts it as the frozen backbone for infinite video editing.
4 Data Collection Pipeline
The data pipeline constructs and filters preceding-video, edit-instruction, and target-video triplets for training. It generates targets with edit-aware connection frames, then aligns formats and manually scores quality criteria.
- Triplet Construction: Training examples are triplets containing a preceding video, an edit instruction, and a target that continues the preceding video while applying the edit.Source videos are sampled from UltraVideo, and target videos are synthesized rather than directly reused.
- Edit-type-aware Target Video Generation: The pipeline synthesizes target videos with Wan2.2-I2V-A14B and presents the full process in Figure 1.The target-generation stage uses the constructed source video and edit instruction as inputs.
- Edit Instruction Generation: Basic edit types are expanded into concrete instructions using source-video captions and grouped entities.The instruction-generation process uses Gemini 3 Flash to produce detailed edit descriptions.
- Edit-type-aware Target Video Generation: Target generation anchors the first target frame to the source video’s last frame to preserve temporal connection.Appearance edits first modify the connection frame, while camera-view edits keep it unchanged before image-to-video generation.
- Data Post-processing: Post-processing resizes videos, matches input and output frame-count formats, and manually filters triplets using four quality criteria.The criteria include edit alignment, source-target consistency, target rationality, and source visual quality.
5 Methodology
InfinityEdit combines a training recipe and inference pipeline with a lightweight adapter attached to a frozen streaming video generator for repeated editing.
- InfinityEdit attaches a lightweight edit adapter to a frozen streaming video generator for infinite editing.The system includes separate training and inference components for repeated editing.
5.1 Model Architecture
The architecture preserves a streaming backbone while adding adapter blocks that anchor history, propagate signals causally, and inject edit instructions.
- The backbone generates one latent video chunk at a time using multi-scale history, an initial scene prompt, and an anchor frame.History tokens remain clean while the target chunk is denoised.
- Adapter blocks are inserted after transformer layers, while base weights remain frozen and outputs begin as identity mappings.Only the adapter blocks and shared noise-level module are trained, with zero-initialized output projections.
- History Cross-Attention for History Anchoring: History cross-attention uses leading-frame queries and history keys and values to anchor the denoising chunk.The result is added to the leading frames, which then pass cues forward within the chunk.
- Temporal Causal Self-Attention for Forward Propagation: Temporal causal self-attention operates over each spatial position’s temporal sequence and lets cues flow only from earlier frames to later ones.A causal mask and temporal rotary position encoding match streaming generation’s autoregressive order.
- Edit Cross-Attention for Instruction Injection: Edit cross-attention lets every current-chunk token query the edit instruction and adds the result to all denoising tokens.The preceding attention stages preserve history anchoring and temporal coherence.
5.2 Training Recipe
Training freezes the backbone and uses flow matching, corrupted histories, and noise-level sampling designed to teach editing across the inference schedule.
- The adapter is trained with a flow-matching objective while all backbone weights remain frozen.The clean target and Gaussian noise are interpolated, and the adapter predicts the corresponding velocity.
- History corruption replaces clean history latents with noisy versions during training to expose the adapter to imperfect feedback.Corruption is applied independently across long, middle, and short history scales, while clean history is also retained.
- Mixture-Gaussian Sampling of Noise Levels: The backbone’s logit-normal noise prior is replaced with an N-component Gaussian mixture because edits can target different visual scales.The mixture samples centers aligned with the discrete noise levels used for editing at inference.
- The first training phase uses uniform noise-level sampling and flat frame weights for balanced supervision.The second phase shifts sampling toward middle- and low-noise centers and increases weights for later frames to refine details.
5.3 Inference Pipeline
At each edit round, InfinityEdit ignites the edit in one chunk, then resumes backbone-only continuation while resetting the anchor to the edited stream.
- The adapter is activated only for the first chunk after an edit instruction, while later chunks use the frozen backbone.The ignition chunk uses a single-stage Euler schedule; subsequent chunks use the backbone’s pyramid scheduler.
- A near-zero denoising step helps the ignition chunk express edits targeting fine details.The adapter refines an almost-clean latent, and the same low-noise step is included during training.
- History-Window Sliding and Anchor Resetting for Infinite Editing: The sliding history window retains only recent frames, keeping memory cost constant as edits accumulate.This preserves the backbone’s original history-window design during repeated editing.
- History-Window Sliding and Anchor Resetting for Infinite Editing: After each ignition chunk, the anchor resets to its first edited frame and remains fixed until the next edit instruction.The reset is intended to keep later backbone continuations in the edited stream.
6 Experiments
InfinityEdit is evaluated on sequential editing through automatic metrics, VLM judgments, round-wise stability, qualitative comparisons, and long-horizon generation. Across these tests, it preserves edit faithfulness, source content, temporal coherence, and stream continuity as edits accumulate.
- Experimental Settings: The evaluation uses a 200-sample sequential-editing task with three editing rounds, comparing representative Pure Backbone, In-Place Editing, and Prompt Switching baselines.The study reports VBench metrics and a VLM-as-Judge evaluation; no prior method directly targets infinite video editing.
- VBench Metrics: InfinityEdit achieves the best Camera Motion and Temporal Flickering results while remaining close to the best Motion Smoothness, with an approximately +0.22 gain over the second-best Camera Motion method.Temporal Flickering is measured in the third round to assess stability after accumulated edits.
- VLM-as-Judge Evaluation: InfinityEdit ranks first on Edit Faithfulness, Visual Quality, Scene Identity Preservation, and Cross-Edit Coherence in the VLM-as-Judge evaluation.In-Place Editors score lower on cross-edit coherence, while Prompt Switching methods score below 2.0 on preservation because they lack source-video grounding.
- Stability across Editing Rounds: Across editing rounds, source-conditioned baselines lose 0.02 to 0.04 in Aesthetic Quality, whereas InfinityEdit remains nearly flat with ∆≈0.The comparison excludes Prompt Switching methods because they do not take source video as input.
- Stability across Editing Rounds: InfinityEdit remains both high and stable in edit faithfulness, staying near 3.8 from the first edit to the last with standard deviation 0.023.The method is the only one reported as maintaining high, stable performance across rounds; competing families show distinct weaknesses.
- Qualitative and Long-Video Evaluation: Qualitative cases show that InfinityEdit applies each instruction at the intended stream position, preserves continuity across segment boundaries, and remains stable beyond 1000 frames.The examples include camera-movement and style edits on scenery, entity transformation on a human-centric video, and persistence of edited attributes during later instructions.
7 Conclusion
The paper introduces infinite video editing for applying edits to ongoing streams beyond fixed clips. InfinityEdit uses a lightweight adapter with a frozen streaming generator and remains faithful and stable as edits accumulate.
- InfinityEdit applies edits to an ongoing video stream beyond a fixed input clip while generating temporally subsequent segments.
- The adapter uses history cross-attention, temporal causal self-attention, and edit cross-attention to ground, propagate, and inject edits.
- During inference, the adapter activates when an edit arrives, while the frozen generator carries the edited stream forward with a sliding history window and reset anchor frame.
- Experiments show more faithful edit following, continuation without frame-wise rewriting, and stable quality as edits accumulate over long sequences.
8 Future Work
The editor currently accepts only natural-language instructions, and edit-boundary transitions can remain abrupt. Future work targets reference inputs and smoother transitions between successive edits.
- The editor currently uses only natural-language instructions, limiting direct reference-based control over target appearance.
- Adding image or video references could improve control, especially for edits that are difficult to describe verbally.
- Although continuation remains coherent, transitions to specific-type instructions can still be abrupt at edit boundaries.
- Smoother transitions between successive edits remain an important direction for future work.