Source-linked AI summary
ChronoEdit: Towards Temporal Reasoning for Image Editing and World Simulation
Jay Zhangjie Wu, Xuanchi Ren, Tianchang Shen, Tianshi Cao, Kai He, Yifan Lu, Ruiyuan Gao, Enze Xie, Shiyi Lan, Jose M. Alvarez, Jun Gao, Sanja Fidler, Zian Wang, Huan Ling
TL;DR
ChronoEdit addresses the gap in physical consistency for image editing, which is especially important for world simulation. It reframes editing as two-frame video generation and uses temporary reasoning tokens to guide plausible transformations. On PBench-Edit and general editing benchmarks, it achieves state-of-the-art or leading performance among the reported baselines.
Problem
Image-editing models often fail to preserve existing objects, properties, and geometry while applying the requested change, limiting physical consistency for simulation tasks.
Method
ChronoEdit treats input and edited images as consecutive video frames and jointly denoises intermediate temporal reasoning tokens to guide physically coherent edits.
Results
ChronoEdit-14B-Think reaches a state-of-the-art overall score of 4.53 on PBench-Edit, while ChronoEdit-14B scores 4.42 on general-purpose editing.
Takeaways & Limitations
ChronoEdit preserves coherence between input and edited outputs while producing plausible transformations and performs strongly across open-source and proprietary comparisons.
Abstract
from arXiv · showhide
Recent advances in large generative models have greatly enhanced both image editing and in-context image generation, yet a critical gap remains in ensuring physical consistency, where edited objects must remain coherent. This capability is especially vital for world simulation related tasks. In this paper, we present ChronoEdit, a framework that reframes image editing as a video generation problem. First, ChronoEdit treats the input and edited images as the first and last frames of a video, allowing it to leverage large pretrained video generative models that capture not only object appearance but also the implicit physics of motion and interaction through learned temporal consistency. Second, ChronoEdit introduces a temporal reasoning stage that explicitly performs editing at inference time. Under this setting, target frame is jointly denoised with reasoning tokens to imagine a plausible editing trajectory that constrains the solution space to physically viable transformations. The reasoning tokens are then dropped after a few steps to avoid the high computational cost of rendering a full video. To validate ChronoEdit, we introduce PBench-Edit, a new benchmark of image-prompt pairs for contexts that require physical consistency, and demonstrate that ChronoEdit surpasses state-of-the-art baselines in both visual fidelity and physical plausibility. Project page for code and models: https://research.nvidia.com/labs/toronto-ai/chronoedit
1 INTRODUCTION
ChronoEdit targets physical consistency in image editing for simulation, where edits must preserve scene objects and geometry while reflecting intended changes. It reframes editing as two-frame video generation and adds temporal reasoning to produce plausible transformations, evaluated with PBench-Edit.
- Physical consistency is essential for simulation-oriented editing because unintended object hallucinations or geometry changes can misrepresent scenes and compromise downstream systems.
- ChronoEdit repurposes pretrained video generative models by treating the input and edited images as consecutive frames, leveraging temporal priors for object fidelity.
- Temporal reasoning denoises intermediate frames as reasoning tokens that plan how an edit unfolds, improving physical plausibility while preserving temporal alignment with the input.
- PBench-Edit evaluates general-purpose and physically or temporally consistent edits, and ChronoEdit achieves state-of-the-art results while narrowing the gap with proprietary systems.
- The framework is presented as a foundation model, a video-to-editing adaptation, a temporal reasoning inference stage, and a benchmark for world simulation applications.
2 RELATED WORK
Prior image-editing work uses large foundation models and video-derived key frames to improve editing and temporal coherence. Related approaches also synthesize intermediate video frames without training, then select the frame that best satisfies the edit.
- Recent image-editing systems target instruction alignment, multi-turn editing, unified generation and editing, or precise high-fidelity edits through large foundation-model architectures.
- Bagel, UniReal, and OmniGen use video-derived key frames to create temporally coherent image pairs for editing.
- Rotstein et al. use a pretrained image-to-video diffusion model to synthesize intermediate frames and select the frame that best satisfies the edit without training.
3 CHRONOEDIT
ChronoEdit repurposes pretrained video generators for physically consistent image editing by treating image pairs as two-frame videos and using intermediate temporal reasoning. Its inference procedure adds reasoning tokens only during early denoising steps, then completes editing efficiently and can be accelerated through distillation.
- 3.2 RE-PURPOSING VIDEO GENERATIVE MODELS FOR EDITING: ChronoEdit reframes image editing as a two-frame video generation problem, using pretrained video models’ temporal priors to maintain consistency between source and target images.The input image is the first frame and the edited image is the last frame.
- 3.2 RE-PURPOSING VIDEO GENERATIVE MODELS FOR EDITING: Temporal reasoning tokens are noisy intermediate latent frames jointly denoised with the target to guide plausible transitions and preserve object identity, geometry, and physical coherence.They encourage trajectory-based editing instead of abrupt single-step regeneration.
- 3.2 RE-PURPOSING VIDEO GENERATIVE MODELS FOR EDITING: ChronoEdit unifies image-pair and video training by treating pairs as two-frame videos while using intermediate video frames as reasoning tokens when full videos are available.The target frame is repeated four times to match the video VAE’s temporal compression, while reasoning tokens remain optional at inference.
- 3.2 RE-PURPOSING VIDEO GENERATIVE MODELS FOR EDITING: The training corpus contains 1.4M synthetic videos designed to provide diverse scene evolutions while separating object dynamics from camera motion.The curation emphasizes static-camera and world-simulation scenarios, including egocentric driving scenes.
- 3.3 INFERENCE WITH TEMPORAL REASONING: Inference uses temporal reasoning tokens only in the first Nr denoising steps, carries forward partially denoised target latents, and then completes sampling without those tokens.Setting r = 0 or Nr = 0 recovers standard sampling without temporal reasoning.
- 3.4 FEW-STEP DISTILLATION FOR FAST INFERENCE: DMD distillation trains an 8-step student model to improve inference speed while maintaining prompt-following ability and image-editing quality.The objective uses teacher and trainable fake score estimates under a forward diffusion process.
4 EXPERIMENTS
ChronoEdit is evaluated across general-purpose editing and physically grounded world-simulation tasks, including quantitative, qualitative, efficiency, and temporal-reasoning analyses. It achieves strong overall performance, particularly on action-oriented edits, while temporal reasoning improves physical consistency and lightweight variants preserve much of the quality at lower cost.
- General-Purpose Image Editing Results: 4.42 overall: ChronoEdit-14B achieves the highest score on the ImgEdit Basic-Edit Suite, surpassing FLUX.1 Kontext [Dev] by +0.90 overall.It also matches or outperforms Qwen-Image across tasks, with stronger results on background change and action/motion edits.
- Efficiency and Model Variants: ChronoEdit-Turbo runs in 5.0s versus 30.4s for ChronoEdit-14B while scoring only 0.3 points lower, and ChronoEdit-2B performs on par with ChronoEdit-14B-Turbo.The Turbo comparison was measured on two Nvidia-H100 GPUs.
- World Simulation Editing Results: ChronoEdit-14B achieves 4.01 Action Fidelity on PBench-Edit, exceeding Qwen-Image at 3.76 and FLUX.1 Kontext [Dev] at 2.88 while maintaining competitive identity and anatomical coherence.The action-fidelity dimension directly reflects physical consistency in real-world interactions.
- World Simulation Editing Results: 4.53 overall: ChronoEdit-14B-Think with Nr = 10 achieves a new state-of-the-art score on PBench-Edit, including 4.31 Action Fidelity.ChronoEdit-2B-think with Nr = 10 nearly matches ChronoEdit-14B-Think.
- Qualitative Evaluation: Qualitative evaluations show faithful instruction following, preserved scene structure and fine details, and generalization across Physical-AI tasks such as self-driving dynamics and robotic manipulation.Retaining reasoning tokens throughout denoising also reveals plausible intermediate trajectories for edits such as adding a cat or placing a cake by hand.
- Reasoning Timestep: Using Nr = 10 reasoning steps instead of Nr = 50 reduces runtime from 55.5s to 35.3s while achieving comparable performance to reasoning across the full trajectory.This setting adds only 4.9s compared with omitting temporal reasoning.
5 CONCLUSION
ChronoEdit is presented as a foundation model designed to enforce physical consistency in image editing. It combines a pretrained video diffusion model with temporal reasoning and achieves state-of-the-art performance among open-source models.
- ChronoEdit combines a pretrained video diffusion model with a temporal reasoning inference stage to preserve coherence and produce plausible edits.
- ChronoEdit achieves state-of-the-art performance among open-source image-editing models.
A RELATED WORK
Image editing has progressed from domain-specific GANs to diffusion, instruction-tuned, and multimodal foundation models. Recent work also uses video priors to improve temporal coherence in image editing.
- GAN-based editing can produce photorealistic outputs in constrained domains but struggles with out-of-domain edits and requires domain-specific training.
- Training-free diffusion methods enable text-guided edits through noise injection, cross-attention guidance, or image inversion.
- Instruction-tuned and multimodal foundation models learn open-domain edits from image-text conditioning and large-scale architectures.
- Video-prior methods create temporally coherent image pairs or synthesize intermediate frames to support image editing.
B ADDITIONAL RESULTS
Additional qualitative comparisons show that ChronoEdit produces coherent and physically consistent edits against baseline methods.
- Additional qualitative comparisons further highlight ChronoEdit's effectiveness in producing coherent and physically consistent edits.
C ADDITIONAL ABLATION STUDY
The ablations examine pretrained initialization, reasoning horizons, and image-pair encoding choices. They indicate that pretrained weights improve convergence, short reasoning is efficient, and joint encoding is preferred despite comparable reconstruction quality.
- Pretrained image-to-video initialization enables faster and more stable convergence than training from scratch.
- The qualitative examples evaluate reasoning steps and timesteps, including PBench-Edit cases using 10 temporal reasoning steps.
- Jointly encoding concatenated frames achieves 39.82dB PSNR versus 40.21dB for individual encoding, but is selected because its latents better match native video latents.
D ADDITIONAL DETAILS ON VIDEO DATA CURATION
The video-data curation process captions paired frames as image-editing instructions while treating them as the endpoints of a video transformation. Ablations examine initialization, reasoning duration, and the diversity of reference scenarios.
- Captioning uses the first video frame as input and the 40th and 80th frames as target frames, with Qwen2.5-VL-72B-Instruct generating instructions.
- Video-pretrained initialization converges faster and more stably than training from scratch, while reasoning for Nr = 10 of N = 50 steps performs comparably to reasoning across the full trajectory.
- The captioning prompt treats paired images as the first and last frames of a video and asks for a self-contained instruction describing the transformation.
- Instructions prioritize the most prominent changes, describe appearance and spatial location, and remain in English, self-contained, and under 200 words.
- PBench-Edit’s reference-image gallery spans diverse, challenging real-world interaction scenarios.