PaperlayerView exact revision on arXiv

Source-linked AI summary

From Statics to Dynamics: Physics-Aware Image Editing with Latent Transition Priors

Liangbing Zhao, Le Zhuo, Sayak Paul, Hongsheng Li, Mohamed Elhoseiny

arXiv:2602.21778v2cs.CV

TL;DR

Instruction-based editors often fail to preserve physical plausibility because discrete image-pair mappings underspecify transition dynamics. This paper models editing as predictive physical state transitions, introduces the PhysicTran38K video dataset, and builds PhysicEdit with textual-visual dual thinking. PhysicEdit achieves state-of-the-art performance among evaluated open-source methods and remains comparable to leading proprietary models.

  • Problem

    Existing editing models can achieve semantic alignment while failing to render physically plausible outcomes for complex causal dynamics such as refraction.

  • Method

    The paper reformulates editing as physical state transitions, constructs PhysicTran38K video supervision, and proposes PhysicEdit with textual-visual dual thinking and learnable transition queries.

  • Results

    PhysicEdit achieves state-of-the-art performance among evaluated open-source models while performing comparably to leading proprietary models.

  • Takeaways & Limitations

    Video-based transition priors and physically grounded reasoning provide a basis for physics-aware editing while preserving standard single-image inference.

  • Takeaways & Limitations

    The authors note that more realistic manipulated images could be misused to create misleading content or misinformation that is harder to distinguish from reality.

Abstract

from arXiv · show

Instruction-based image editing has achieved remarkable success in semantic alignment, yet state-of-the-art models frequently fail to render physically plausible results when editing involves complex causal dynamics, such as refraction or material deformation. We attribute this limitation to the dominant paradigm that treats editing as a discrete mapping between image pairs, which provides only boundary conditions and leaves transition dynamics underspecified. To address this, we reformulate physics-aware editing as predictive physical state transitions and introduce PhysicTran38K, a large-scale video-based dataset comprising 38K transition trajectories across five physical domains, constructed via a two-stage filtering and constraint-aware annotation pipeline. Building on this supervision, we propose PhysicEdit, an end-to-end framework equipped with a textual-visual dual-thinking mechanism. It combines a frozen Qwen2.5-VL for physically grounded reasoning with learnable transition queries that provide timestep-adaptive visual guidance to a diffusion backbone. Experiments show that PhysicEdit improves over Qwen-Image-Edit by 5.9% in physical realism and 10.1% in knowledge-grounded editing, setting a new state-of-the-art for open-source methods, while remaining competitive with leading proprietary models.

1. Introduction

Instruction-based editing increasingly targets complex, reasoning-based changes, but models often preserve semantic content while violating physical laws. The paper reframes editing as physical state transition prediction, introduces video transition supervision, and proposes PhysicEdit for physically faithful single-image editing.

  • Motivation: Complex edits such as inserting a straw into water expose failures in rendering optical refraction despite correct object and location semantics.Models tend to preserve the straw’s rigid geometry instead of depicting its disjointed or bent appearance at the water surface.
  • Problem Reformulation: The paper models editing as a physical state transition in which the source image is an initial state and the instruction triggers a subsequent state governed by physical laws.This replaces a static mapping between independent images with a formulation involving an external interaction or trigger.
  • Dataset: Video provides intermediate evidence of state evolution that paired-image supervision lacks, motivating PhysicTran38K for interaction-driven, law-governed transitions.The dataset spans five physical domains, 16 sub-domains, and 46 transition types.
  • Method: PhysicEdit uses textual-visual dual thinking to combine physically grounded reasoning with implicit visual thinking and video-learned transition priors.The framework is designed to learn from video trajectories while remaining compatible with single-image inference.
  • Dataset: PhysicTran38K contains approximately 38k video-instruction pairs organized by hierarchical physics categories.
  • Results: PhysicEdit achieves state-of-the-art performance among evaluated open-source models and performs comparably to leading proprietary models.

2. Related Works

Related work has progressed from diffusion-based editing toward instruction-tuned and unified multimodal models. The paper situates its contribution within this evolution while emphasizing physics-aware transition modeling.

  • Diffusion-Based Editing: Diffusion-based editors manipulate cross-attention or invert latents, but often lack fine-grained control under complex structural changes.
  • Instruction-Tuned and Multimodal Models: Recent instruction-tuned and unified multimodal models use multimodal reasoning capabilities to support more complex image editing.

3. Method

PhysicEdit reformulates editing as a physical state transition and learns transition priors from videos to guide single-image diffusion editing. Its data pipeline spans structured generation, viewpoint and physics verification, and constraint-aware annotation, while its framework combines textual reasoning with timestep-adaptive visual guidance.

  • Problem formulation: Physics-aware editing models the source image as an initial state and the instruction as an interaction trigger governing evolution under physical laws.This replaces the conventional black-box mapping from source image and instruction to target image.
  • Problem formulation: Paired-image supervision supplies boundary conditions but leaves transition dynamics underspecified, whereas videos constrain intermediate state evolution.The formulation motivates video supervision and latent transition representations for inference without intermediate frames.
  • Physics-driven data construction: The dataset pipeline generates structured videos, filters camera movement and physical correctness, and annotates retained clips with transition reasoning and intermediate keyframes.Constraint-aware annotation excludes contradicted or unknown principles and yields approximately 38,000 samples.
  • Physics-driven data construction: PhysicTran38K organizes physical transitions across five domains, 16 sub-domains, and 46 transition types, including refraction, melting, and germination.The dataset uses hierarchical categories and specialized object pools to cover physical dynamics.
  • PhysicEdit framework: PhysicEdit combines frozen Qwen2.5-VL reasoning with learnable transition queries that reconstruct missing visual evolution from source images, instructions, and reasoning.The transition queries are trained against visual features from intermediate keyframes and provide visual guidance at inference.
  • PhysicEdit framework: Timestep-aware modulation mixes structure- and texture-level transition guidance to match diffusion’s coarse-to-fine generation trajectory.Higher-noise steps emphasize global structure, while lower-noise steps refine local texture.

4. Experiments

PhysicEdit is evaluated on physical-realism and physics-related knowledge benchmarks against open-source and proprietary models, with quantitative, qualitative, and ablation analyses. Results show strong overall performance and complementary benefits from textual reasoning and visual transition guidance.

  • Main Results: PhysicEdit achieves an overall physical-realism score of 64.86, establishing a new state-of-the-art among open-source models.It improves all physical dimensions over Qwen-Image-Edit-2509.
  • Main Results: Light Source Effects increases from 61.19 to 76.16, Deformation rises by 12.0 points to 60.76, and Causality improves from 48.95 to 59.23.Improvements are largest in categories requiring implicit dynamics, with additional gains in Refraction and Local State Transition.
  • Main Results: PhysicEdit achieves an overall KRISBench score of 72.16, surpassing all open-source baselines and proprietary models such as Gemini-2.0 and Doubao.Temporal Perception rises from 71.73 to 76.13, while Natural Science improves by 11.9 points to 71.57.
  • Qualitative Analysis: Qualitative PICABench results span Optics, Mechanics, and State Transitions, where PhysicEdit produces more physically plausible and visually natural edits than comparison models.In a lamp-switching case, it renders global illumination decay and shadow propagation rather than merely darkening the image.
  • Ablation Studies: The full textual-visual dual-stream model achieves the highest performance across all ablation metrics, whereas single streams specialize in different physical dimensions.Physically-Grounded Reasoning raises Mechanics to 57.43, while Implicit Visual Thinking raises Optics to 68.05 but lowers Mechanics to 52.38.
  • Ablation Studies: DINO-only features achieve the highest GST of 70.16, while VAE features improve LST to 60.40 but lack global coherence.The full model relaxes DINO’s structural rigidity to support substantial physical deformations.
  • Ablation Studies: Hard switching between DINO and VAE underperforms the full fusion model because an abrupt timestep cutoff introduces a discontinuity in the guidance signal.The comparison uses DINO for t ∈[1, 0.7] and VAE for t < 0.7.

5. Conclusion

The paper formulates image editing as continuous Physical State Transition and introduces video-based transition supervision through PhysicTran38K and PhysicEdit. The reported results support transition-centric supervision as a basis for physically plausible editing and future physics-aware generation research.

  • Conclusion: The paper models image editing as a continuous Physical State Transition rather than a static mapping between images.It presents this as a paradigm shift for physics-aware editing.
  • Conclusion: PhysicTran38K provides video-based supervision, while PhysicEdit combines physically grounded reasoning with latent transition queries for single-image inference.The framework is designed to leverage video supervision without requiring intermediate states at inference.
  • Conclusion: The method significantly outperforms existing baselines in physical plausibility, supporting the effectiveness of transition-centric supervision.The paper positions the framework as a foundation for future physics-aware generation research.

Impact Statement

PhysicEdit and PhysicTran38K aim to improve the physical plausibility of generative editing. The impact statement highlights benefits for creative industries, virtual prototyping, and education alongside misuse risks involving misleading content and misinformation.

  • Benefits: PhysicEdit and PhysicTran38K improve the visual realism of edited content by enforcing adherence to physical laws.The stated application areas include creative industries, virtual prototyping, and education.
  • Risks: More realistic manipulated images could be misused to create misleading content or misinformation that is harder to distinguish from reality.The paper advocates responsible use and research into detection methods for synthetic physical inconsistencies.

A. Detailed Dataset Statistics and Taxonomy

PhysicTran38K organizes filtered video-instruction data into a balanced taxonomy spanning five physical domains, 16 sub-domains, and 46 transition types. Its categories cover mechanical, biological, optical, thermal, and material-state changes.

  • 38,620 high-quality video-instruction pairs remain after filtering, distributed relatively evenly across five primary physical domains.
  • The taxonomy contains 5 primary domains, 16 intermediate sub-domains, and 46 distinct Transition Types.
  • Mechanical State: Mechanical State covers positional, motion, structural, integrity, surface-condition, and mechanical-property transitions.
  • Biological State: Biological State covers vital, growth, functional, health, and surface-biological changes.
  • Optical, Thermal, and Material States: Optical, thermal, and material categories represent light interactions, heat and phase changes, and intrinsic material-property changes.

B.1. Model Specifications and Assets

The implementation uses established diffusion, reasoning, data-construction, and visual-encoding components. PhysicEdit is initialized from Qwen-Image-Edit-25092, while visual encoders provide structural and texture information.

  • PhysicEdit is built on Qwen-Image-Edit-25092, whose diffusion backbone remains frozen during visual encoder training.
  • GPT-5-mini generates prompts, transition principles, and verification steps for the data-construction pipeline.
  • DINOv2-Base with Registers captures structural semantics, while the Qwen-Image-Edit VAE captures fine-grained texture aligned with the backbone latent space.
  • Figures 5 and 6 provide detailed illustrations of Mechanical, Biological, Thermal, Optical, and Material data.

B.2. Implementation Details of Implicit Visual Thinking

Implicit Visual Thinking distills physical dynamics from uniformly sampled video keyframes into learnable transition queries. The queries are conditioned on both source-image context and physically grounded textual reasoning.

  • Transition Query Injection and Parameterization: K learnable transition queries are appended after the edit instruction and reasoning text, then processed by frozen Qwen2.5-VL.
  • Video Supervision Strategy: Six intermediate keyframes are uniformly sampled from each video clip to supervise physical transitions.
  • Video Supervision Strategy: Uniform sampling is sufficient in this setting because more complex motion-based or attention-based strategies provide negligible performance gains.
  • Feature Extraction and Transition Delta: Frozen DINOv2 and VAE encoders extract frame representations, which a Perceiver Resampler-like mechanism compresses for alignment with one-dimensional transition queries.
  • The surrounding editing literature includes diffusion-based editing and video-informed reasoning approaches that motivate this implementation.

D. General Editing and Reasoning Editing Results

PhysicEdit preserves and improves general image-editing performance while producing much larger gains on reasoning-based editing. Its strongest improvements occur on temporal and causal dimensions, where dynamic understanding is required.

  • General Image Editing: The ImgEdit-Bench overall score rises from 4.35 to 4.40, while GEdit-Bench-EN G O increases from 7.56 to 7.87.
  • Reasoning-Based Editing: RISE-Bench overall performance more than doubles from 8.9 to 18.6 over the base Qwen-Image-Edit.
  • Reasoning-Based Editing: RISE-Bench Temporal improves from 4.7 to 21.2, and Causal improves from 10.0 to 23.3.
  • Reasoning-Based Editing: PhysicEdit attains the highest overall performance among evaluated open-source models on RISE-Bench.

E. Analysis and Discussion

The analysis compares explicit frame-rollout and implicit latent-transition approaches, and examines whether transition-focused data supports principle acquisition. PhysicEdit outperforms ChronoEdit on both benchmarks, while its localized adaptation avoids the trade-off observed with explicit trajectory fine-tuning.

  • Explicit vs. Implicit Visual Thinking: ChronoEdit’s explicit intermediate-frame synthesis is computationally intensive and can accumulate errors over long horizons.Rollout degradation may produce background distortion or object disappearance in final results.
  • Explicit vs. Implicit Visual Thinking: Fine-tuning ChronoEdit on PhysicTran38K raises PICABench to 61.43 but lowers KRISBench to 69.59, indicating a benchmark trade-off.PhysicEdit instead localizes transition-specific learning in lightweight queries and adapters while keeping the VLM frozen.
  • Principle Acquisition vs. Instruction Overfitting: PICA-100K fine-tuning yields only marginal improvements, whereas principle-driven construction exposes optical paths and material deformations as pure state transitions.The paper describes this contrast as evidence for principle acquisition and generalization to knowledge-grounded benchmarks.

F. More Qualitative Results

The qualitative-results section presents additional PICABench examples covering refraction, light propagation, and light-source effects. It also references the system prompts and principle-generation materials used to support physics-grounded video instructions and frame-consistency evaluation.

  • Qualitative Comparisons: Figures 8, 9, and 10 provide additional PICABench qualitative comparisons for light propagation, light-source effects, and refraction.Figure 8 covers light propagation, Figure 9 light-source effects, and Figure 10 refraction.
  • Physics-Grounded Instruction Generation: The physics-grounded video-instruction prompt requires a fixed camera pose and physically consistent evidence, environment, and driver.It also restricts outputs to a single transition and observable mid-transition evidence.
  • Principle Verification: The principle-generation materials take a prompt, transition name, state category, and maximum-principle cap as inputs for video-frame verification.They build principles from commonly accepted, domain-general knowledge for a video generative model.
Loading 2602.21778v2…