PaperlayerView exact revision on arXiv

Source-linked AI summary

VideoPhysEdit: Physical Counterfactual Video Editing via Rigid-Body Physical Scene Reconstruction

Conghan Yue, Yuanjie Chen, Yue Han, Ya Gao, Yunyan Xiao, WeiYao Zhang, Zhineng Chen

arXiv:2609.35134v1cs.CV

TL;DR

Physical video edits can change later motion and interactions, but those consequences are less explored and lack paired data and dedicated metrics. VideoPhysEdit reconstructs a simulated physical scene and uses counterfactual trajectories to guide editing; it achieves higher physical edit accuracy than compared methods while retaining competitive visual fidelity.

  • Problem

    Physical edits can alter subsequent motion and interactions, yet these consequences are less explored and paired factual-counterfactual data and dedicated physical-edit metrics are lacking.

    Source: §1 · p. 2

  • Method

    VideoPhysEdit is a training-free pipeline for rigid-body scenes that reconstructs a scene reproducing observed motion under simulation and uses simulated counterfactual trajectories to guide video generation.

    Source: §1 · p. 2

  • Results

    VideoPhysEdit achieves substantially higher physical edit accuracy than open-source methods and commercial models while maintaining competitive visual fidelity; its Physical Edit Score is 0.376, the only positive score among compared methods.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Qualitative comparisons show that VideoPhysEdit applies to real-world videos and better depicts edit-induced downstream motion and interactions than compared methods.

    Source: §6 · p. 10

    The current method targets static-camera rigid-body scenes with planar supports and sphere or box geometry, excluding moving cameras, nonplanar supports, complex geometry, and articulated or actively controlled agents.

    Source: D Limitations and Future Work · p. 46

Abstract

from arXiv · show

Video editing has advanced substantially in recent years, with methods increasingly accounting for the visual consequences of edits, such as changes to shadows and occlusions. However, the physical consequences of edits, including changes to subsequent motion and interactions, remain less explored. We formulate this problem as physical counterfactual video editing (PCVE), which aims to generate a counterfactual video depicting the resulting motion and interactions given a source video, a physical edit, and its execution frame. PCVE is challenging because it requires understanding scene physics and inferring the downstream motion and interactions induced by a physical intervention, while paired factual and counterfactual data and dedicated evaluation metrics are lacking. We introduce VideoPhysEdit, a new training-free pipeline for PCVE in rigid-body scenes. It makes physical reasoning explicit through a novel physical scene reconstruction method that recovers a scene reproducing the observed motion and interactions under simulation, enabling the pipeline to apply physical edits as interventions and use the resulting trajectories to guide counterfactual video generation. We further construct PCVE-RigidBench, a synthetic benchmark with paired source and counterfactual target videos and physical ground truth, and introduce the Physical Edit Score. VideoPhysEdit achieves substantially higher physical edit accuracy than open-source methods and commercial models while maintaining competitive visual fidelity. Its Physical Edit Score is 0.376, the only positive score among the compared methods. Qualitative comparisons on real videos further show that VideoPhysEdit applies to real-world scenes and better depicts the downstream motion and interactions induced by the edits than the compared methods. Code: https://github.com/Hammour-steak/VideoPhysEdit

1 Introduction

Physical counterfactual video editing (PCVE) aims to show how a physical intervention changes subsequent motion and interactions while preserving the source video’s history before the intervention. VideoPhysEdit reconstructs a simulated physical scene to guide counterfactual generation, and outperforms compared methods on physical edit accuracy.

Problem

  • PCVE generates a counterfactual video that preserves the factual history before an intervention and depicts the evolution induced by the altered physical conditions afterward.Edits include object insertion or removal, motion-state changes, and changes to physical parameters.Source: §1 · p. 2

Motivation

  • The task remains challenging because methods must infer downstream physical effects, while paired factual and counterfactual data and metrics for physical correctness are lacking.Conventional video-editing metrics do not measure whether resulting motion and interactions are physically correct.Source: §1 · p. 2

Method

  • VideoPhysEdit reconstructs a physical scene whose simulation reproduces observed motion and interactions, then applies the edit as an intervention and uses simulated trajectories to guide video generation.The pipeline is training-free and targets rigid-body scenes.Source: §1 · p. 2

Results

  • VideoPhysEdit achieves substantially higher physical edit accuracy than open-source methods and commercial models while maintaining competitive visual fidelity; its Physical Edit Score is 0.376, the only positive score among compared methods.It reduces trajectory error by 54.0% relative to the strongest competing method.Source: §1 · p. 2
  • Qualitative comparisons show that VideoPhysEdit applies to real videos and better depicts edit-induced downstream motion and interactions than compared methods.Source: §1 · pp. 2–3

2 Related Work

Prior work addresses visual editing, physics-aware editing, physical scene reconstruction, and video benchmarks, but the paper distinguishes PCVE by its focus on diverse interventions and their downstream physical consequences.

2.1 Physics-Aware Video Editing

  • Physics-aware editing methods use physical models or reasoning, but existing approaches rely on predefined models, external 3D information, multi-view capture, or specialize in object removal rather than diverse interventions.VOID uses masks and paired synthetic removal data, but does not cover insertion, motion-state modification, or physical-parameter editing.Source: §2.1 · p. 3

2.2 4D Reconstruction and Physical Modeling

  • Physical reconstruction methods recover geometry, motion, or physical quantities, while recent simulation-ready methods differ in their representations, training assumptions, or scene-specific collision modeling.VideoPhysEdit refines a reconstructed scene by matching simulated and observed masks for counterfactual generation.Source: §2.2 · p. 3

2.3 Physical Video Benchmarks

  • Existing benchmarks assess physical realism, editing fidelity, removal effects, or counterfactual predictions with the same event type; PCVE-RigidBench evaluates interventions against counterfactual videos and physical ground truth.Source: §2.3 · p. 4

3 Method

The method defines physical counterfactual editing around an intervention at a specified frame, then reconstructs a physical scene to simulate the resulting evolution. Its pipeline uses that simulated counterfactual motion to guide video generation.

3.1 Problem Formulation

  • A physical edit changes object presence, motion state, or physical parameters; the intervention occurs at a specified frame, with factual evolution preserved beforehand and altered evolution shown afterward.Source: §3.1 · p. 4

3.2 VideoPhysEdit Overview

  • VideoPhysEdit tracks objects, analyzes motion into stable intervals and transition episodes, reconstructs geometry and a motion prior, and optimizes states, parameters, and collision proxies to match observed motion and interactions.It then applies the edit in the reconstructed scene and uses simulated counterfactual trajectories with an edited reference image to guide generation.Source: §3.2 · p. 4

3.3 Physical Scene Reconstruction

  • Because different geometries, 3D states, and physical parameters can explain the same 2D observations, reconstruction seeks a scene whose simulation reproduces observed motion and interactions.The scene representation includes visual meshes, collision proxies, a camera, physical parameters, and simulated object states.Source: §3.3 · p. 4

5. Physical Inversion �����

VideoPhysEdit reconstructs a physical scene whose simulation reproduces observed motion, applies an edit as an intervention, and uses the resulting motion and appearance controls to generate a counterfactual video.

5. Physical Inversion �����

  • Object masks, motion observations, canonical scenes, and anchor scenes establish shared coordinates for reconstructing object poses and motion.The motion is organized into stable intervals and transitions, then lifted into a continuous 6DoF motion prior.Source: §5 · pp. 5–6
  • Rigid-body constraints from support relations and the motion prior estimate initial states and physical parameters that reproduce observed motion under simulation.Stable intervals constrain force balance, friction, rolling, and energy; contact events constrain momentum balance, restitution, and friction.Source: §5 · p. 6
  • Simulation search progressively adds stable intervals or transitions, then refines the best full-sequence simulation into an executable physical scene.The search compares simulated visible masks with observed masks.Source: §5 · p. 6

3.4 Physical Intervention and Counterfactual Video Generation

  • At the execution frame, a parsed Add, Delete, or Set operation changes the factual state, and the subsequent simulation yields counterfactual trajectories.Projected trajectories and an edited reference image guide a pretrained video generator in producing the counterfactual continuation.Source: Figure 2 · p. 5, §3.4 · p. 6

4 PCVE-RigidBench

PCVE-RigidBench pairs source videos and physical edits with counterfactual targets and physical ground truth, while Physical Edit Score measures improvement over leaving the source unchanged.

4 PCVE-RigidBench

  • The benchmark contains 129 editing tasks across 20 simulated rigid-body scenes, pairing each source video and edit with a counterfactual target and physical ground truth.Scenes cover impacts, rebounds, rolling, sliding, falls, and collision chains; edits include object insertion or removal and changes to velocity, mass, friction, or restitution.Source: §4 · pp. 6–7
  • Physical Edit Score evaluates trajectory-error reduction against the counterfactual target relative to the unchanged source video, for source objects whose motion or presence changes.Tasks include interventions at the first frame or partway through the video, with quantitative and qualitative edit descriptions.Source: §4 · pp. 6–7
  • A score of one means zero scored error against the target, zero means no improvement over the unchanged source, and a negative score means worse performance than that baseline.Source: §4 · p. 7

5 Experiments

VideoPhysEdit is evaluated against open-source and commercial methods on PCVE-RigidBench, with additional qualitative and pipeline analyses. It leads in physical edit accuracy while retaining comparable visual fidelity.

5.1 Experimental Setup

  • The comparison includes VACE, Ditto, MiniMax H3, Seedance 2.5, and an unchanged-source baseline, with VOID additionally evaluated for object removal.Metrics cover physical edit accuracy and visual fidelity; implementation uses pretrained components for scene reconstruction, simulation, and video generation.Source: §5.1 · p. 7

5.2 Main Results on PCVE-RigidBench

  • In qualitative examples, VideoPhysEdit follows requested motion and interaction changes while preserving the source scene; comparison methods more often retain source motion or alter appearance.Examples include altered marble motion after a mass change and a toy car no longer colliding after its initial speed is reduced.Source: §5.2 · p. 7, Figure 3 · p. 9
  • 54.0%: VideoPhysEdit reduces TE relative to the strongest competing method, leads physical edit accuracy, and is the only method with positive PES.It achieves the highest Mask IoU and PES, remains close to Seedance 2.5 on PSNR, SSIM, LPIPS, and CLIP similarity, and achieves the best FVD.Source: §5.2 · p. 7, Table 1 · p. 8

5.3 Analysis

  • Across pipeline stages, simulation search and refinement improve the factual rollout and counterfactual trajectories over calibrated initialization, while generation largely preserves motion.On matched objects and frames, TE remains nearly unchanged at generation, while PES and Mask IoU improve slightly; generation primarily restores source appearance.Source: Table 3 · p. 8, §5.3 · pp. 9–10

5.2 Main Results on PCVE-RigidBench

  • Visually plausible generation alone does not ensure a successful physical edit; explicit downstream-consequence instructions also do not yield consistent improvements.The authors suggest grounding interventions in an executable scene whose simulation reproduces observed motion and interactions as a more reliable basis.Source: §5.2 · p. 8

5.3 Analysis

  • The pipeline resolves incomplete or ambiguous observations progressively, combining geometry, support relations, motion anchors, evidence across time, and adaptive temporal scaling.The interaction ROI helps follow fast trajectories and preserve small objects during contact.Source: §5.3 · p. 10

6 Conclusion

VideoPhysEdit guides counterfactual video generation with simulated trajectories from a reconstructed physical scene, and qualitative real-video comparisons show better depiction of edit-induced motion and interactions than the compared methods. The conclusion identifies future extensions, while the appendix outlines the pipeline’s algorithmic details.

6 Conclusion

  • VideoPhysEdit achieves substantially higher physical edit accuracy than compared methods, and real-video comparisons show better depiction of edit-induced downstream motion and interactions.The conclusion also states that the pipeline applies to real-world scenes.Source: §6 · p. 10
  • The pipeline reconstructs an executable physical scene and uses simulated counterfactual trajectories to guide video generation.Source: §6 · p. 10
  • Future work will explore camera motion, richer geometry, and physical models for articulated or actively controlled agents.Source: §6 · p. 10

A VideoPhysEdit Algorithmic Details

  • The algorithmic-details appendix follows the pipeline through motion analysis, scene and motion-prior reconstruction, physical inversion and intervention, and counterfactual video generation.Source: A VideoPhysEdit Algorithmic Details · p. 16

A.1 Motion Observation Analysis

This section describes how VideoPhysEdit reconstructs scene geometry and motion from image observations, then uses that reconstruction to constrain and calibrate simulation. It also covers applying physical edits to guide video generation and defines PCVE-RigidBench’s task inputs and evaluation protocol.

A.1.1 Motion Observations and Reliability

  • Object-mask point tracks yield positions, orientations, and reliability scores; local change detection and cross-object motion and contact cues identify transition boundaries and candidate interactions.When boundary truncation leaves too few shared track points, position is estimated from points visible across a short temporal window; invisible intervals are treated as observation gaps.Source: §A.1.1 · p. 16, §A.1.2 · p. 17

A.1.2 Stable Intervals and Transition Episodes

  • Scale-normalized residuals segment observations into stable intervals and transitions, using constant-acceleration, rotation-induced, and stopping models with boundaries refined by local change detection.Stopping models have zero terminal rate and remain constant after the stop; image-plane models determine segmentation and motion types, while 3D models are fitted later.Source: §A.1.2 · pp. 16–17

A.1.3 Canonical Frame and Motion Anchor Frames

  • A shared canonical frame is selected from visibility, truncation, stability, separation, texture, and resting-object cues, then each stable interval receives a motion anchor frame.The canonical frame aggregates observation scores across objects, whereas anchor selection applies them within each object’s interval; adjacent stable boundaries support transition fitting.Source: §A.1.3 · pp. 17–18

A.2 Canonical and Anchor Scene Reconstruction

  • Canonical-frame point clouds and masks produce category-based sphere or box meshes, support candidates, and object placements optimized against geometry, silhouettes, and support constraints.Anchor scenes reuse a shared world coordinate system and canonical object scales, fitting poses to anchor point clouds and reconciling support relations across each stable interval.Source: §A.2.1 · p. 18, §A.2.2 · pp. 18–19, §A.2.3 · p. 19

A.3 Motion Prior Reconstruction

  • Motion priors lift image observations into 3D and fit stationary, constant-acceleration, or stopping models, constraining supported motion to support planes and unsupported translation to gravity.For rotations, the solver compares fixed orientation, a fixed world axis, and a support contact-line pivot when geometry permits; transition fitting is used when observations are insufficient or conflict with support.Source: §A.3 · p. 19, §A.3.1 · pp. 19–20

A.3.2 Transition Boundaries and Box Symmetry

  • Transition fitting resolves equivalent box orientations jointly across a temporal graph, then constructs position and orientation curves that satisfy available boundary states and velocities.When an onset falls between boundaries, two curves meet at a shared pose while allowing a velocity jump; observation-based corrections preserve the boundary conditions.Source: §A.3.2 · pp. 20–21, §A.3.3 · pp. 21–22

A.3.3 Transition Curves and Local Contact Geometry

  • For supported-sphere rebounds unexplained by reconstructed surfaces, adjacent stable velocities estimate local contact geometry and restitution; stable, anchor, and transition states are assembled into a per-frame motion sequence.A contact hypothesis is retained only when the approach and departure directions, restitution range, image motion, and other object tracks are consistent.Source: §A.3.3 · p. 22

A.4 Physical Inversion

  • Physical inversion maps motion priors, support relations, and contacts to a PyBullet scene with gravity, mass and inertia scales, contact materials, damping, initial state, and collision proxies.The parameterization reflects that observations may constrain pairwise contact coefficients rather than individual material factors, and masses only up to one scale per dynamic-contact component; stable intervals also yield analytic rigid-body constraints.Source: §A.4 · p. 22, §A.4.1 · p. 22, §A.4.2 · pp. 22–23

A.4.2 Constraints from Stable Motion

  • Stable-motion constraints use observed translation, rotation, and rolling to constrain forces, dissipation, contact equilibrium, and rolling dynamics.For fixed-axis rotation, a reduced energy relation initializes effective dissipation; simulation search separately calibrates linear and angular damping.Source: §A.4.2 · pp. 23–24

A.4.3 Constraints from Contact Events

  • Contact-event constraints use extrapolated motions and impact laws, with finite response windows and aggregate momentum balance for extended or incompatible events.When participants have observable box orientations and share a support normal, the method also imposes a necessary angular-momentum condition.Source: §A.4.3 · p. 24

A.4.4 Layered Initialization

  • Layered initialization fits observation relations within parameter bounds, preserves earlier-layer constraints, and uses priors to initialize remaining physical variables.The initialization order includes gravity, unobserved state, inertia, mass scale, damping, friction, and contact coefficients.Source: §A.4.4 · p. 25

A.4.5 Calibration and Simulation Search

  • Calibration tests fixed-axis rotation, supported stable motion, and toppling before a prefix-based simulation search optimizes initial states, physical parameters, and collision proxies.Candidates are simulated continuously from the first frame, and selected variables with their rollout define the executable scene.Source: §A.4.5 · p. 25

A.5.1 Edit Parsing and Counterfactual Rollout

  • Edits are parsed into add, delete, or set operations, applied to the reconstructed scene at the execution frame, and simulated to produce subsequent counterfactual motion.Insertions receive a location, collision proxy, initial state, and physical parameters; set operations change a supported physical parameter or scale velocity.Source: §A.5.1 · pp. 25–26

A.5.2 Motion and Appearance Controls

  • Motion and appearance controls combine object-surface trajectories and background points with an execution-frame reference image prepared for set, delete, or add edits.Identity and depth maps determine visibility, while sphere controls follow simulated translation without rotating the observed appearance.Source: §A.5.2 · p. 26

A.6 Counterfactual Video Generation

  • Counterfactual generation conditions a pretrained video model on simulated point trajectories, an edited reference image, and a scene prompt, then assembles the continuation with the source history.Adaptive temporal scaling preserves counterfactual motion and video duration; small contacting objects may use an enlarged interaction-region crop.Source: §A.6 · p. 26

B PCVE-RigidBench Evaluation Protocol

  • PCVE-RigidBench defines task-level physical-edit and visual-fidelity metrics and includes 129 paired tasks spanning parameter or velocity changes, removals, and insertions.Each task records an execution frame and physical ground truth; 117 edits occur at the first frame and 12 partway through, with videos of 96 frames at 24 fps.Source: B PCVE-RigidBench Evaluation Protocol · p. 27, §B.1 · p. 27, Table 4 · p. 27

B.2 Object Tracking and Evaluation Groups

The appendix defines how objects are tracked and physical edits are evaluated, then reports benchmark results and additional pipeline analyses. Its metrics distinguish trajectory changes, object masks, removals, and visual fidelity.

B.2 Object Tracking and Evaluation Groups

  • Evaluation tracks objects in prediction, target, and source videos with shared descriptions; it includes inserted objects and stratifies motion errors by edited, affected, and measurable groups.The groups are used for analysis and do not determine which objects contribute to PES.Source: §B.2 · pp. 27–28

B.3.1 Trajectory Error and Physical Edit Score

  • Trajectory Error measures motion relative to an alignment frame, while PES compares trajectory error with the unchanged source-video baseline.PES sums errors before taking the ratio, clamps task scores at −1, and treats zero as matching the source baseline.Source: §B.3.1 · pp. 28–29

B.3.2 Mask IoU; B.3.3 Removal; B.4 Visual Fidelity

  • Mask IoU captures object position and extent beyond centroid trajectories; removal evaluation penalizes premature or failed removal, while correct post-execution absence has zero error.Visual fidelity is measured with PSNR, SSIM, LPIPS, CLIP image similarity, and FVD.Source: §B.3.2 · p. 29, §B.3.3 · p. 29, §B.4 · p. 29

C Additional Experiments and Analysis; C.1.3 Complete Benchmark Results

  • The additional experiments include complete-benchmark results and analyses of pipeline stages, simulation search, runtime and memory, downstream-consequence prompts, and qualitative outputs.Table 6 reports results for generated videos and the complete benchmark.Source: C Additional Experiments and Analysis · p. 29, §C.1.3 · p. 30

C.1.1 VideoPhysEdit Settings; C.1.2 Baseline Settings

  • The pipeline uses released pretrained checkpoints without additional training or fine-tuning, and its baseline settings standardize video encoding and frame comparisons.All videos are encoded at 24 fps; VideoPhysEdit covers the complete benchmark duration.Source: §C.1.1 · pp. 29–30, §C.1.2 · p. 30, Table 5 · p. 30

C.1.3 Complete Benchmark Results

  • PES is positive for interventions applied at the first frame and partway through the video.Table 7 reports results by execution timing.Source: Table 7 · p. 30, §C.1.3 · p. 30
  • VideoPhysEdit has positive PES for Add, Delete, and Set and every edited-property group; presence and initial velocity show the largest gains, while restitution is lowest.For Add, PES evaluates changes in existing objects and excludes the inserted object, which has no source trajectory.Source: §C.1.3 · pp. 30–31, Table 8 · p. 31, Table 10 · p. 31
  • For other affected objects, VideoPhysEdit reduces TE to 72.15 pixels and is the only method with positive PES.It also obtains positive PES for directly edited objects.Source: Table 9 · p. 31, §C.1.3 · p. 31

C.2.1 Evaluation of Pipeline Stages

Pipeline-stage evaluations locate where alignment is lost and test controls for preserving simulated motion in generated video. Further examples show how contact evidence, temporal observations, and simulation search address incomplete or ambiguous observations.

C.2.1 Evaluation of Pipeline Stages

  • Physical inversion causes the largest alignment losses in scenes with several contacts: small contact-position or timing errors alter outgoing velocity and displace later states.Motion-prior reconstruction usually preserves anchor-scene alignment; simpler contact sequences retain closer alignment.Source: §C.2.1 · pp. 31–32, Table 11 · p. 32, Figure 5 · p. 32
  • Stage 7 retains Stage 6 motion, with TE changing by about 1.4 pixels and PES and Mask IoU improving slightly.Stage 6 applies the intervention and projects trajectories; Stage 7 uses them as motion control to generate the video.Source: Table 12 · p. 32, §C.2.1 · pp. 32–33
  • With temporal scaling, the generated car leaves the image at the target frame and TE falls from 497.19 to 23.10 pixels.Without scaling, the car falls behind the projected trajectory.Source: §C.2.1 · p. 33, Figure 6 · p. 33
  • For a contacting pair with a roughly 13-pixel smaller object, interaction-ROI generation preserves a radius ratio of 0.62, versus 1.00 with full-image generation.The simulated ratio is 0.57; both variants use identical point trajectories.Source: §C.2.1 · p. 33, Figure 6 · p. 33

C.2.2 Robustness to Incomplete and Ambiguous Observations

  • In drop_centered, selecting a stable contact frame reduces normalized 3D ball-center error from 2.71 to 0.156 and recovers support relations despite similar Mask IoU.The airborne frame has Mask IoU 0.952 without a ball-block support relation; the contact frame has Mask IoU 0.969.Source: §C.2.2 · pp. 33–34, Figure 8 · p. 35
  • Stage 3 merges compatible support-plane fragments and transfers the ball’s support relation, reducing active planes from six to three.The refitted plane combines background points from P0, P1, and P4; candidate P5 is excluded.Source: §C.2.2 · p. 34, Figure 9 · p. 35
  • Stage 4 reconstructs continuous motion across gaps: the apple is observed in only 31 of 96 frames, yet its observed-frame Mask IoU is 0.867.Motion models fit stable intervals and boundary-constrained transition curves connect them.Source: §C.2.2 · pp. 34–35, Figure 10 · p. 36
  • In ball_block, simulation search and final refinement raise Mask IoU from 0.409 to 0.852 by using later observations to distinguish similar early-fitting candidates.Stage 5 retains up to three candidate simulations.Source: §C.2.2 · p. 36, Figure 10 · p. 36

C.2.3 Error Analysis

The error analysis traces failures to reconstruction, physical inversion, and video generation, while the ablation tests simulation search and the prompt study tests explicit descriptions of downstream effects. Runtime measurements distinguish one-time source-video stages from per-edit stages.

C.2.3 Error Analysis

  • Some source scenes yield no valid edited videos: for two scenes covering 13 tasks, unchanged source videos are used as predictions in the complete benchmark evaluation.Source: §C.2.3 · p. 36
  • Errors arise at multiple pipeline stages: identity merging or sparse 3D correspondences impair scene reconstruction, dense or overlapping contacts challenge physical inversion, and video generation can depart from correct simulated motion.In bowling, separated pins share one identity; in domino_chain, repeated appearance leaves too few spatially distributed correspondences. In dining_chain, three closely spaced contacts challenge one rollout.Source: §C.2.3 · p. 36
  • In table_drop_collision, inseparable impulses within one response window reduce Mask IoU from 0.914 at Stage 4 to 0.192 at Stage 5.The physical rollout departs immediately after contact.Source: §C.2.3 · p. 36, Figure 12 · p. 37

C.3 Ablation Study

  • Simulation search and final refinement improve all three Stage 6 metrics and reduce TE by 18.7%.The reported result links a more accurate physical rollout of the source video with more accurate counterfactual trajectories after intervention.Source: Table 14 · p. 38, §C.3 · pp. 38–39

C.4 Runtime and Memory

  • Runtime and peak GPU memory are reported by stage; Stages 1–5 run once per source video, while Stages 6–7 run for each edit.Stage 7 timing includes two overlapping Wan-Move windows; Stage 4 motion fitting and Stage 5 physical simulation run on the CPU.Source: Table 15 · p. 39, §C.4 · p. 39

C.5 Effect of Explicit Downstream Consequences

  • Adding expected-motion descriptions does not consistently improve PES across four tasks; baselines either preserve source motion or make changes that often differ from the target.The study compares original quantitative instructions with the same instructions augmented by qualitative target-motion descriptions.Source: Table 16 · p. 39, §C.5 · p. 39

C.6 Additional Qualitative Results

Additional comparisons show VideoPhysEdit depicting the requested edits and their downstream motion in benchmark and real-video examples. The comparisons also cover multiple edit types across rigid-body scenes.

C.6 Additional Qualitative Results

  • VideoPhysEdit removes selected objects and changes subsequent motion after friction or mass edits while preserving preceding interactions in four additional PCVE-RigidBench examples.Competing methods often retain removed objects or continue the source motion.Source: §C.6 · p. 39
  • Figures 15 and 16 compare baselines with and without explicit downstream-consequence text, while VideoPhysEdit uses the original instruction and all methods share three time points per task.Figure 16 matches Figure 15 in tasks, time points, and layout.Source: Figure 15 · p. 40, Figure 16 · p. 41
  • In two real removal scenes without paired counterfactual targets, VideoPhysEdit lets the tabletop blue ball continue rightward and keeps the ruler-scene small ball at rest.VOID preserves the tabletop reversal and moves the ruler-scene ball; the commercial outputs also differ from the described target behavior.Source: §C.6 · pp. 42–44
  • Across four removal examples, VideoPhysEdit produces the expected subsequent motion, while VOID retains source-interaction motion or moves an object that should remain at rest.The Figure 18 benchmark examples include removing a wooden block so a basketball continues across the floor and removing an apple to prevent a soccer ball's displacement.Source: §C.6 · pp. 40, 41, 44
  • Real-video edits also show VideoPhysEdit slowing a can, extending a ball's time on a ramp after increased friction, and changing collision responses after mass changes.Baseline results frequently preserve original motion or change scene appearance.Source: §C.6 · p. 45
  • Additional examples apply Add, Delete, and Set edits across rigid-body interactions, including an inserted stone that changes a later interaction between two original stones.Figures 20 and 21 group edits by source scene and show the same six frames for each derived edit.Source: §C.6 · p. 45, Figure 20 · p. 45, Figure 21 · p. 46

D Limitations and Future Work

VideoPhysEdit is currently limited to static-camera rigid-body scenes with static planar supports and sphere- or box-modeled geometry.

D Limitations and Future Work

  • The current formulation does not cover moving cameras, nonplanar supports, complex geometry, or articulated and actively controlled agents such as people and robots.The supported setting assumes static planar support surfaces and rigid-body objects represented as spheres or boxes.Source: D Limitations and Future Work · p. 46
  • Fixed thresholds in observation filtering, geometric fitting, and motion analysis can be sensitive to scene scale, object size, and observation quality.Source: D Limitations and Future Work · p. 46
  • Future work will extend reconstruction and simulation to moving cameras, richer geometry and collision proxies, and articulated or controlled agents, while adapting thresholds to scale and observation confidence.Source: D Limitations and Future Work · p. 46

Related papers

Loading 2609.35134v1…