PhysEvolve: Verifier-grounded Self-evolution for Physical Reasoning with Executable Worlds

1 INTRODUCTION

• Naive single-trajectory self-play cannot verify interventions, while continuous proximity rewards can inflate learnability signals for uniformly incorrect predictions.The paper identifies paired intervention branches and discrete, exact ground-truth tasks as necessary design requirements.Source: §1 · p. 2
• PhysEvolve trains VLMs on simulation-authored tasks from policy-proposed executable worlds, with a deterministic executor verifying physical validity and counterfactual ground truth.The executor’s exact Solver feedback updates the shared policy and an adaptive curriculum over physical reasoning intents.Source: §1 · p. 2
• 8.78 percentage points on PhysBench Throwing and 5.08 points on Judgment; ContPhy Counterfactual Removal and Goal-Pit improve by 7.38 and 24.44 points.The reported gains also include CausalPhys Anticipation (+2.46) and Intervention (+2.27), without human-annotated intervention data.Source: §1 · p. 2

Physical reasoning in vision-language models

• Prior studies report that VLMs are less reliable on temporal prediction, intervention, and counterfactual reasoning than on static appearance and geometry.Counterfactual benchmarks use paired simulator-generated alternative trajectories and intervention questions.Source: §2 · pp. 2–3
• Simulator-grounded work converts traces into verifiable supervision; unlike offline trajectory-to-QA pipelines such as Sim2Reason [Prabhudesai et al., 2026], PhysEvolve synthesizes counterfactual supervision online over policy-proposed worlds.Source: §2 · p. 3

Self-evolving agents and environment co-design

• Self-evolving systems generate curricula and environments; PhysEvolve extends this approach from primarily discrete symbolic spaces to continuous multi-body physics.The passage cites Absolute Zero [Zhao et al., 2025] as shared-policy proposer–solver self-play with code interpreters.Source: §2 · p. 3

Reinforcement learning with verifiable rewards

• RLVR uses deterministic verifiers in domains with objectively computable answers; PhysEvolve grounds multimodal physics updates in discrete, simulation-verified causal outcomes.Source: §2 · p. 3

3.1 PROBLEM FORMULATION AND PITFALLS OF NAIVE PHYSICAL SELF-PLAY

• Single-trajectory execution cannot verify intervention outcomes, and graded proximity rewards can make uniformly incorrect predictions appear learnable.The proposed design principles are paired counterfactual branches and discrete, answer-bound tasks with exact ground truth.Source: §3.1 · p. 4

3.2 FRAMEWORK OVERVIEW

• A task-conditioned Proposer creates an executable world and evidence schema; a fixed executor verifies and freezes the scene, simulates matched branches, and synthesizes an answer-bound item.The Solver sees only the baseline video, while the executor alone accesses simulator states and determines counterfactual ground truth.Source: Figure 2 · p. 5, §3.2 · p. 5

3.3 TASK-CONDITIONED PROPOSER

• The Proposer receives a reasoning intent and outputs a PhysDSL world program plus an evidence schema whose entity roles must support that intent.The intents span projectile motion, collisions, and continuous kinematics and contact interactions.Source: §3.3 · p. 5

3.4 INTERVENTION-GROUNDED TASK SYNTHESIS

• The executor admits only valid, physically meaningful worlds, then freezes them so branch outcomes differ only through declared interventions.Matched branches share the same initial state, seed, and time horizon; tasks are suppressed when readouts are ambiguous, infeasible, or within sensitive physical margins.Source: §3.4 · p. 6
• For interventional intents, the executor removes selected bodies from the frozen world and compares those outcomes with the factual trajectory to bind a unique answer.Inverse tasks test candidate removals and retain an item only when exactly one branch achieves the specified goal, or none does.Source: §3.4 · p. 6

3.5 CO-EVOLUTION TRAINING AND ADAPTIVE CURRICULUM

• Exact binary Solver correctness drives joint GRPO updates to the shared policy and an adaptive curriculum over reasoning intents and world exploration.The macro curriculum samples intents based on rolling Solver accuracy, concentrating on task families near the model’s competence frontier.Source: §3.5 · pp. 6–7

4.1 EXPERIMENTAL SETUP

• Evaluation uses PhysBench, ContPhy, and CausalPhys with shared prompts, frame sampling, greedy decoding, and official protocols; matched comparisons include Base, naive self-play, and PhysEvolve.Variants start from Qwen3-VL-8B-Instruct and train for 100 steps using one-epoch GRPO.Source: §4.1 · p. 7

4.2 MAIN RESULTS: TARGETED AND EXTERNAL TRANSFER

• 8.78 points on PhysBench Throwing and 5.08 on Judgment; overall accuracy rises by 0.55 while most non-target partitions remain close to or above Base.Naive self-play changes Throwing by −0.24 and Judgment by +0.26.Source: §4.2 · p. 7
• 24.44 points on ContPhy Goal-Pit and 7.38 on Counterfactual Removal; CausalPhys gains are +2.46 on Anticipation and +2.27 on Intervention.ContPhy gains transfer zero-shot to cloth, fluid, rope, and soft bodies absent from rigid-body training templates.Source: §4.2 · p. 8, Figure 3 · p. 8
• Fluid Flow improves by 11.00 points and Action Substitution by 9.09; at the terminal checkpoint, net correct-item gains are 36 on Throwing, 39 on Judgment, and 25 on Motion.The branch-disabled control trails PhysEvolve by 23.00 points on Fluid Flow; Figure 3 shows task-level transfer and Figure 4 shows checkpoint gains and paired correctness.Source: Figure 3 · p. 8, Figure 4 · p. 8, §4.2 · p. 9

4.3 ABLATION STUDIES

• Disabling branch execution and its dependent Counterfactual/Inverse task families reduces ContPhy Counterfactual Removal by 8.20 points and Goal-Pit by 23.33, back to Base levels.Table 2 evaluates endpoint accuracy under identical protocols with the branch-enabled path deactivated.Source: §4.3 · p. 9, Table 2 · p. 9
• Freezing Proposer updates while training only Solver drives accuracy to 15.01% on CausalPhys and 17.85% on ContPhy, with answer parse rates collapsing to 26.07% and later 0%, respectively.The authors report that Proposer updates maintain diverse, physically non-trivial world generation.Source: §4.3 · p. 9, Table 3 · p. 9

4.4 AUDITING THE SINGLE-TRAJECTORY INTERFACE

• The naive policy exploits graded rewards by outputting stationary initial velocities 746 times or the median parameter rung 145 times without solving the physical events.The audit links continuous proximity scoring to partial credit for uniformly incorrect rollouts.Source: §4.4 · pp. 9–10
• Across 100 rounds, naive self-play produces zero intervention and counterfactual tasks, leaving downstream PhysBench dynamics accuracy stagnant.Its continuous proximity rewards can remain high despite zero exact Forward matches, and Inverse and Parameter rewards reach 0.664 and 0.787 while accuracy stays below 7%.Source: §4.4 · pp. 9–10, Table 4 · p. 9

5 CONCLUSION

• PhysEvolve anchors self-play in executable 3D world programs and paired counterfactual branches, supporting targeted gains and zero-shot transfer to unseen continuous media and real-world video.The conclusion attributes this transfer to verified physical outcomes and reports training without human-annotated physical-reasoning supervision.Source: §5 · p. 10

A.1 COMPLETE CALLABLE SURFACE

• A proposal is a Python module with build_scene(api) and a literal SCENE MANIFEST; construction calls mutate a builder, while only the executor freezes the scene.Table 5 lists the callable surface used by the executor.Source: §A.1 · p. 15, Table 5 · p. 15

A.2 REPORTED EVIDENCE SCHEMAS

• The reported run enables three evidence schemas, and the executor resolves every declared role against the frozen world before constructing an item.Stage 2 retains submitted mass, friction, and restitution values within their validation intervals rather than snapping them to Stage-1 levels.Source: §A.2 · p. 15, §A.1 · p. 15, Table 6 · p. 15

B AUDITABLE WORLD-PROGRAM GALLERY

• Three human-authored reference worlds—Throwing, Collision, and Motion—illustrate the executable world-program interface, with programs and execution records stored as auditable examples.The examples use seed 7 in MuJoCo 3.6.0 and retain their colors, materials, and lighting.Source: B AUDITABLE WORLD-PROGRAM GALLERY · pp. 15–16, Table 7 · p. 16

B.1 THROWING: ONE PROGRAM, FOUR EXECUTABLE INPUTS

• In Throwing, the orange query ball settles in the left pit at baseline and the middle pit after wall A and the other balls are removed.Matched inputs retain the retained bodies’ initial conditions, and the executed difference supplies the counterfactual answer; the evidence also supports inverse questions.Source: §B.1 · p. 16, Figure 5 · p. 16, Table 8 · p. 17

B.2 COLLISION: ROLES AND CONTACT EVIDENCE

• The Collision schema assigns the swing striker as trigger and blue plate as target, enabling attribution, final-state, and relation questions.The reference frames bracket contact between the orange ball and blue plate.Source: §B.2 · p. 17, Figure 6 · p. 17

B.3 MOTION: AN APPARATUS FOR MOTION AND CONTACT

• In Motion, the green ball descends a ramp, contacts an amber slider, and the slider subsequently contacts a blue ball.This trajectory supplies evidence for the active motion-state family.Source: §B.3 · p. 18, Figure 7 · p. 18, Table 10 · p. 19

C INTERVENTION-GROUNDED TASK CONTRACTS

• The implementation first defines physical semantics, tests whether executed evidence makes them decidable, maps admitted cases to answer semantics, and then realizes questions and options.Tables 9 and 10 enumerate the eight active families.Source: C INTERVENTION-GROUNDED TASK CONTRACTS · p. 19, Table 11 · p. 20

C.1 BRANCH MATRIX AND UNIQUE-ANSWER BINDING

• A counterfactual(A) item requires a valid branch readout different from baseline; inverse items require all three branch views and admit only a unique achiever or a justified “none.”Ambiguous targets are skipped, so “none” is an executor-derived negative rather than a catch-all for failed execution.Source: §C.1 · p. 20

C.2 SURFACE REALIZATION BOUNDARY

• Only after an answer semantic is admitted does the interface choose a compatible stem and four-option layout, preserving the executor-determined physical decision across linguistic realizations.A canonical family layout is used if a sampled surface lacks the required semantic.Source: §C.2 · p. 20

D VERIFICATION, REPAIR, AND BRANCH EXECUTION

• Verification assigns each proposal one status, applies bounded repairs without adding causal structures, then freezes the world and executes matched branches under the baseline seed and horizon.At most six distinct removal sets run per world; failed or capped branches suppress dependent items.Source: D VERIFICATION, REPAIR, AND BRANCH… · p. 20

E OPTIMIZATION AND ACTIVE CURRICULUM

• Equation 4 separates terminal failure, insufficient retained evidence, and admitted worlds; the Solver receives exact option-match reward.Proposal status prices vary by failure type, while trivial worlds receive zero status price.Source: E OPTIMIZATION AND ACTIVE CURRICULUM · pp. 20–21
• The macro curriculum uses rolling Solver accuracy to adapt intent sampling and Proposer credit, while the micro level explores valid, physically non-trivial worlds conditioned on the sampled intent.The reported setup begins with an eight-step prior-weighted cold start; executable demonstrations remain available as syntactic references.Source: E OPTIMIZATION AND ACTIVE CURRICULUM · p. 21, Table 12 · p. 22
• Exact Solver outcomes update rolling family statistics, directing generation toward the Solver’s competence boundary while validity, repair, diversity, and saturation controls shape world generation.Family-difficulty scaling adjusts Proposer credit across intents.Source: E OPTIMIZATION AND ACTIVE CURRICULUM · p. 21

F REPRODUCIBILITY, TRAINING, AND EVALUATION CONFIGURATION

• As Proposer, the shared policy receives a reasoning intent and returns a world program plus manifest.The output is enclosed in <propose> tags.Source: §F.1 · p. 22
• As Solver, the shared policy receives rendered video and an executor-authored multiple-choice item, then returns one option without simulator access.The answer is enclosed in <answer> tags.Source: §F.1 · p. 22
• The fixed executor validates programs, runs required baseline and branch simulations, and binds each admitted answer.Source: §F.1 · p. 22

• The launcher fixes a 100-step horizon, saves every 25 steps, and reports the terminal step-100 checkpoint.Source: §F.2 · p. 22

• All models within a benchmark share the item list, visual preprocessing, prompt, decoding, answer parser, and scorer; unparsed responses count as incorrect.Source: §F.3 · p. 22
• PhysBench frames go directly to the Qwen processor, CausalPhys excludes five annotations with non-A–D gold labels, and ContPhy normalizes extracted responses.ContPhy evaluates one question per generation and lowercases responses while stripping whitespace.Source: §F.3 · p. 22
• Accuracy and percentage-point differences are reported on the shared item set within each benchmark.Source: §F.3 · p. 23

G.2 COMPLETE CONTEXTUAL REFERENCE TABLE

• The contextual-reference table includes open-weight models on heterogeneous scales, which are not used for component attribution; Base, Naive, and PhysEvolve form the matched 8B comparison.Source: §G.2 · p. 23

G.3 CAUSALPHYS CATEGORIES AND CONTPHY DOMAINS

• Table 18 reports all four CausalPhys categories and ContPhy physical domains, alongside the intervention-operation views used in the main text.Task, subtype, and ability levels overlap, whereas rows within each top-level partition do not.Source: §G.3 · pp. 23–24

G.1 PHYSBENCH TAXONOMY

• Table 16 gives complete top-level PhysBench task and ability partitions, while its dynamics-subtype rows are nested rather than a complete partition.Source: §G.1 · p. 23, Table 16 · p. 24

G.4 JOINT PROPOSER–SOLVER UPDATES

• The Solver-only update run retains Proposer generation, the executor, and the family set, but its training answer parse rate falls to zero for steps 65–100.Its parse rate remains valid through step 64; Table 3 compares endpoint accuracy and parse rates for joint and Solver-only updates.Source: §G.4 · p. 24

• Figure 8 shows matched ContPhy Counterfactual Removal and Goal-Pit examples from held-out soft-body scenes, including three of ten input frames and model and reference answers.The Goal-Pit reference answer is “None of the above.”Source: Figure 8 · p. 25
• Table 19 gives complete task-level transfer accuracies, while Table 20 partitions matched PhysBench items by Base and PhysEvolve correctness.The difference between discordant counts divided by the row sample size recovers the accuracy change; rows are interpreted individually because views can overlap.Source: §G.5 · p. 24, Table 20 · p. 26, Table 19 · p. 26