Source-linked AI summary
PHYRE: A New Benchmark for Physical Reasoning
Anton Bakhtin, Laurens van der Maaten, Justin Johnson, Laura Gustafson, Ross Girshick
TL;DR
Physical reasoning agents need to generalize across novel puzzles while solving them efficiently, but current methods may struggle in this setting. The paper introduces PHYRE, a 2D physics-puzzle benchmark with simulator feedback and evaluates modern learning algorithms. Baselines show limited efficiency and generalization, motivating agents that use richer observations and learned physics models.
Problem
Physical reasoning benchmarks should test whether agents can generalize to novel puzzles and solve them with few attempts, an ability current methods are expected to lack.
Method
PHYRE provides deterministic 2D physics puzzles with symbolic goals, restricted body-placement actions, simulator feedback, and unseen-task evaluation.
Results
Current baselines have limited generalization: DQN-O reaches 56.2% AUCCESS on PHYRE-B but 39.6% on PHYRE-2B, while online-learning agents outperform offline counterparts.
Takeaways & Limitations
PHYRE motivates sample-efficient agents that use intermediate observations, counterfactual reasoning, and learnable forward-prediction models to improve physical problem-solving.
Takeaways & Limitations
The baseline methods studied remain far from PHYRE’s goal of human-like generalization, and contextual-bandit approaches did not work well with the benchmark’s complex observation and action spaces.
Abstract
from arXiv · showhide
Understanding and reasoning about physics is an important ability of intelligent agents. We develop the PHYRE benchmark for physical reasoning that contains a set of simple classical mechanics puzzles in a 2D physical environment. The benchmark is designed to encourage the development of learning algorithms that are sample-efficient and generalize well across puzzles. We test several modern learning algorithms on PHYRE and find that these algorithms fall short in solving the puzzles efficiently. We expect that PHYRE will encourage the development of novel sample-efficient agents that learn efficient but useful models of physics. For code and to play PHYRE for yourself, please visit https://player.phyre.ai.
1 Introduction
PHYRE targets physical reasoning and generalization by presenting simple 2D physics puzzles that agents must solve in few attempts. The benchmark is designed to expose weaknesses in current learning methods and motivate sample-efficient physics models.
- Motivation and benchmark: PHYRE evaluates whether agents can generalize from solved physics puzzles to previously unseen related tasks.The paper identifies generalization as a key aspect of physical intelligence and structures PHYRE around unseen puzzles.
- Motivation and benchmark: Current machine-learning methods are expected to struggle with quickly solving new physical puzzles, motivating sample-efficient learning and efficient physics models.The paper frames this as a hypothesis and anticipates fundamental improvements rather than claiming a demonstrated solution.
- Motivation and benchmark: Each puzzle specifies a goal state and can be solved by placing one or more new bodies before running the simulator.Scenes use simple physical interactions, while the agent must choose placements that achieve the terminal goal.
- Benchmark design: PHYRE restricts scenes to balls and rectangular bars, deterministic collision, gravity, and friction, and symbolic goals without natural language.These restrictions keep tasks simple while retaining nontrivial physical reasoning.
- Benchmark design: Agents receive feedback from failed attempts and can refine later attempts using observations of what happened.This setup is intended to encourage learning agents with abilities resembling human trial-and-refinement.
2 Related Work
PHYRE connects intuitive physics, visual reasoning, computer games, and simulated robotics while emphasizing dynamic multi-task generalization. Its simplified deterministic environment enables more systematic study of physical reasoning than many prior settings.
- Positioning: PHYRE was developed concurrently with the Tools game and relates to prior work on intuitive physics, visual reasoning, and simulated environments.These connections place the benchmark across several neighboring research areas.
- Intuitive physics: Intuitive-physics research models physical reasoning with simplified theories, simulations, and neural predictions, often on constrained block-toppling tasks.The cited work spans probabilistic physical simulations and pixelwise or qualitative future-state prediction.
- Visual reasoning: Unlike visual-reasoning benchmarks centered on static scenes, PHYRE requires reasoning about interactions and world dynamics.PHYRE shares multi-object scene reasoning with visual reasoning tasks but adds physical evolution over time.
- Learning in computer games: Unlike many computer-game agents specialized for individual games, PHYRE requires one model to solve a wide range of tasks.The benchmark also emphasizes learning efficiently from limited simulator interactions.
- Simulated robotics: Simulated-robotics studies commonly use realistic, nondeterministic 3D environments with partial observability, whereas PHYRE uses a simpler setting for systematic analysis.The contrast concerns experimental control and the study of reasoning capabilities.
3 PHYRE Physical Reasoning Benchmark
PHYRE is a deterministic 2D Newtonian-physics benchmark in which agents place bodies to satisfy symbolic goals under restricted action tiers. It evaluates generalization and sample efficiency through repeated simulator attempts, observations, and AUCCESS.
- Environment and tasks: PHYRE simulates deterministic 2D Newtonian physics with gravity, friction, and static or dynamic non-deformable bodies.Bodies come from a small vocabulary and vary in scale, location, and orientation.
- Environment and tasks: A task pairs an initial body configuration with a goal represented as a subject–relation–object triplet, currently requiring touching for at least 3 seconds.The environment can later be extended with additional relations.
- Learning setting: The agent places one or more dynamic bodies in a single action, then receives binary success feedback and intermediate observations after simulation.Invalid placements are rejected, and failed tasks reset before another attempt.
- Observations: Agents observe rasterized 256×256 images rather than exact positions, orientations, masses, and velocities.The image grid uses seven values to encode goal objects, confounding bodies, agent-placed bodies, and background.
- Benchmark tiers: PHYRE-B uses single-ball actions in 3D, while PHYRE-2B uses pairs of balls in 6D, making the latter action space more complex.Both tiers are restricted because unrestricted body choices, properties, and counts would create a large action space.
- Benchmark tiers: Each tier contains 25 templates with 100 related tasks per template, supporting within-template and cross-template generalization evaluation.Tasks vary initial-world parameters while sharing a common goal within each template.
- Evaluation: AUCCESS weights early success heavily: the first 10 attempts contribute approximately 0.5 of the score.It is the area under the success-percentage curve on a logarithmic attempt scale, encouraging sample-efficient agents.
4 Experiments
The experiments establish baseline agents and evaluate efficient physical reasoning across PHYRE tiers and generalization settings. Results show that online learning can help, but harder action spaces and cross-template transfer remain challenging.
- Experimental Setting: PHYRE experiments evaluate within-template and cross-template generalization separately on the PHYRE-B and PHYRE-2B tiers.The benchmark provides deterministic environments, fixed 10-fold task splits, and standardized success-percentage and AUCCESS measures.
- Baseline Agents: Five baseline agents rank actions from initial-state observations: RAND, MEM, DQN, and online-learning counterparts MEM-O and DQN-O.RAND samples uniformly; MEM ranks memorized actions, while DQN predicts rewards for observation-action pairs.
- Main Results: DQN-O reaches 56.2% AUCCESS on PHYRE-B but only 39.6% on PHYRE-2B, showing that the larger two-ball action space substantially reduces success.Online-learning agents substantially outperform their offline counterparts.
- Main Results: The PHYRE-2B cross-template setting presents a significant challenge for all agents, especially in the low-attempt regime emphasized by AUCCESS.Table 1b reports the percentage of tasks solved within 10 attempts.
- Analysis: Oracle ranking solves all PHYRE-B tasks and 95% of PHYRE-2B tasks with fewer than 100,000 ranked attempts, while DQN ranks better than MEM.Non-oracle performance eventually plateaus or decreases because rankings lack action diversity and may propose similar attempts.
- Analysis: MEM-O is fairly insensitive to online-update weight, whereas DQN-O varies with test-time updates and is impeded by online updates within-template.Figure 5 analyzes how aggressively online agents re-weight training and testing examples.
5 Discussion and Future Work
PHYRE is intended to support physical-reasoning agents that generalize like humans, but the evaluated baselines remain far from that goal. The authors identify simulator feedback, learned prediction, diversified attempts, active testing, and causal methods as promising directions.
- The baseline methods demonstrate limited generalization abilities relative to PHYRE’s intended human-like physical reasoning.
- Agents should use intermediate simulator observations after unsuccessful attempts to refine subsequent actions through counterfactual reasoning.The authors argue that failing to use these observations makes agents sample-inefficient because they contain task-specific information.
- A learnable forward-prediction model could mimic the simulator and provide predicted states as additional inputs to a DQN.
- The authors propose diversifying attempts, using active test-time strategies, and applying invariant causal prediction because tasks share an underlying physics model.
- PHYRE is extensible: its initial tiers are approachable yet challenging, while future tiers may use substantially larger and more complex action spaces.
- The authors discourage internal simulators and extensive PHYRE-specific hand-coded biases because they violate the benchmark’s intended spirit.
A Ablation Study of Deep Q-Network (DQN)
The ablation study examines action-encoder size, observation–action fusion location, and class balancing in the baseline DQN. Batch balancing is especially important, while fusion choices and larger encoders have mixed effects.
- Figure 6 compares mean AUCCESS on PHYRE-{B, 2B} across six DQN variants changing architecture, fusion, or training-batch balancing.Error bars show one standard deviation.
- Class-balancing training batches is critical, particularly on PHYRE-2B, where only 0.3% of randomly chosen actions yield positive examples.
- Early fusion of action information into the ResNet-18 observation encoder does not help and prevents caching its output during action ranking.
- Fusing through channel-wise bias and gain modulation before ResNet-18 conv5 is preferable to fusion after globally pooled features, which substantially deteriorates AUCCESS.
- Larger action encoders can improve performance, but their gains are not consistent across settings.
B PHYRE Tasks
PHYRE organizes 5,000 deterministic 2D physics tasks into two tiers with one-ball or two-ball action spaces. Its design enforces solvability, stable solutions, template-level diversity, and increasing difficulty.
- The benchmark contains 5,000 tasks across PHYRE-B and PHYRE-2B, with 25 templates and 100 related tasks per template.
- PHYRE-B permits placing one ball, whereas PHYRE-2B requires placing two balls in the initial scene.
- A stable solution both solves a task and remains successful after translations of 0.5 pixels along each axis.The benchmark tests eight such shifts.
- Task solvability is estimated stochastically because the continuous action space prevents exhaustive search, using sampled actions and a binomial test.
- The solvability procedure uses p0 = 10^-5 for PHYRE-B and p0 = 10^-6 for PHYRE-2B.
- PHYRE-2B templates are strictly harder because fewer than 50% of their tasks may be solvable with a single ball.
- Templates avoid a single master solution while allowing shared solutions because tasks within a template share design and high-level physical strategy.
- PHYRE-2B tasks generally require more diverse solutions than PHYRE-B tasks, based on histograms of sampled actions solving template tasks.
C Comparing Agents
The paper compares agents using paired AUCCESS values across folds and generalization settings. A one-sided Wilcoxon test determines whether one agent outperforms another.
- Agent performance is compared with a one-sided Wilcoxon test applied to paired AUCCESS vectors.
- Table 2 reports AUCCESS scores on a 0.0 to 1.0 scale for five agents across both generalization settings and 10 folds.