Source-linked AI summary
Learning Visual Predictive Models of Physics for Playing Billiards
Katerina Fragkiadaki, Pulkit Agrawal, Sergey Levine, Jitendra Malik
TL;DR
The paper addresses how an agent can learn external-world dynamics that generalize across varied environments and support goal-directed action planning. It learns an object-centric visual predictive model from billiards interactions, using object-centered glimpses and internal simulations to evaluate actions. The model generalizes to unseen environments and successfully plans billiards actions, including reaching target positions and hitting other balls.
Problem
Agents need models of external environments that generalize across variation, but such environments are harder to model than the agent itself.
Method
The paper learns object dynamics from object-centered visual glimpses and applied forces, then uses multiple internal simulations to choose actions.
Results
The learned model generalizes to environments unseen during training and supports planning actions in novel billiards environments without task-specific supervision.
Takeaways & Limitations
Object-centric visual prediction can provide a dynamical world model for planning billiards actions across novel environments.
Takeaways & Limitations
Applying the method to complex real-world settings requires more nuanced mechanisms for creating visual renderings.
Abstract
from arXiv · showhide
The ability to plan and execute goal specific actions in varied, unexpected settings is a central requirement of intelligent agents. In this paper, we explore how an agent can be equipped with an internal model of the dynamics of the external world, and how it can use this model to plan novel actions by running multiple internal simulations ("visual imagination"). Our models directly process raw visual input, and use a novel object-centric prediction formulation based on visual glimpses centered on objects (fixations) to enforce translational invariance of the learned physical laws. The agent gathers training data through random interaction with a collection of different environments, and the resulting model can then be used to plan goal-directed actions in novel environments that the agent has not seen before. We demonstrate that our agent can accurately plan actions for playing a simulated billiards game, which requires pushing a ball into a target position or into collision with another ball.
1 INTRODUCTION
The paper asks how agents can learn generalizable external-world dynamics from vision and use internal simulations to plan actions in unfamiliar billiards environments. It proposes object-centric prediction to exploit physical translation invariance and demonstrates goal-directed billiards planning without task-specific supervision.
- Motivation: Visual imagination lets an agent simulate possible future world states under different actions before selecting and executing an action.The internal model supports planning by comparing simulated outcomes.
- Motivation: Modeling external environments is harder than modeling the agent because environments vary more broadly and require broader generalization.The paper emphasizes that complex environments require models of both the agent and the external world.
- Problem setting: Moving-ball worlds vary in object count, geometry, sizes, colors, and applied forces, preventing one explicitly defined state space across worlds.Their combinatorial variation makes these worlds a controlled setting for studying generalization.
- Approach: Object-centric prediction models each object’s temporal evolution from object-centered glimpses, incorporating translation invariance and model sharing across worlds and instances.This contrasts with frame-centric prediction from the entire billiards image.
- Demonstration: In simulated billiards, the learned model predicts forces for moving a ball to a desired location or colliding with another ball without goal-specific supervision.The model is evaluated on planning actions in varying-geometry billiards environments.
2 PREVIOUS WORK
Previous work learned visual control policies, low-dimensional control representations, physics parameters, or video predictions, but generally did not establish a generalizable internal model for planning in novel environments. This paper positions its contribution at the intersection of visual prediction, physics modeling, and model-based control.
- Vision-based control: Vision-based control methods have used deep networks to map visual inputs directly to actions in tasks such as Atari and robotic control.These approaches focus on policy learning from visual observations.
- Vision-based control: Other methods transform raw sensory observations into low-dimensional features for reinforcement learning and control, mainly in relatively simple simulated domains.Examples include synthetic robotic arms and inverted pendulums.
- Physics models and control: Internal models of external-world physics have been proposed for planning or explaining human sensorimotor learning, but prior work was theoretical or human-focused.The paper distinguishes its objective of building an internal visual model for action planning.
- Learning physics from images: Methods estimating Newtonian parameters from images or videos can use known billiards physics, but different collision models may govern ball-ball interactions.This motivates learning a predictive model rather than assuming one fixed dynamic model.
- Video prediction: Video-prediction models generated bouncing-ball images but were not shown to generalize to novel environments or represent agent influence on the environment.Related robotic-arm prediction was demonstrated only for the same object in the same visual environment.
- Motion prediction: Visual-tracking predictors commonly operate on Cartesian coordinates, potentially discarding useful visual information about tracked targets.The paper contrasts these approaches with prediction from visual content centered on objects.
3 LEARNING PREDICTIVE VISUAL MODELS
The model learns object dynamics from tracked visual glimpses and applied forces across billiards worlds with varying object counts and wall geometries. It predicts future velocities for each object, enabling collision-aware predictions across novel configurations.
- Training setup: Training uses billiards worlds containing multiple interacting balls and walls, with different numbers of balls and wall geometries held out for testing.The setup targets generalization to previously unseen environments.
- Network architecture: For each object, the network receives four temporally stacked object-centered glimpses, the current applied force, and recurrent hidden states, then predicts future displacements.The architecture predicts outputs across h future frames.
- Object-centric prediction: The same predictive model is applied to every object, while object-centered inputs avoid directly predicting image pixels and represent each object’s motion.The formulation predicts current and future velocity from visual fixations and applied forces.
- Assumption: Training assumes perfect object tracking, which supplies the ground-truth velocities used to train the predictor.The paper characterizes this as a mild assumption because tracking is well studied and infants can anticipate object motion during smooth pursuit.
- Qualitative evaluation: Predictions accurately capture ball velocities after collisions across varied table configurations.Figure 3 compares predicted trajectories with ground-truth positions over different prediction horizons.
- Data generation: Training data comprise randomly generated ball-motion sequences from rectangular and non-rectangular wall configurations.The implementation pre-generated 10K sequences and sampled minibatches of 20 consecutive frames.
4 MODEL EVALUATION
The evaluation compares object-centric, frame-centric, and constant-velocity prediction models using velocity errors, collision-focused accuracy, scalability, and generalization across varied billiards worlds. Object-centric models generally predict collision dynamics more accurately and generalize across larger ball counts and wall geometries.
- Prediction evaluation: The evaluation measures overall velocity error and error near collisions in models trained and tested on random billiards worlds.The compared models are Constant Velocity (CV), Frame Centric (FC), and Object Centric (OC).
- Prediction evaluation: The object-centric model outperforms the frame-centric model and substantially outperforms the constant-velocity baseline, especially near collisions.Collision-focused evaluation is emphasized because balls follow linear trajectories except when they collide.
- Scaling with ball count: As the number of balls increases from one to three, model performance degrades only slightly, while the object-centric model generally remains more accurate than the frame-centric model.Figure 4 reports angular error near collisions over a horizon of h = 20 time steps.
- Generalization to wall geometry: The models accurately predict post-collision ball velocities in environments with non-rectangular walls, indicating learning beyond a specific environment.Qualitative trajectory visualizations support generalization across varied wall configurations.
- Generalization to wall geometry: Testing wall lengths of 800-1200 pixels after training on 300-550 pixel walls demonstrates generalization to substantially larger wall configurations.The result supports generalization across different wall geometries.
- Generalization to more balls: Models trained on two- and three-ball worlds perform well when tested on worlds containing three, four, and six balls, without additional training.The object-centric model generalizes substantially better than the frame-centric model in these larger-ball settings.
5 GENERATING VISUAL IMAGINATIONS
The model generates future visual states iteratively by translating balls according to predicted velocities and feeding each rendered image back as input. These visual imaginations capture collision dynamics, generalize to much larger environments, and rely on LSTM memory when local glimpses are uninformative.
- Iterative visual imagination: The model generates each next world image by translating every ball according to its predicted velocity, then iterates this process for future states.The generated image becomes the next model input, enabling recurrent visual imagination.
- Collision prediction: The imagined trajectories anticipate collisions and remain consistent with ball and collision dynamics, although accumulated errors can produce deviations from ground truth.Trajectory reversals at collisions are captured, but predictions are not perfect.
- Large environments: Long-range LSTM memory supports accurate velocity predictions when glimpses contain only a ball against a white background.Without LSTM units, imagined trajectories reverse direction unexpectedly and exhibit other errors.
6 USING PREDICTIVE VISUAL MODELS FOR ACTION PLANNING
The predictive models are used to plan forces for reaching target states without goal-specific supervision. Planning compares imagined outcomes under different forces and selects the force whose predicted state is closest to the target, with evaluation against actual physics-engine outcomes.
- Planning tasks: The agent plans forces for moving a ball to a target location and for making it hit a second moving ball without direct goal-specific supervision.The second task is more challenging because it involves collision with another moving ball.
- Action-planning method: The planner runs multiple visual-imagination simulations under different forces and selects the force producing the state closest to the target.CMA-ES searches for the optimal force instead of exhaustively enumerating all forces.
- Target-location results: 56% hit accuracy brings the ball within 25 pixels of the target, compared with 100% for the oracle; the OC model significantly outperforms the FC model.Hit accuracy counts trials whose closest trajectory point lies within p pixels of the target.
7 DISCUSSION AND CONCLUSION
The paper presents an object-centric visual prediction approach that exploits translation invariance to model physical dynamics and plan actions in unseen environments without task-specific supervision. Applying the method to complex real-world settings remains constrained by the need for more nuanced visual-rendering mechanisms.
- Discussion and conclusion: The object-centric approach learns world dynamics directly from visual inputs by exploiting translation invariance in physical systems.The model is intended to handle combinatorial structure in external-world dynamics.
- Discussion and conclusion: The model generalizes to environments not encountered during training and supports action planning in novel environments without task-specific supervision.This conclusion concerns the demonstrated scope of the model.
- Limitations and future work: Complex real-world use requires more nuanced mechanisms for creating visual renderings.The authors identify latent abstract feature spaces and visual exemplars as possible directions for improvement.