Source-linked AI summary
Learning to Poke by Poking: Experiential Learning of Intuitive Physics
Pulkit Agrawal, Ashvin Nair, Pieter Abbeel, Jitendra Malik, Sergey Levine
TL;DR
The paper asks whether robots can acquire an intuitive physics model from exploratory interaction and use it for manipulation planning. It jointly learns forward and inverse dynamics in an abstract visual feature space from poking experience. The resulting model generalizes to novel visual conditions and supports multi-step displacement of objects to desired locations, while pose-setting remains unsatisfactory under the tested control regime.
Problem
Robots need models that can reason about object dynamics for manipulating previously unseen objects, but pixel-level forward prediction and action inference from images are difficult.
Method
A deep neural network jointly learns inverse actions and forward dynamics from before-and-after images, using a shared latent feature space rather than predicting pixels.
Results
The joint model generalizes to different visual statistics, novel object geometry and texture, and goal displacements farther than single pokes while supporting multi-step planning.
Takeaways & Limitations
Joint forward-inverse modeling produces useful representations for planning object displacement from visual interaction experience.
Takeaways & Limitations
Pose-setting performance is not satisfactory, possibly because the robot executes large discrete-time pokes; continuous control with smaller pokes is proposed for future work.
Abstract
from arXiv · showhide
We investigate an experiential learning paradigm for acquiring an internal model of intuitive physics. Our model is evaluated on a real-world robotic manipulation task that requires displacing objects to target locations by poking. The robot gathered over 400 hours of experience by executing more than 100K pokes on different objects. We propose a novel approach based on deep neural networks for modeling the dynamics of robot's interactions directly from images, by jointly estimating forward and inverse models of dynamics. The inverse model objective provides supervision to construct informative visual features, which the forward model can then predict and in turn regularize the feature space for the inverse model. The interplay between these two objectives creates useful, accurate models that can then be used for multi-step decision making. This formulation has the additional benefit that it is possible to learn forward models in an abstract feature space and thus alleviate the need of predicting pixels. Our experiments show that this joint modeling approach outperforms alternative methods.
1 Introduction
The paper asks whether a robot can learn intuitive physics from its own exploratory interactions and use that model to plan manipulation actions. It frames random poking as experiential data for learning how actions change object states.
- Humans may use extensive object play to learn intuitive physics models that predict how actions affect object motion.
- The robot collects before-and-after visual states together with applied pokes to learn mappings between actions and changes in object state.
- Forward and inverse dynamics are jointly modeled so the robot can learn action-relevant representations without requiring pixel-level future-image prediction.
- The proposed joint model supports generalization and multi-step planning across substantially different visual statistics, object geometries, textures, and goal distances.
2 Data
The robot autonomously gathers interaction data by randomly poking objects on a table, while real-world contact variability produces both expected and unexpected motions. Depth information accelerates collection, but the learned system ultimately relies on RGB images at test time.
- The setup uses a Kinect camera and gripper, with one to three objects selected from 16 distinct objects on a table.
- The robot executes pokes by moving its finger along the XZ plane at a fixed height from the table.
- Random target points are restricted to object surfaces during collection so the robot avoids wasting pokes in free space.
- Point-cloud information is used only for data collection, while testing requires only RGB image data.
- Nonlinear interactions between the finger and objects can cause unexpected motion, including objects moving in the wrong direction after flipping.
3 Method
The method jointly learns inverse actions and forward state transitions in a shared latent visual space, then uses greedy iterative planning to reach distant goals. Evaluation compares learned control with a geometry-agnostic baseline using location and pose errors.
- 3 Method: The inverse model maps initial and goal states to an action, while the forward model predicts the next state from the current state and action.
- 3.1 Model: The network transforms consecutive images into latent features, predicts discretized poke location, angle, and length inversely, and predicts the next feature representation forwardly.
- 3.1 Model: The joint objective combines inverse cross-entropy losses with a weighted forward L1 loss, allowing the forward model to regularize the learned feature space.
- 3.2 Evaluation Procedure: Greedy planning repeatedly predicts and executes pokes using the current and goal images until a no-poke prediction or 10-poke limit is reached.
- 3.2 Evaluation Procedure: Evaluation measures relative location error and pose error, with pose error defined as the angle between the final and goal objects’ major axes.
- 3.2 Evaluation Procedure: The blob baseline estimates object positions and computes pokes from their displacement, without modeling object geometry.
4 Results
The robot generalizes poking across longer displacements and novel object geometries, while jointly learned dynamics improve pose-directed pushing and data-efficient planning. Greedy planning remains unable to route objects around obstacles.
- Generalization: The robot displaced training-set objects and objects with novel geometry and texture to goal locations significantly farther apart than training pokes.Figure 5 includes a Nutella bottle from training and a red cup with previously unseen geometry.
- Planning limitation: Greedy planning failed to push an object around an obstacle and produced zig-zag rather than straight trajectories.The robot pushed the obstacle along with the target object; using the learned forward model for planning was proposed as future work.
- Representation: The learned feature space was sensitive to object location despite receiving no direct supervision for locating objects.Nearest-neighbor visualizations suggest location sensitivity, while the authors note they cannot prove the model explicitly learned object detection.
- Pose-directed pushing: Both inverse and joint models outperformed the blob model on single-poke pushes toward nearby desired poses, indicating representation of object geometry.The blob baseline ignores object geometry.
- Forward-model regularization: With 10K or 20K simulation examples, the joint model outperformed the inverse model and reached closer to the goal in fewer actions.At 100K examples, the two models were at par, so the inverse model no longer relied on the forward model for regularization.
- Forward-model regularization: On the real robot, the joint model outperformed the inverse model when objects had to be displaced by long distances.The evaluation was explicitly framed as a generalization test beyond the displacement distances represented in training pairs.
5 Related Work
The paper differs from policy-learning and hand-designed-feature approaches by learning a general predictive model of object interaction directly from raw images. It also contrasts intuitive-physics modeling with simulator-based and vision-based model-control methods.
- Policy learning: Unlike model-free reinforcement-learning methods that learn goal-specific policies, this work learns intuitive physical models intended for varied goals at test time.The cited reinforcement-learning methods include Atari and simulated or real-world robot control.
- Policy learning: Unlike grasping methods targeting one concrete task, this work learns a general predictive model for achieving a variety of goals.Grasping permits full control of the grasped object, whereas the paper studies non-prehensile manipulation.
- Model-based control: Prior model-based control work includes hand-designed features, known poses, key locations, and systems operating in robotic state space rather than vision.The paper positions its approach as learning from visual inputs instead of relying on those engineered representations.
- Learned representations: The paper learns an intuitive physics model for pushing from raw images, allowing autonomous environmental exploration without human intervention.This distinguishes the approach from prior robotic manipulation methods using hand-designed visual features and known object information.
6 Discussion and Future Work
The discussion favors intuitive physics models for robust, representation-flexible manipulation but identifies data inefficiency, limited control, and untested scope as important boundaries. Future work includes richer environments, continuous control, and improved planning and data collection.
- Discussion: Intuitive physics models may be more robust but less precise than simulator-based models, making robustness quantification an open direction.The discussion notes that small execution errors are acceptable in many practical manipulation tasks.
- Discussion: Simulator-based approaches face difficulties with deformable objects because simulation is hard and requires handcrafted features unlikely to generalize.The paper presents intuitive physics as requiring no object-specific assumptions and as extendable to deformable objects.
- Limitations: Poking provides incomplete control of object state, making action outcomes harder to predict and plan.The paper reports unsatisfactory desired-pose performance, possibly because the robot used large discrete-time pokes.
- Scope: The approach was evaluated on a specific manipulation task, although the authors state it makes no task-specific assumptions and may apply elsewhere.
- Future work: Future evaluations should test more complex environments, diverse object collections, other manipulation skills, and non-manipulation tasks such as navigation.The authors also identify forward-model planning and better-than-random data collection as future directions.