Source-linked AI summary
UMPNet: Universal Manipulation Policy Network for Articulated Objects
Zhenjia Xu, Zhanpeng He, Shuran Song
TL;DR
Articulated objects vary in geometry and kinematic structure, making their manipulation trajectories differ and object-specific policies difficult to generalize. UMPNet learns closed-loop actions from visual observations through self-guided exploration, including an Arrow-of-Time attribute for choosing action direction. The paper reports that it outperforms alternative approaches in both open-ended exploration and goal-conditioned manipulation.
Problem
Object-specific manipulation policies are time-consuming to design and often fail to generalize across objects with different articulation structures.
Source: §I · p. 1Method
UMPNet learns closed-loop action trajectories from visual observations through self-guided exploration, using an Arrow-of-Time attribute to guide action direction.
Source: §I · pp. 1–2Results
UMPNet outperforms alternative approaches in open-ended state exploration and goal-conditioned manipulation.
Source: §V · p. 8Takeaways & Limitations
Reversing the Arrow-of-Time while using the goal state supports goal-conditioned manipulation as well as exploration toward novel states.
Source: §I · p. 2The fixed camera position makes the agent not robust to occlusion, and the approach assumes bidirectional trajectories, excluding irreversible actions such as locking.
Source: E. Limitations and failure cases · pp. 7–8
Abstract
from arXiv · showhide
We introduce the Universal Manipulation Policy Network (UMPNet) -- a single image-based policy network that infers closed-loop action sequences for manipulating arbitrary articulated objects. To infer a wide range of action trajectories, the policy supports 6DoF action representation and varying trajectory length. To handle a diverse set of objects, the policy learns from objects with different articulation structures and generalizes to unseen objects or categories. The policy is trained with self-guided exploration without any human demonstrations, scripted policy, or pre-defined goal conditions. To support effective multi-step interaction, we introduce a novel Arrow-of-Time action attribute that indicates whether an action will change the object state back to the past or forward into the future. With the Arrow-of-Time inference at each interaction step, the learned policy is able to select actions that consistently lead towards or away from a given state, thereby, enabling both effective state exploration and goal-conditioned manipulation. Video is available at https://youtu.be/KqlvcL9RqKM
I. INTRODUCTION
Object-specific manipulation policies are time-consuming to design and often fail to generalize across articulation structures. UMPNet learns closed-loop action sequences from visual observations through self-guided exploration, and its experiments report strong performance in state exploration and goal-conditioned manipulation.
I. INTRODUCTION
- UMPNet is a single policy network that discovers manipulation policies for articulated objects from RGB-D observations.The motivation is that object-specific policies can be time-consuming to design and fail to generalize across articulation structures.Source: §I · p. 1
- UMPNet represents continuous SE(3) actions as an initial 3D position and a sequence of directions, allowing trajectories of varying lengths.The representation is intended to describe complex motion trajectories with few constraints.Source: §I · p. 1
- Closed-loop prediction conditions each next action on the object’s initial and current states, allowing visual observations to inform adjustments.This addresses the challenge of predicting a full trajectory from the initial state amid error accumulation and partial observation.Source: §I · pp. 1–2
- Arrow-of-Time inference uses interaction history to select actions toward the future or back toward the past, supporting exploration and goal-conditioned manipulation with reversed AoT.For goal-conditioned manipulation, the initial state is swapped with the goal state.Source: §I · p. 2
- UMPNet outperforms alternative approaches in open-ended state exploration and goal-conditioned manipulation, without human demonstrations, scripted policies, or predefined goals.The framework learns across diverse objects and generalizes to unseen objects and categories.Source: §I · p. 2
II. RELATED WORK
- Prior open-loop methods estimate object pose before executing task-specific trajectories, which may not transfer across tasks and require difficult pose estimation for unknown articulated objects.UMPNet does not require object detection, pose estimation, or part segmentation.Source: §II · p. 2
- Learning manipulation from demonstrations can require expensive collections of high-quality action- and pose-annotated data, whereas UMPNet generates training data through active interaction.The cited demonstration approaches include motion capture, video, and virtual reality.Source: §II · p. 2
- Single-step affordance methods predict where an action can be applied, but Where2Act’s single-step formulation does not generate the long-horizon trajectories targeted here.The paper focuses on long-horizon motion trajectories for goal-conditioned manipulation.Source: §II · p. 2
III. APPROACH
The approach learns action position, distance, and direction selection through self-guided exploration. Given initial and current RGB-D observations, its policy outputs an SE(3) action; interaction begins with a grasp and continues with directions applied relative to the object surface.
III. APPROACH
- The exploration policy must select an interaction position, choose an action direction with sufficient movement, and consistently choose directions that reach novel states.These requirements correspond to position selection, action distance, and Arrow-of-Time inference.Source: §III · p. 2
- The system learns through self-guided exploration without human demonstrations, scripted policies, or predefined goal conditions.The policy is intended to discover manipulation policies from visual observations.Source: §III · pp. 2–3
A. Problem formulation
- At each step, the policy maps initial and current RGB-D observations to an action represented by end-effector position and moving direction in SE(3).The action direction is a unit 3D vector.Source: A. Problem formulation · p. 3
- The first interaction selects a 3D position for a suction grasp; subsequent actions move the end-effector 0.18 m along a selected direction while keeping the grasp position fixed relative to the object surface.The end-effector orientation remains aligned with the surface normal during interaction.Source: A. Problem formulation · p. 3
B. Position inference
UMPNet selects interaction positions from RGB-D affordance maps and evaluates action directions using predicted motion distance and Arrow-of-Time (AoT). Conditioning AoT on the initial and current observations distinguishes actions that move the object back toward its initial state from those that move it forward.
B. Position inference
- The position network predicts per-pixel affordance scores from RGB-D input, with positive labels assigned when an executed action changes the object state in a future step.A movable-link location alone is insufficient: near-joint-axis contacts may not move the part, and incorrect directions can make a correct position receive a negative label.Source: B. Position inference · p. 3
C. Direction inference
- Iterative CEM narrows direction sampling from uniform SO(3) candidates toward higher-scoring regions; the final model uses 64 samples.Candidate resampling probability is correlated with the score, with temperature T = 20.Source: C. Direction inference · p. 3
- DistNet and DistDecoder estimate action-induced joint displacement from the current observation embedding and candidate action.The direction module combines predicted distance and AoT attributes when evaluating candidates.Source: C. Direction inference · p. 3
- AoT inference uses the initial and current observations to distinguish actions that move the object back toward its initial state from actions that move it forward.This addresses multi-step back-and-forth motion that may change the object without exploring new states.Source: C. Direction inference · pp. 3–4
D. Training
Training data come from interaction trials generated by the policy itself, with contradictory AoT choices used within trajectories. The exploration comparison contrasts inferred AoT labels and tracks unique states visited across interaction steps.
D. Training
- All training data come from policy-generated interaction trials, stored in a FIFO replay buffer of size 6400.The policy is trained from scratch without demonstrations or scripted data in this setup.Source: D. Training · p. 4
- Each trajectory first executes positive-AoT actions to move away from the initial state, then negative-AoT actions to encourage return.Trajectories begin at length 4 and grow by 2 every 400 epochs after epoch 1000, up to length 20.Source: D. Training · p. 4
- In the exploration comparison, UMPNet infers the correct AoT labels while SingleStep and Where2Act assign red labels to all directions; the figure also plots unique states visited by step.The unique-state plot uses laptop testing instances and error bars measured across five random seeds.Source: Figure 4 · p. 4
E. Goal conditioned manipulation with reversed AoT
A policy trained through open-ended exploration can be applied to goal-conditioned manipulation without additional training. It uses reversed AoT and goal-observation differences to select actions and interaction positions relevant to the target state.
E. Goal conditioned manipulation with reversed AoT
- The policy trained only with open-ended exploration can be applied directly to goal-conditioned manipulation without additional training.This task directs actions toward a specified goal state rather than an arbitrary novel state.Source: E. Goal conditioned manipulation with… · p. 4
- For goal seeking, the policy substitutes the goal observation for the initial observation and executes actions with reversed, negative AoT to move toward the goal.The trajectory terminates if all candidate directions have non-negative AoT predictions.Source: E. Goal conditioned manipulation with… · p. 4
- A difference mask between initial and target observations filters the position heatmap so interaction is selected from pixels that changed.This helps select the appropriate movable link when an object has multiple movable links.Source: E. Goal conditioned manipulation with… · p. 4
IV. EVALUATION
The evaluation tests whether UMPNet can explore meaningful and novel articulated-object states, then examines the contributions of Arrow-of-Time and distance prediction. Results emphasize improved exploration over alternatives, while the supplied passages give no table values.
A. Open-ended state exploration
- State exploration must produce significant joint changes and visit states not previously seen; evaluation measures these with single-action effects and the unique-state ratio.The single-action metric uses thresholds of 0.15 m for prismatic joints and 8.6° for revolute joints; states within the corresponding threshold count as the same.Source: A. Open-ended state exploration · p. 5
- Arrow-of-Time awareness helps UMPNet visit novel states more frequently; observation-only methods often revisit states, while a heuristic filter is sensitive to errors that affect later actions.Where2Act achieves similar single-action effects, but Where2Act and SingleStep do not effectively explore novel states across multiple steps.Source: A. Open-ended state exploration · p. 5
- UMPNet achieves a better single-action effect across object categories, leading to more efficient state exploration over the interaction sequence.Source: A. Open-ended state exploration · p. 6
- Separating AoT classification from distance regression helps disentangle the concepts and improves prediction accuracy for both metrics compared with SignedDist.Source: A. Open-ended state exploration · p. 6
B. Goal conditioned manipulation
Goal-conditioned manipulation asks UMPNet to move an object from an initial state toward a target shown as an RGB-D image, then stop near that state. The reported comparisons find more precise target-reaching than alternative approaches, while noting weaknesses in inverse-model and AoT-only variants.
B. Goal conditioned manipulation
- The task uses a target RGB-D image and requires actions that move the object toward that state, halting when the target is reached.Source: B. Goal conditioned manipulation · p. 6
- Performance is evaluated by normalized distance to the target and success rate, with success defined as normalized distance below 0.1.Initial and goal states are selected from opposite joint limits, and initial states may be adjusted to make completion within 15 steps possible.Source: B. Goal conditioned manipulation · p. 6
- UMPNet achieves more precise goal-conditioned manipulation than the Inverse model and other alternatives, moving objects closer to target states.The Inverse model often makes sub-optimal early action choices when the current state is far from the goal; AoTOnly lacks action-effect magnitude information.Source: B. Goal conditioned manipulation · pp. 6–7
C. Inferring objects’ articulation structure from interactions
The policy’s selected actions are used to infer joint parameters as a visualization of its implicit structural belief, and the inferred axes are evaluated without joint-parameter supervision. Real-world trials demonstrate qualitative manipulation, while identifying practical gaps in sensing, object stability, and robot constraints.
C. Inferring objects’ articulation structure from interactions
- The method infers prismatic axes by averaging selected action directions and revolute axes from a common action plane and voted intersections of perpendicular lines.Source: C. Inferring objects’ articulation… · p. 7
- Action-derived joint-axis estimates have average orientation error below 11.6° for revolute joints and below 32.2° for prismatic joints, despite no joint-parameter supervision.The reported prismatic-joint error is higher because those objects tolerate more sub-optimal action directions.Source: C. Inferring objects’ articulation… · p. 7
D. Real-world experiment
- Real-world tests show microwave opening and closing trajectories and qualitative goal-conditioned manipulation on a platform with an RGB-D camera.Depth-camera noise, movement of objects without fixed bases, and unmodeled robot reachability, safety, and suction-surface constraints can affect real-world performance.Source: D. Real-world experiment · p. 7
- The authors identify large-scale real-world training as challenging and note that the policy does not account for robot reachability, trajectory safety, or suction-surface requirements.Source: D. Real-world experiment · p. 7
E. Limitations and failure cases
UMPNet has assumptions that constrain its operating scope and several documented failure cases. The conclusion reports strong performance in exploration and goal-conditioned manipulation, while a real-world experiment tests a simulation-trained model.
E. Limitations and failure cases
- The system assumes reversible action trajectories, a suction-based end-effector, and one planar 1-DoF articulated object with an image of the same scene as the goal.The reversibility assumption excludes irreversible actions such as locking, and the policy is not expected to generalize to grippers requiring more precise grasp poses.Source: E. Limitations and failure cases · p. 7
- Reported failures include ambiguous door grasp positions, actions blocked by a microwave door boundary, predicted collisions, and end-effector occlusion.The fixed camera makes the agent not robust to occlusion; better embodiment modeling could address the collision and occlusion cases.Source: E. Limitations and failure cases · pp. 7–8
- A simulation-trained model was tested on a real-world platform with an RGB-D camera and a UR5 suction gripper.For goal-directed action, the policy selects the action with the largest distance prediction and a negative Arrow-of-Time label.Source: Figure 8 · p. 8
V. CONCLUSION
- The authors report that self-guided UMPNet performs well in open-ended exploration and goal-conditioned manipulation, outperforming alternative approaches in both tasks.Training uses no human demonstrations, scripted policy, or pre-defined goal conditions.Source: §V · p. 8
APPENDICES
- The appendices provide additional details on the object dataset, network architectures, sampling strategy, and per-category quantitative results.Source: APPENDICES · p. 9
A.1. ADDITIONAL DETAILS: OBJECT DATASET
The appendix describes the object splits, network components, and action-direction sampling evaluation. It reports that CEM sampling outperforms uniform sampling at equal sample counts, motivating the final choice of 64 CEM samples.
A.1. ADDITIONAL DETAILS: OBJECT DATASET
- The experiments use 11 training and 10 testing categories from PartNet-Mobility, plus a Toy category with wave-shaped training and zig-zag-shaped testing instances.The Toy category tests complex trajectories that require the policy to change action direction at every interaction step.Source: §A.1 · p. 9
- The maximum interaction-step count for each category is set using average joint range divided by δ.Many dataset instances contain both revolute and prismatic joints; instance statistics and joint types are listed in Table A1.Source: §A.1 · p. 9
A.2. ADDITIONAL DETAILS: NETWORK STRUCTURE
- PositionNet converts RGB-D, pixel world coordinates, and surface normals into a pixel-wise two-channel position-affordance output.Its U-Net has four down-sampling and four up-sampling blocks.Source: §A.2 · p. 9
- The direction and distance branches predict a scalar distance for an observation-action pair, while the Arrow-of-Time branch uses the current and initial observations for three-way classification.The appendix describes the direction and distance network architecture and notes the AoT branch's doubled input channels and three-class output.Source: §A.2 · p. 9
A.3. EFFECT OF DIRECTION SAMPLING STRATEGY
- CEM sampling consistently achieves better goal-conditioned manipulation performance than Uniform sampling with the same number of samples.Performance improves with more samples, and the final model uses CEM with 64 action samples.Source: §A.3 · p. 9
A.4. ADDITIONAL RESULTS: OPEN-ENDED STATE EXPLORATION
The additional results describe open-ended exploration and goal-conditioned manipulation. The figures report better performance for UMPNet in long-horizon goal-conditioned tasks and attribute exploration gains to consistently directed actions with large state changes.
A.4. ADDITIONAL RESULTS: OPEN-ENDED STATE EXPLORATION
- The exploration figure compares the number of unique states visited at each step; its caption attributes better long-sequence performance to consistent action direction and maximum state change.Source: §A.4 · p. 9, Figure A3 · p. 11
A.5. ADDITIONAL RESULTS: GOAL-CONDITIONED MANIPULATION
- UMPNet outperforms other baselines on long-horizon goal-conditioned manipulation tasks.The normalized distance to target is plotted over interaction steps.Source: §A.5 · p. 9, Figure A4 · p. 12