Source-linked AI summary
FlingBot: The Unreasonable Effectiveness of Dynamic Manipulation for Cloth Unfolding
Huy Ha, Shuran Song
TL;DR
Quasi-static cloth manipulation struggles with highly crumpled configurations, varied cloth types, and sizes beyond robot reach. FlingBot learns dual-arm pick–stretch–fling actions through self-supervision from visual observations, achieving strong unfolding performance across simulated and real settings. The results support dynamic actions as effective for cloth unfolding, while real-world grasp failures and simulation collision handling remain limitations.
Problem
Prior cloth-manipulation methods mainly use single-arm quasi-static actions, requiring many interactions for challenging configurations and limiting cloth size by robot reach.
Method
FlingBot uses self-supervised learning to select dual-arm pick, stretch, and fling actions from visual observations, with a four-scalar top-down grasp representation.
Results
FlingBot achieves over 80% coverage within 3 actions on novel cloths, handles cloths beyond reach, generalizes to T-shirts, and exceeds the quasi-static baseline in real-world coverage.
Takeaways & Limitations
Dynamic manipulation can efficiently unfold cloths across challenging configurations, cloth types, and sizes beyond the system’s reach range.
Takeaways & Limitations
Real-world failures were due to unsuccessful cloth grasps, while simulation had unrealistic collision handling that could cause cloths to self-unfold.
Abstract
from arXiv · showhide
High-velocity dynamic actions (e.g., fling or throw) play a crucial role in our everyday interaction with deformable objects by improving our efficiency and effectively expanding our physical reach range. Yet, most prior works have tackled cloth manipulation using exclusively single-arm quasi-static actions, which requires a large number of interactions for challenging initial cloth configurations and strictly limits the maximum cloth size by the robot's reach range. In this work, we demonstrate the effectiveness of dynamic flinging actions for cloth unfolding with our proposed self-supervised learning framework, FlingBot. Our approach learns how to unfold a piece of fabric from arbitrary initial configurations using a pick, stretch, and fling primitive for a dual-arm setup from visual observations. The final system achieves over 80% coverage within 3 actions on novel cloths, can unfold cloths larger than the system's reach range, and generalizes to T-shirts despite being trained on only rectangular cloths. We also finetuned FlingBot on a real-world dual-arm robot platform, where it increased the cloth coverage over 4 times more than the quasi-static baseline did. The simplicity of FlingBot combined with its superior performance over quasi-static baselines demonstrates the effectiveness of dynamic actions for deformable object manipulation.
1 Introduction
FlingBot addresses the inefficiency and reach limitations of quasi-static cloth manipulation by learning dual-arm pick–stretch–fling actions from visual observations. It targets efficient, generalizable unfolding from arbitrary configurations, including cloths beyond the robot’s reach.
- Motivation: Prior quasi-static approaches require many interactions for highly crumpled cloths and limit cloth size to the robot arm’s reach range.They may also rely on strong assumptions such as predefined keypoints.
- Design Goals: The unfolding task seeks high cloth coverage from arbitrarily crumpled configurations using few actions, without relying on predefined heuristics.The desired system should also handle unseen cloth types and sizes beyond the physical workspace.
- Approach: FlingBot learns a dual-arm pick, stretch, and fling primitive through self-supervision from visual observations.The supervision signal is the change in cloth coverage before and after each action.
- Results: FlingBot achieves over 80% coverage within 3 actions on novel cloths and generalizes to T-shirts despite training on rectangular cloths.The system also handles cloths whose dimensions exceed the robot arm’s reach range.
- Contributions: The paper demonstrates dynamic manipulation as an effective approach to cloth unfolding and proposes a dual-arm grasp parameterization satisfying system safety constraints.The grasp parameterization transfers a simple single-arm grasping technique to a dual-arm setup.
- Contributions: FlingBot’s custom simulator supports arbitrarily shaped cloth meshes, including cloth types more complex than rectangular fabrics.This simulator is built on PyFlex and supports loading arbitrary cloth meshes.
2 Related Work
Prior cloth-unfolding research largely used quasi-static actions, heuristics, or expert demonstrations, leaving severely crumpled, varied, and oversized cloths challenging. FlingBot instead uses self-supervised trial and error from visual input to address these cases without expert demonstrations or ground-truth state information.
- Problem Setting: Cloth unfolding maximizes workspace coverage to expose visual features, but fully unfolding severely crumpled cloth remains challenging.Efficiently handling diverse cloth types, including cloths larger than the system’s reach range, is also difficult.
- Quasi-static Methods: Prior heuristic methods based on wrinkles, corners, or edges fail when configurations are self-occluded, keypoints are invisible, or cloths are non-square.These methods depend on identifying predefined cloth features.
- Expert Demonstrations: Expert demonstrations avoid exploration but can be sub-optimal, brittle when generated by hard-coded heuristics, or expensive when provided by humans.These trade-offs limit their appeal as a general solution for cloth unfolding.
- Self-supervision: Self-supervised prior methods removed expert dependence but operated entirely in quasi-static action spaces.They included factorized pick-and-place unfolding and spatial action maps for goal-conditioned folding.
- Dynamic Manipulation: Dynamic manipulation uses high-velocity actions that build object momentum, allowing manipulated objects to continue moving after the robot’s end-effector stops.This produces an effective increase in reach range.
- FlingBot: FlingBot addresses severely crumpled configurations and cloths beyond the robot’s reach through self-supervised trial and error from visual input.It does so without expert demonstrations or ground-truth state information.
3 Method
FlingBot learns dual-arm fling actions from visual observations by maximizing cloth coverage with self-supervised value prediction. Its method combines a constrained pick–stretch–fling primitive with transformed observations, dense action maps, and delta-coverage supervision.
- 3 Method: Dynamic actions use cloth momentum to unfold fabric beyond what quasi-static manipulation can efficiently achieve.The approach targets arbitrary crumpled states and can extend effective manipulation beyond the robot’s direct reach.
- 3 Method: FlingBot uses a dual-arm primitive that grasps, stretches taut, flings forward, and places the cloth.Stretching unfolds the cloth in one direction, while flinging uses its mass to unfold it in another.
- 3 Method: The grasp is parameterized by center point C, planar angle θ, and width w to independently enforce arm-order and safety constraints.Constraining θ to [−90°, 90°] keeps L left of R, while w is restricted to system limits.
- 3 Method: Spatial action maps recover grasp values across cloth transformations by predicting dense value maps on rotated and scaled observations.The implementation evaluates 12 rotations and 8 scale factors before rejecting grasp pairs outside either arm’s reach.
- 3 Method: Self-supervision labels each fling with normalized delta-coverage computed from top-down cloth masks before and after actions.The value network is trained in simulation and then finetuned in the real world; episodes stop after 10 timesteps or when no improving grasp is predicted.
4 Evaluation
FlingBot is evaluated against quasi-static baselines for efficiency, reach beyond the workspace, generalization to unseen cloth types, and real-world performance. Across these settings, dynamic flinging reaches high coverage in fewer actions, handles large cloths and shirts, and remains effective after real-world finetuning.
- Evaluation setup: FlingBot is evaluated on efficiency, reach range, and generalization using Normal Rect, Large Rect, and Shirt cloths.Simulation uses 2,000 training cloths and 600 novel evaluation tasks; real-world evaluation uses 10 novel test tasks per cloth type.
- Evaluation setup: Coverage is measured by final coverage, delta-coverage, and interaction count, with coverage normalized by the cloth’s maximum possible flattened coverage.Episodes run for up to 10 steps or stop when the policy predicts grasps on the floor.
- Efficiency: +63.1% coverage for FlingBot exceeds Pick&Place (+29.2%) and Pick&Drag (+24.2%) on Normal Rect, while FlingBot surpasses 80% within 3 interactions.Quasi-static baselines do not reach similarly high coverage even with significantly more steps or stretching.
- Increased Reach Range: 79.2% coverage on Large Rect gives FlingBot a +52.0% improvement, roughly twice the quasi-static baselines’ gains (+27.1%, +24.8%, +23.1%).Large Rect cloths include at least one edge larger than the robot’s 0.70m reach range.
- Generalize to Unseen Cloth Types: FlingBot reaches 93.3% on Shirt cloths despite training only on rectangular cloths, while quasi-static baselines take upwards of 8 steps to reach lower maximum coverages.The policy grasps visible corners or edges, or flings to reveal them.
- Evaluating Real-World Unfolding: After real-world finetuning, FlingBot exceeds 80% coverage on all cloth types and outperforms Pick&Place by over 40%.Finetuning used 257 experience steps across 150 cloth tasks; pre-finetuning Normal Rect coverage was 69.8%, improving by 12.1%.
- Real World Failure Cases: Real-world pipeline failures were exclusively grasping failures, with grasp success rates of 78.0%, 45.0%, and 75.8% for Normal Rect, Large Rect, and Shirt.Large-cloth grasps failed more often because the bath towel was thicker and stiffer.
5 Conclusion and Future Work
The paper concludes that dynamic flinging can make cloth unfolding efficient, generalizable, and effective beyond the robot’s reach range in simulation and the real world. It also notes that dynamic actions alone are insufficient for more complex tasks such as goal-conditioned folding.
- Conclusion: FlingBot combines a dynamic fling primitive with self-supervised learning of grasp parameters for cloth unfolding.The policy is described as efficient, generalizable, and capable of handling cloth sizes beyond the system’s reach range.
- Future Work: Dynamic actions alone are insufficient for complex deformable-object tasks such as goal-conditioned folding.Future work could combine dynamic and quasi-static, two-arm and single-arm action spaces with vision-based cloth pose estimation.
6 Supplementary Materials
The supplementary materials characterize failure modes, simulation-to-real limitations, hardware considerations, and design choices for FlingBot’s dynamic flinging pipeline.
- Failure Cases: FlingBot’s simulation failures include misaligned dual-arm grasps that become single-arm flings and prematurely terminated episodes for folded large cloths.For shirts, sleeve–collar entanglement and inner/outer-surface grasps can also produce low coverage.
- Simulation-to-Real Gap: Poor collision handling in simulation creates unrealistic self-unfolding, helping explain why simulated shirt performance is higher across all approaches.The simulator can untwist cloths or separate colliding shirt layers in configurations not observed in the real world.
- Real-world Failures: Real-world failures were caused by unsuccessful cloth grasps, which the authors suggest mitigating through specialized grippers or grasp-success estimation.The proposed hardware remedies include higher-friction fingertips, appropriate workspace compliance, and more accurate depth sensing.
- Designing Dynamic Motion Primitives: The fling primitive is tuned for high end-effector speed under joint torque limits, using greater upper-joint motion and trajectory blending for smoother cloth swinging.The parameters are optimized assuming a good dual-arm grasp on a stretched rectangular cloth.
- Designing Dynamic Motion Primitives: Fling speed may need to increase for light, thin cloths because their air resistance relative to flinging momentum is higher.Learning fling speed may require visual or additional physical information about cloth density and thickness.