PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Imitation Learning of Sequential Fabric Smoothing From an Algorithmic Supervisor

Daniel Seita, Aditya Ganapathi, Ryan Hoque, Minho Hwang, Edward Cen, Ajay Kumar Tanwani, Ashwin Balakrishna, Brijen Thananjeyan, Jeffrey Ichnowski, Nawid Jamali, Katsu Yamane, Soshi Iba, John Canny, Ken Goldberg

arXiv:1910.04854v2cs.ROcs.AIcs.CV

TL;DR

Fabric smoothing is difficult because fabric configurations and dynamics are complex. The paper trains imitation policies from a full-state algorithmic supervisor in simulation and evaluates RGB, depth, and RGBD inputs. On harder physical starting states, RGBD policies achieve the highest coverage among the tested policies.

  • Problem

    Fabric manipulation is challenging because fabric has an infinite-dimensional configuration space and unknown dynamics.

    Source: §I · p. 1

  • Method

    The paper trains fabric-smoothing imitation policies with DAgger using a simulator and full-state algorithmic supervisor, then evaluates them on a surgical robot.

    Source: §VII · p. 4, §VIII · p. 5

  • Results

    On harder physical starting states, RGBD policies achieve the highest coverage among tested policies; mean final coverage across tiers is 89.8% (RGBD), 86.3% (RGB), and 69.0% (D).

    Source: B. Physical Experiment Results · p. 6, §IX · p. 7

  • Takeaways & Limitations

    RGBD policies trained in simulation can smooth physical fabric, and depth is beneficial on harder starting configurations.

    Source: B. Physical Experiment Results · p. 6, §IX · p. 7

Abstract

from arXiv · show

Sequential pulling policies to flatten and smooth fabrics have applications from surgery to manufacturing to home tasks such as bed making and folding clothes. Due to the complexity of fabric states and dynamics, we apply deep imitation learning to learn policies that, given color (RGB), depth (D), or combined color-depth (RGBD) images of a rectangular fabric sample, estimate pick points and pull vectors to spread the fabric to maximize coverage. To generate data, we develop a fabric simulator and an algorithmic supervisor that has access to complete state information. We train policies in simulation using domain randomization and dataset aggregation (DAgger) on three tiers of difficulty in the initial randomized configuration. We present results comparing five baseline policies to learned policies and report systematic comparisons of RGB vs D vs RGBD images as inputs. In simulation, learned policies achieve comparable or superior performance to analytic baselines. In 180 physical experiments with the da Vinci Research Kit (dVRK) surgical robot, RGBD policies trained in simulation attain coverage of 83% to 95% depending on difficulty tier, suggesting that effective fabric smoothing policies can be learned from an algorithmic supervisor and that depth sensing is a valuable addition to color alone. Supplementary material is available at https://sites.google.com/view/fabric-smoothing.

I. INTRODUCTION

Fabric smoothing is challenging because fabric has an infinite-dimensional configuration space and unknown dynamics. The paper develops simulator-supervised imitation policies, compares image inputs, and evaluates transfer to a surgical robot.

I. INTRODUCTION

  • The approach uses a finite-element fabric simulator, a full-state algorithmic supervisor, DAgger, and color and camera domain randomization.Source: §I · p. 1
  • The contributions include an open-source simulator and dataset, policies trained across three starting-state difficulty tiers, and physical RGB, depth, and RGBD comparisons.Source: §I · p. 1
  • 43% to 95% coverage in a four-action simulation episode and 49% to 92% in a seven-action physical episode illustrate learned-policy smoothing.The physical episode used the da Vinci surgical robot.Source: §I · p. 1

II. RELATED WORK

  • Many prior fabric-manipulation methods rely on gravity to expose corners; this work instead adjusts fabric strewn across a surface with a single arm.The surface-based setting is presented as better suited to larger fabrics or robots with limited range of motion.Source: §II · p. 1

A. Reinforcement Learning for Fabric Manipulation

  • Prior reinforcement-learning studies address fabric folding, pants manipulation, or gauze tensioning, while this work targets smoothing highly rumpled fabric without tensioning.The cited examples include Matas et al., Ebert et al. [11], and Thananjeyan et al..Source: A. Reinforcement Learning for Fabric… · p. 2

B. Fabric Smoothing

  • Earlier smoothing methods use fixed-angle pulls or wrinkle detection, but the cited evaluations used only a few trials or nearly flat starting fabrics.The methods and evaluations are described by Willimon et al. and Sun et al.,.Source: B. Fabric Smoothing · p. 2
  • Unlike Seita et al., this work learns pick points and pull vectors jointly; concurrent work by Wu et al. [51] also transfers image-based smoothing policies from simulation.Source: B. Fabric Smoothing · p. 2

III. PROBLEM STATEMENT

The task is to manipulate a deformable fabric toward maximal coverage of a matching flat plane, using image observations to select sequential actions. The paper describes a spring-mass fabric simulator that supplies full-state simulation and rendered observations.

III. PROBLEM STATEMENT

  • The task is to move fabric from a start state toward maximal coverage of a rectangular fabric plane, measured as the percentage of the plane covered.An episode is one instance of the smoothing task.Source: §III · p. 2
  • The policy maps image observations to actions and executes sequentially until a coverage or iteration threshold is reached.Observations may be depth, color, or combined RGBD images.Source: §III · p. 2

IV. FABRIC AND ROBOT SIMULATOR

  • The custom FEM simulator exposes full fabric state and renders image observations from a triangular mesh using Blender.It also implements fabric-fabric collisions by applying a separating force when points are too close.Source: §IV · pp. 2–3
  • The simulator represents fabric as a 25 × 25 point-mass grid connected by structural, shear, and flexion springs.Point masses are subject to gravity and spring correction forces.Source: §IV · pp. 2–3
  • Spring forces depend on the displacement between connected masses relative to the default spring length, and point positions are updated using Verlet integration.The simulator adds damping to model friction-related energy loss and scales down velocity.Source: §IV · p. 3

A. Actions

Actions specify where to grasp the fabric and the direction in which to pull it. The supplied figure illustrates simulated and physical fabric images used as policy inputs.

A. Actions

  • Each action is a four-dimensional vector containing a pick point on the fabric plane and a pull vector relative to that point.The pull deltas specify x- and y-direction movement, and the simulator pulls the grasped point upward.Source: A. Actions · p. 3
  • The simulator grasps the top fabric layer at the selected point; a pick point with no fabric produces a missed grasp.A successful grasp is pulled upward and along the specified direction.Source: A. Actions · p. 3
  • The figure groups initial configurations by difficulty tier, showing simulated RGB and depth images alongside real camera images.The simulated examples shown do not use domain randomization.Source: Figure 3 · p. 3

B. Starting State Distributions

The paper defines three randomized starting-state tiers with decreasing initial coverage and describes several baseline smoothing policies. The baseline table reports final coverage and actions per episode, with both oracle policies performing best.

B. Starting State Distributions

  • Starting states span three randomized difficulty tiers: high at 78.3 ± 6.9% coverage, medium at 57.6 ± 6.1%, and low at 41.1 ± 3.4%.Figure 3 shows simulated and physical initial-state images for all three tiers.Source: B. Starting State Distributions · pp. 3–4

V. BASELINE POLICIES

  • The baseline table reports final coverage and actions per episode over 2,000 episodes with tier-specific starting states; both oracle policies perform best.Source: Table I · p. 4

1) Random:

  • The random baseline uniformly selects a pick point and pull direction at random.Source: 1) Random · p. 4

2) Highest (Max z):

  • The highest-point baseline grasps the fabric’s highest point and pulls toward where that point would lie if the fabric were flat.Prior work reported reasonably high coverage, particularly when the highest point is a corner fold on the uppermost layer.Source: 2) Highest (Max z) · p. 4

3) Wrinkle:

  • The wrinkle baseline identifies the point of largest local height variance and pulls perpendicular to a neighboring point’s variance-defined wrinkle.This implements a two-stage approach of finding wrinkles and flattening the largest one.Source: 3) Wrinkle · p. 4

4) Oracle:

The oracle pulls the fabric corner furthest from its target, while Oracle-Expose pulls away from an occluded corner to reveal it. In simulation, the oracle policies lead the baselines in coverage; the learned policies use DAgger and domain randomization.

4) Oracle:

  • The oracle policy selects the fabric corner furthest from its plane target and pulls it toward that target using complete fabric-state information.If the corner is underneath a layer, the policy grasps the fabric directly above it.Source: 4) Oracle · p. 4

5) Oracle-Expose:

  • Oracle-Expose pulls away from an occluded corner when other corners remain off-target, revealing the hidden corner for a subsequent action.Source: 5) Oracle-Expose · p. 4

VI. SIMULATION RESULTS FOR BASELINE POLICIES

  • Each baseline policy is evaluated over 2,000 randomized episodes per difficulty tier, with at most 10 actions and early termination at 92% coverage or when the fabric goes too far out of bounds.The out-of-bounds threshold permits some fabric to extend beyond the plane, but pick points must remain on the plane.Source: §VI · p. 4
  • In simulation, both oracle policies have nearly identical, highest baseline coverage—about 95% across all tiers—while wrinkles is next best at 91.3%, 87.0%, and 73.6% final coverage across three tiers, requiring substantially more actions per episode.Source: §VI · p. 4
  • When a corner is occluded, the oracle grasps the upper fabric layer above it; the pull can still move the hidden corner closer to its target, helping later actions increase coverage.In the illustrated Tier 3 episode, five actions raised coverage from 38.4% to 95.5%.Source: §VI · p. 4, Figure 4 · p. 5

VII. IMITATION LEARNING WITH DAGGER

  • DAgger supplements offline supervisor data by requesting labels for states encountered by the learned policy, addressing poor generalization beyond the training distribution.It requires continued access to the supervisor, which the state-informed oracle can provide efficiently in simulation.Source: §VII · p. 4

A. Policy Training Procedure

  • Training starts with behavior cloning on offline supervisor data, then uses DAgger; structured randomization varies RGB appearance, depth brightness, and camera pose.The camera pose is randomized for both RGB and depth images.Source: A. Policy Training Procedure · pp. 4–5

B. Simulation Experiments

In simulation, learned policies reached strong coverage, with RGB and RGBD outperforming depth-only and DAgger improving final coverage. In physical trials, policies trained in simulation smoothed fabric, with RGBD strongest on harder starting states.

B. Simulation Experiments

  • 6.1% was the average final-coverage improvement from DAgger over behavior cloning across nine simulated conditions; RGB outperformed D by 10.8%, 8.3%, and 10.9% across tiers, while RGBD performed similarly to RGB.Evaluation used 50 new tier-specific starting states not seen during training.Source: B. Simulation Experiments · p. 5, Figure 5 · p. 6
  • RGB exceeded the wrinkles policy in all tiers: 94.8% versus 91.3%, 89.6% versus 87.0%, and 91.2% versus 73.6%; depth-only exceeded it on tier 3, 80.3% versus 73.6%.RGB policies came close to the corner-pulling supervisor despite using image observations rather than underlying fabric state.Source: B. Simulation Experiments · p. 5

A. Physical Experiment Protocol

B. Physical Experiment Results

  • 89.8% final coverage for RGBD, 86.3% for RGB, and 69.0% for D were averaged across physical trials; RGBD appeared stronger on harder starts without considerably more actions.Net coverage gains were 33.4%, 25.2%, and 7.8%, respectively; the results summarize 180 episodes.Source: B. Physical Experiment Results · p. 6, Table II · p. 6
  • RGB- and RGBD-trained policies effectively fine-tune fabric with several short pulls to trigger at least 92% coverage; Figure 6 shows a tier-3 RGBD policy smoothing highly wrinkled fabric with hidden corners.Maximum coverage gives a more nuanced view because policies may achieve above 80% coverage initially, then lose coverage after a single counterproductive final action.Source: B. Physical Experiment Results · p. 6

C. Failure Cases

The policies can make counterproductive pulls, with depth-only policies particularly vulnerable to center picks on nearly smooth fabric. The conclusion reports stronger coverage for RGB and RGBD than depth, with RGBD strongest on harder starts.

C. Failure Cases

  • Depth-only policies can pull near the fabric center when it is nearly smooth, causing poor coverage and potentially cascading into states not seen during training.The resulting action may make recovery difficult.Source: C. Failure Cases · p. 6
  • Equally distant fabric corners can make the next corner to pull ambiguous; a mixture model is proposed to resolve this ambiguity.The authors also describe augmenting depth with RGB-based data as an approach to mitigate the issue.Source: C. Failure Cases · pp. 6–7
  • A tier-3 RGBD policy smoothed a highly wrinkled fabric with hidden corners from 40.1% to 92.2% coverage in seven actions.The figure shows camera views and the processed RGB and depth inputs passed to the policy.Source: Figure 6 · p. 7

IX. CONCLUSION AND FUTURE WORK

  • RGB-based and RGBD-based policies achieved higher coverage than depth-based policies on similarly colored fabric, while RGBD had the highest coverage on harder starting configurations.The authors suggest depth is beneficial in those harder configurations.Source: §IX · p. 7
  • Future work proposes testing fabric shapes where corner-pulling may give poor coverage, using deep reinforcement learning and higher-fidelity simulation, and extending to folding, wrapping, and other deformable objects.The authors also plan model-based approaches to improve sample efficiency.Source: §IX · p. 7

We structure this appendix as follows:

The appendix outlines the simulator, baseline pick methods, imitation-learning pipeline, robot setup, and yellow-fabric experiments. It describes a low-fidelity simulator and the grasp-and-pull mechanics used to model fabric actions.

We structure this appendix as follows:

APPENDIX I FABRIC ENVIRONMENT

  • The simulator is low fidelity relative to ARCSim but is easier to adapt to smoothing and enables massive data generation.Source: Appendix I · p. 9

A. Actions

  • Each simulated action has six stages: grasp, pull up, pause, linear pull to a target, a second pause, and drop.The simulator updates unpinned fabric points as pinned coordinates change.Source: A. Actions · p. 9

1) Grasp:

  • A grasp pins fabric points within a 0.003-unit sphere around the pick point, lowering the gripper until at least one point is within range.Usually 2–5 of the cloth’s 625 points are grasped.Source: 1) Grasp · p. 9

3) First Pause:

  • During the first pause, the simulator holds pinned points fixed for 80 iterations while non-pinned points settle.Source: 3) First Pause · p. 9

4) Linear Pull to Target:

The simulator implements pulls by moving pinned fabric points, then settling and releasing the cloth. Three randomized starting-state tiers create different smoothing challenges, while generally excluding a commonly studied visible single-corner fold.

4) Linear Pull to Target:

  • The simulator moves all pinned points incrementally in x and y while keeping z fixed, and updates non-pinned points according to the physics model.The number of update iterations varies with pull length.Source: 4) Linear Pull to Target · p. 9

5) Second Pause:

  • After a linear pull, pinned points remain fixed for 300 iterations while non-pinned points settle.This pause is longer than the first because more non-pinned points typically move after a linear pull than after an upward pull.Source: 5) Second Pause · p. 9

6) Drop:

  • At the end of an action, the simulator unpins the cloth and lets the entire fabric settle for 1000 iterations before the next action.Source: 6) Drop · p. 9

B. Starting State Distributions

  • Tier 1 uses short random pulls, Tier 2 starts with a vertical drop and corner drags, and Tier 3 uses a high random pull before dropping the fabric.The process samples a distribution so the agent does not see the same starting state; the distributions generally omit a single visible corner fold often shown in prior work.Source: B. Starting State Distributions · p. 9

APPENDIX II DETAILS ON BASELINE POLICIES

The appendix details state-based analytic baselines and the DAgger training pipeline. The policies use fabric geometry to choose pulls, while the oracle-expose policy additionally reasons about corner visibility.

A. Highest (Max z)

  • The highest-point baseline picks the fabric point with maximum z, while a variant randomizes among the five highest points to avoid repeatedly selecting the same point.The randomized variant addresses cases where a very short pull resets the fabric to its prior state.Source: A. Highest (Max z) · p. 10

B. Wrinkles

  • The wrinkle baseline locates the point of greatest local height variance, estimates wrinkle direction from a neighboring point, and pulls perpendicular to it toward the fabric edge.Unlike Sun et al., it uses the exact perpendicular line rather than constraining direction to eight cardinal angles.Source: B. Wrinkles · p. 10

C. Oracle

  • The oracle policy chooses the fabric corner furthest from its target and pulls it toward that target, using the known corner coordinates.Source: C. Oracle · p. 10

D. Oracle Expose

  • Oracle expose uses corner positions and visibility states; it reveals an occluded corner when all are hidden or visible corners are already near their targets.A revealing action pulls a fixed distance 180 degrees away from the target direction.Source: D. Oracle Expose · p. 10

A. DAgger Pipeline

  • Each DAgger iteration adds 200 oracle-labeled samples, trains on 240 minibatches, and repeats with the updated policy.Training is capped at 50,000 DAgger steps due to compute and time limitations; the network has about 3.4 million parameters.Source: A. DAgger Pipeline · p. 10

B. Domain Randomization and Simulated Images

The training images use domain randomization, while physical images are processed before they reach the policy. The robot also uses a visual check to correct a pick that barely misses the fabric.

B. Domain Randomization and Simulated Images

  • Training randomizes fabric colors, plane shading, and camera pose for RGB and depth inputs, but does not randomize simulator parameters.RGB brightness and pixel noise are varied, and simulated depth images are darkened and noised.Source: B. Domain Randomization and Simulated… · pp. 10–11
  • When the robot barely misses the fabric, it detects no fabric change, moves the pick closer to the center, and then pulls to achieve high coverage.Source: Figure 8 · p. 11

A. Image Processing Pipeline

  • Physical color and depth images are cropped to 100×100; depth images are in-painted, clipped, scaled, and histogram-equalized, and both modalities are filtered and denoised.The processed images are passed to the policy network.Source: A. Image Processing Pipeline · p. 11

B. Physical Experiment Setup and Procedures

The physical setup calibrates policy positions to the robot frame, and the appendix tests policies on yellow fabric outside the RGB training color range. In that test, depth outperforms RGB in ending coverage.

B. Physical Experiment Setup and Procedures

  • Checkerboard calibration maps policy coordinates to robot positions by bilinear interpolation, with 1–2 mm positioning error on the fabric plane.A heuristic also helps the robot grasp fabric after an initial miss of 1–2 mm.Source: B. Physical Experiment Setup and… · p. 11

APPENDIX V ADDITIONAL EXPERIMENTS

  • Yellow lies outside the colors covered by training randomization, and five test conditions were recreated from cases where blue fabric reached at least 92% coverage in one action.Source: Appendix V · p. 11
  • On yellow fabric, RGB coverage falls from 81.5% to 71.7%, while the depth-only policy reaches 85.9% ending coverage.Only two of five RGB trajectories reach at least 92%; the 85.9% depth result exceeds the 78.8% reported in Table II because the starting states were relatively easier.Source: Appendix V · pp. 11–12
  • Simulated starting-state examples include randomized color and depth images, whereas the physical camera examples have no domain randomization and depict manipulated fabrics resembling simulated states.The physical color and depth examples correspond to one another.Source: Figure 9 · p. 12, Figure 10 · p. 12, Figure 11 · p. 13
  • In the illustrated tier 1 RGB episode, the first pull moves fabric away from the plane, and the plane remains barely covered after several actions.Source: Figure 13 · p. 13
Loading 1910.04854v2…