PaperlayerView exact revision on arXiv

Source-linked AI summary

Same Action, Different Outcome: Variability in Dynamic Cloth Manipulation

Mahed Dadgostar, Guillem Alenyà, Júlia Borràs

arXiv:2610.10801v1cs.RO

TL;DR

Repeated identical dynamic cloth motions can produce different outcomes, a variability that matters for assessing manipulation reliability and simulation fidelity. The paper quantifies it across tasks and cloths, then finds it is significant in every experimental configuration and is not reproduced by four modern cloth simulators.

  • Problem

    Variability across nominally identical dynamic cloth trials has not been systematically characterized, limiting assessment of policy reliability and simulation fidelity.

    Source: §I · p. 1

  • Method

    The authors formalize interaction variability and floor variability, then analyze 269 executions across four tasks and replay them in four cloth simulators.

    Source: A. Interaction variability · p. 2, §I · p. 2, §VII · p. 8

  • Results

    Variability is significant across all experimental configurations, depends mainly on cloth properties and less on speed, and is not reproduced in magnitude or ordering by four simulators.

    Source: §I · p. 2, §VII · p. 8

  • Takeaways & Limitations

    Reporting cloth physical properties matters for interpreting robotic results, and the dataset supports training perception methods and evaluating simulators and cloth-dynamics models.

    Source: §VII · p. 8

Abstract

from arXiv · show

Although cloth is known to exhibit different outcomes under repeated fast dynamic motions, even when the same trajectory is applied, this variability has not yet been systematically characterized. Quantifying it is essential to assess the reliability of learned manipulation policies and the extent to which simulation can reproduce real-world behavior. To study this, we execute the same trajectory ten times across four dynamic tasks, two of which are novel, each tested with three cloths of very different properties and at up to three execution speeds, with a total of 269 recorded rollouts. For all of them, we record small marker positions on the cloth and synchronized stereo camera. We then formalize different metrics to quantify variability, and our results show how it is significant in every test condition, in most cases one to three orders of magnitude above the repeatability inherent to the robot and sensing noise. Our results also show variability is driven mainly by the cloth physical properties but also grows with speed. By replaying all the rollouts in four calibrated modern cloth simulators, we show that none of them can reproduce the variability magnitude we observed in real cloth, nor its ordering. Together with our analysis, we publish the dataset with synchronized OptiTrack, vision and robot logs and their corresponding simulator twins.

I. INTRODUCTION

The introduction contrasts dynamic cloth manipulation with predominantly quasi-static datasets and notes that repeated dynamic executions can produce different cloth states. It presents a formal analysis, a multi-task dataset, and a simulator evaluation to characterize that variability.

I. INTRODUCTION

  • Dynamic cloth studies and VLA systems typically report task success or handle quasi-static tasks, leaving variability across nominally identical trials uncharacterized.Dynamic execution also faces aerodynamic, frictional, and other environmental effects.Source: §I · p. 1
  • Real cloth trajectories can diverge under repeated identical motions because simulators coarsely model aerodynamics and omit turbulence and fine-grained cloth-state changes.The paper argues that domain randomization cannot cover effects that are not simulator parameters.Source: §I · p. 1
  • The paper formalizes how identical dynamic cloth manipulations can yield different outcomes depending on the manipulation and cloth features.Its analysis covers four dynamic tasks.Source: §I · p. 2
  • None of four calibrated cloth simulators reliably reproduces the magnitude or ordering of real-world variability.The simulators replay measured inputs.Source: §I · p. 2
  • The dataset contains 269 motion-captured rollouts across four tasks, including two novel inward-gripper motions, with synchronized stereo video, robot logs, and simulator twins.The novel motions move the grippers toward each other, with and without a swing.Source: §I · p. 2

II. STATE OF THE ART

Prior work observed or assumed trial-to-trial variability in dynamic cloth manipulation but had not characterized its distribution across fabrics, speeds, or grasps. The section formalizes variability using repeated rollout features, terminal marker dispersion, a noise floor, and a speed contrast.

II. STATE OF THE ART

  • Earlier studies observed spread in repeated cloth motions or recorded repeated folds without analyzing their spread across fabrics, speeds, and grasps.Related work also notes that robust MPC and iterative learning control assume disturbances are bounded or repeat across trials [20], while dynamic cloth outcome distributions had not been reported.Source: §II · p. 2
  • The dataset extends earlier motion-capture work with novel dynamic tasks and synchronized image perception.It includes a fling and single-hand dynamic folding, plus two inward-gripper tasks described as previously unstudied.Source: §II · p. 2

A. Interaction variability

  • A rollout configuration comprises the task policy, object, and execution velocity, while its task-dependent feature is modeled over repeated samples of the same policy.The feature distribution is represented as p_real; its state transition is conditioned on the current observation and control input.Source: A. Interaction variability · p. 2
  • Terminal dispersion is the mean distance between corresponding cloth-marker positions across rollout pairs, using markers observed in both trials.Marker identities are manually supervised, so marker matching is unnecessary.Source: A. Interaction variability · p. 3
  • Floor variability uses markers at grasped corners to measure variation from the robot, grasp, and sensing; dispersion above this floor marks a non-repeatable configuration.The logged tool pose and gripper markers repeat to less than 0.4 mm in every rollout.Source: A. Interaction variability · p. 3
  • The speed contrast compares a cloth’s fastest and slowest dispersion without regrasp; a value above 1 means faster execution scatters more.The trajectory-duration multiplier is defined so that the minimum value is the fastest execution.Source: A. Interaction variability · p. 3

A. Experiment Setup

The experiment setup combines bimanual cloth manipulation, motion capture, stereo video, and robot tracking with three same-sized cloths spanning distinct physical properties.

A. Experiment Setup

  • Two Franka Research 3 arms with default hands are arranged side by side across a table.OptiTrack tracks cloth markers at 100 Hz, while a stereo camera records the scene at 30 Hz; markers on table tags register the camera and motion-capture frames.Source: A. Experiment Setup · p. 3
  • The four tasks are tested across 27 configurations.The table caption identifies the scope but does not specify the individual configurations.Source: Table II · p. 3
  • The setup uses three 0.7×0.5 m cloths with markedly different stiffness, elasticity, softness, and friction.The brown wool is thick and stiff; green satin is lighter and less stiff and frictional; red chiffon is very light, elastic, soft, and has the lowest friction coefficient.Source: A. Experiment Setup · p. 3

B. Protocol

The protocol varies cloth, execution speed, and—in selected configurations—grasping while repeating each configuration ten times from the same robot starting pose. Trajectories are teleoperated for tasks 1, 2, and 4, and scripted for task 3.

B. Protocol

  • Each of 27 configurations has ten identical rollouts, varying cloth, speed, and grasp in selected configurations.Fixed-grasp trials retain the same grasp posture; regrasp trials manually regrasp before each rollout while keeping the robot's initial pose the same.Source: B. Protocol · p. 3
  • Tasks 1, 2, and 4 use open-loop teleoperated demonstrations, whereas task 3 uses a scripted joint-space trajectory.The teleoperation device is a rigid body tracked in 6-D with OptiTrack markers.Source: B. Protocol · p. 3
  • Task 4 with the brown cloth at ts = 1.5 has nine recorded rollouts rather than ten.Source: B. Protocol · p. 3

C. Recorded Tasks

The four recorded tasks measure cloth outcomes after swing-place, dynamic-fold, and inward-gripper motions, with the latter performed both linearly and with a swing. Task 4 adds dynamics to the inward motion to study buckling-side direction.

C. Recorded Tasks

  • Swing place measures the resting positions of free-edge markers m1–m5 after the cloth is swung forward and laid flat without release.Resting pose is the mean position over the last 50 motionless frames; distances are projected onto the table plane.Source: C. Recorded Tasks · p. 4
  • Dynamic fold swings free corner m5 toward target m15 while the opposite edge remains in place, and evaluates m5's resting position.Source: C. Recorded Tasks · p. 4
  • Linear inward gripper motion moves held corners m5 and m15 toward each other, then measures buckling of free top-edge markers relative to their line.The cloth ends hanging crumpled; marker positions quantify buckling distance and which side the markers fall on.Source: C. Recorded Tasks · p. 4
  • Swing inward gripper motion adds a cloth swing to the same inward-gripper goal, and is a novel dynamic task.Source: C. Recorded Tasks · pp. 4–5

D. Simulator twins

The simulator twins replay fixed-grasp rollouts in four cloth simulators using fitted starting states, measured gripper trajectories, and a calibrated table plane. Each simulator is calibrated per cloth, then simulated marker data are processed through the real-data analysis pipeline.

D. Simulator twins

  • Four simulators replay every no-regrasp rollout for tasks 1–4 from fitted marker poses, 100 Hz gripper poses, and the measured table plane.The cloth is relaxed quasi-statically under the pinch before being driven; the engines are Clothilde, Isaac Lab PhysX, Genesis PBD, and Newton VBD, all with self-collision enabled.Source: D. Simulator twins · p. 5
  • Each simulator is calibrated separately for each cloth using nine training rollouts per cloth spanning all tasks and speeds.Calibration minimizes mean simulated-to-real marker distance over a coarse parameter grid followed by one-at-a-time refinement; eight held-out rollouts per cloth are used for evaluation.Source: D. Simulator twins · p. 5
  • Held-out marker errors are 37–48 mm for Clothilde, 70–78 mm for Isaac Lab, 82–90 mm for Genesis, and 79–94 mm for Newton.Source: D. Simulator twins · p. 5
  • Simulated markers are written into the motion-capture format so the real-data analysis code can compare simulated and real dispersion per configuration.Source: D. Simulator twins · p. 5

V. RESULTS AND ANALYSIS

Outcome variability differs widely across tasks and configurations, with cloth type explaining more of the observed spread than task or execution speed. Faster motion generally increases variability, while regrasping raises the repeatability floor but does not significantly change variability relative to the comparable fixed-grasp task.

A. Comparison of Variability Across Tasks

  • 1.2 mm for the brown cloth slow swing join to 118 mm for folding with green satin spans a 96-fold range; every no-regrasp configuration exceeds its grasped-marker floor by at least 2.4×.Most configurations exceed that floor by one to three orders of magnitude.Source: A. Comparison of Variability Across… · p. 5
  • For tasks 3 and 4, the figure encodes buckling distance from the gripper-held line for visible top-edge markers and counts which side they buckle toward.Source: Figure 7 · p. 6
  • Cloth explains 61% of the spread in log10 mean pair distance, versus 10% for task and 6% for speed; the two light cloths scatter three to ten times more than the brown cloth.The brown cloth is most repeatable except in the fast swing join, where it reaches the red cloth's variability; the authors suggest stiffness-related buckling could explain this exception.Source: A. Comparison of Variability Across… · p. 6
  • Higher velocity increases variability in all reported speed contrasts except dynamic folding with green satin, where ΓD = 0.97; the maximum is ΓD = 21 for brown wool in swing inward gripper motion.For brown wool in that motion, variability rises from 1.2 mm at slow speed to 25.9 mm at faster speed.Source: A. Comparison of Variability Across… · p. 6
  • Regrasping raises floor variability from 2–4 mm to 12–27 mm, but outcomes still exceed the floor by 2.6–4.2× and regrasping does not significantly affect variability.Source: A. Comparison of Variability Across… · p. 6

B. Analysis of the variability per task

Repeated dynamic cloth motions produce markedly different outcomes, with dispersion varying by cloth properties and, in some tasks, speed. The task-specific analyses show distinct sensitivities, while simulator variability does not match the real variability.

Swing place (task 1)

  • Swing-place resting markers spread 5–9 mm for thick cloth, 23–47 mm for green satin, and 33–81 mm for red chiffon; light-cloth spread grows with speed.Every configuration exceeds floor variability by 14–1250×.Source: B. Analysis of the variability per task · p. 6
  • At high speeds, light-cloth variability becomes noticeable along the Y axis as well as the direction of motion, indicating more complex aerodynamic effects.Source: B. Analysis of the variability per task · p. 6

Dynamic fold (task 2)

  • Dynamic-fold free-corner dispersion is 7–12 mm for thick fabric versus 55–118 mm for light fabrics, and the corner misses its target by 7–41 cm in every configuration.Reducing speed by 40% does not significantly affect the result.Source: B. Analysis of the variability per task · p. 6

Simulators

  • The simulator comparison indicates that none of the simulators captures the real variability.Source: Figure 8 · p. 7

Dynamic fold (task 2)

  • The dynamic-fold floor measure is below 1 mm, 39–800 times smaller than free-corner dispersion, and the task is highly sensitive to cloth properties.The authors note that a policy optimized to reach the target is unlikely to generalize across cloth types.Source: B. Analysis of the variability per task · p. 7

Linear and swing inward gripper motion (task 3 and 4)

  • In the inward-gripper tasks, swing effects depend on fabric: it increases buckling-magnitude variability for stiff wool, directs satin buckling at the tested speed, and has little effect on red chiffon.For satin, the slower motion does not reliably direct buckling; the chiffon can slide back between the grippers before the swing finishes.Source: B. Analysis of the variability per task · p. 7

C. Simulators

The simulator evaluation finds that no engine reproduces real-cloth variability magnitude and ordering reliably. Markerless stereo vision can support qualitative comparisons, but it does not measure dispersion with OptiTrack precision.

C. Simulators

  • No simulator reproduces real variability magnitude or ordering reliably; even the better-performing Clothilde and Genesis preserve the ordering imperfectly.Clothilde does relatively well on swing-to-place, but the same cloth can be under-dispersed in one task and over-dispersed in another.Source: C. Simulators · p. 7
  • Isaac Lab and Newton produce non-identical replays, but their noise is misplaced across tasks and cloths rather than replacing real variability.Isaac Lab noise does not change with cloth or task; Newton noise peaks in a highly repeatable case and vanishes in a less repeatable one.Source: C. Simulators · p. 7

VI. MEASURING THE DISPERSION WITHOUT MARKERS

  • The markerless pipeline segments terminal images with SAM 2, extracts 3-D with FoundationStereo, and uses DINOv3 patches as virtual markers for computing distances.Source: §VI · p. 7
  • Vision preserves the general ordering trend only imperfectly, with over- and under-estimates, and particularly underestimates dynamic-fold dispersion when the corner blends into the cloth.Stereo vision supports qualitative comparisons, not dispersion measurements at OptiTrack precision.Source: §VI · p. 8, Figure 9 · p. 8

VII. CONCLUSIONS

The paper quantifies dynamic cloth variability across four tasks and 27 configurations, finding it significant and driven mainly by cloth properties, with a smaller contribution from speed. Four simulators fail to reproduce the real variability, and the authors release a synchronized dataset.

VII. CONCLUSIONS

  • The study characterizes variability across 269 executions, four dynamic tasks, and 27 trial configurations, including two novel tasks.Source: §VII · p. 8
  • Variability is significant across all experimental configurations and depends more on cloth properties than on speed.The authors conclude that cloth physical properties should be reported to interpret the capabilities of evaluated methods.Source: §VII · p. 8
  • None of four modern cloth simulators reproduces observed real-trial variability; the released labeled dataset includes synchronized stereo and proprioception, with simulator twins and additional analysis.The authors identify addressing the sim-to-real gap and policy generalizability across cloth types as future work.Source: §VII · p. 8
Loading 2610.10801v1…