PaperlayerView exact revision on arXiv

Source-linked AI summary

Outcome-Sensitive Motion Search for Impact-Aware Dexterous Catching

Guorui Pei, Jinsong Wu, Songyuan Su, Jiaming Qi, Sichao Liu, David Navarro-Alarcon, Bin Liu, Peng Zhou

arXiv:2609.29020v1cs.RO

TL;DR

Impact-aware catching is difficult because it couples interception, contact regulation, and grasping, and a capable privileged RL teacher may not supply suitable demonstrations for a deployable IL student. Outcome-Sensitive Motion Search refines and repairs pre-contact motions, then validates complete rollouts under modeled student errors. In simulation, the resulting student surpasses the privileged teacher in catching success and impact mitigation.

  • Problem

    Impact-aware catching couples interception, contact regulation, and stable grasping, while privileged RL teachers may yield incomplete or outcome-variable demonstrations for deployable IL students.

    Source: §I · p. 1

  • Method

    The method searches a task-conditioned manifold of successful OSW motions to refine teacher successes and repair failures, then validates complete rollouts under modeled student action errors.

    Source: §I · pp. 1–2

  • Results

    In simulation, the resulting observation-restricted student surpasses the fixed privileged teacher while reducing impact force.

    Source: §VI · p. 8

  • Takeaways & Limitations

    The simulation results demonstrate the importance of demonstration quality and robustness to modeled student execution errors for impact-aware catching.

    Source: §VI · p. 8

Abstract

from arXiv · show

Skilled humans can catch fast-moving objects softly by coordinating interception, velocity matching, and follow-through to mitigate impact. Learning such impact-aware catching with reinforcement learning (RL), however, is challenging, as the policy must achieve reliable interception and grasping while regulating the sensitive transition into contact. Moreover, even a capable privileged-state RL teacher may not provide ideal demonstrations for a deployable imitation-learning (IL) student: teacher failures limit task coverage, while small variations in pre-contact motion can produce substantially different impact and grasping outcomes. We characterize this phenomenon through interventional outcome sensitivity and introduce the outcome-sensitive window (OSW) to guide targeted demonstration construction. Building on this formulation, we propose Outcome-Sensitive Motion Search, which learns a task-conditioned manifold of successful OSW motions and performs local geodesic search to refine successful teacher rollouts and repair task conditions where the teacher fails. We then validate candidate motions through complete rollouts under a calibrated IL-student action-error model and retain only successful executions as demonstrations. Extensive simulation experiments demonstrate that our method effectively repairs task conditions where the teacher fails and enables the resulting IL policy to outperform the privileged RL teacher in both catching success and impact mitigation.

I. INTRODUCTION

Impact-aware catching requires coordinating interception, contact regulation, and stable grasping, while privileged RL teachers may produce demonstrations poorly suited to deployable IL students. The paper searches outcome-sensitive pre-contact motions and validates candidates under modeled student execution errors.

I. INTRODUCTION

  • Catching requires simultaneous interception, regulation of the transition into contact, and maintenance of a stable grasp.Privileged-state RL can learn effective behaviors, but teacher failures limit success-only demonstration coverage.Source: §I · p. 1
  • Small pre-contact motion variations can substantially change interception, impact, and subsequent grasping outcomes.The paper characterizes this nonuniform action influence as interventional outcome sensitivity and defines the outcome-sensitive window as the short pre-contact interval where local interventions strongly affect downstream outcomes.Source: §I · p. 1
  • Outcome-Sensitive Motion Search learns a task-conditioned manifold of successful OSW motions and searches locally to refine successful rollouts or repair teacher failures.Each candidate is evaluated by a complete simulator rollout to account for downstream contact and grasping consequences.Source: §I · pp. 1–2
  • In simulation, the resulting IL policy surpasses the privileged RL teacher in catching success and impact mitigation.Candidates are evaluated through complete rollouts under a calibrated IL-student action-error model, and only qualified executions are retained for training.Source: §I · p. 2

II. RELATED WORK

Prior work combines model-based impact-aware control with learning-based catching, but learned policies and demonstrations can remain imperfect. The section also describes learner-robustness approaches and defines catching success using contact, impact, follow-through, and grasp criteria.

A. Impact-Aware Manipulation

  • Model-based interception and impact-aware controllers rely on accurate state and dynamics models and may require substantial online computation, while learned catching policies can remain imperfect.The paper uses a learned policy as a teacher and further improves its experience before training a final student.Source: A. Impact-Aware Manipulation · p. 2

B. Policy Learning from Imperfect Data

  • DAgger and DART address learner execution errors, and this work jointly considers demonstration improvement and learner robustness because near-contact deviations can alter downstream outcomes.Prior demonstration filtering, reweighting, and refinement do not guarantee effective closed-loop imitation under unperturbed execution alone.Source: B. Policy Learning from Imperfect Data · p. 2

A. Task and Success Criteria

  • A rollout succeeds only with admissible first contact, bounded contact-relative speed and peak force, sufficient follow-through, and sustained grasp.The task contrasts a privileged-state RL teacher with a student that receives only deployable observation and action history.Source: A. Task and Success Criteria · p. 2, Figure 2 · p. 2

B. Interventional Outcome Sensitivity

Interventional outcome sensitivity describes how local action changes within a trajectory window affect downstream outcomes. For catching, the outcome-sensitive window is aligned to the event just before which motion can most strongly affect the result.

B. Interventional Outcome Sensitivity

  • Interventional outcome sensitivity measures how replacing a window’s action snippet with a local perturbation changes the downstream outcome.The intervention is considered under the same task condition.Source: B. Interventional Outcome Sensitivity · p. 3
  • A temporal segment with high interventional outcome sensitivity is termed an outcome-sensitive window (OSW).The variation between downstream outcomes is measured by dY.Source: B. Interventional Outcome Sensitivity · p. 3
  • For catching, the OSW is aligned to the first contact—or, if contact does not occur, the palm’s closest approach to the object.For successful rollouts, this alignment event coincides with first admissible hand–object contact, localizing search to the motion with the strongest downstream influence while retaining the rest of the teacher behavior.Source: B. Interventional Outcome Sensitivity · p. 3

C. Outcome-Sensitive Demonstration Construction

Demonstration construction seeks successful examples that improve an observation-restricted student. It refines teacher-success cases and repairs teacher-failure cases by changing the OSW motion while retaining teacher actions outside that window.

C. Outcome-Sensitive Demonstration Construction

  • For teacher-success conditions, the search seeks an alternative OSW motion that preserves success, reduces impact, and is robust to modeled student action errors.Actions outside the OSW remain governed by the frozen teacher.Source: C. Outcome-Sensitive Demonstration… · p. 3
  • For teacher-failure conditions, the search seeks an OSW replacement that makes the rollout successful and robust as a demonstration.Actions outside the OSW remain governed by the frozen teacher.Source: C. Outcome-Sensitive Demonstration… · p. 3
  • Only complete rollouts meeting task-success criteria under their original conditions are retained as positive student-training demonstrations.The objective is to construct demonstrations that improve closed-loop performance of an observation-restricted student.Source: C. Outcome-Sensitive Demonstration… · p. 3

IV. OUTCOME-SENSITIVE MOTION SEARCH

Outcome-Sensitive Motion Search generates candidate arm-action snippets within the catching OSW and evaluates complete rollouts, including under modeled student action errors. It learns a task-conditioned manifold of successful OSW motions and searches locally using a decoder-induced metric.

IV. OUTCOME-SENSITIVE MOTION SEARCH

  • The method searches candidate arm-action snippets in the OSW, using a manifold learned from successful teacher experience to generate local geodesic replacements.Candidate motions are evaluated through complete simulator rollouts under unperturbed execution and calibrated student action errors; only successful executions enter the demonstration dataset.Source: §IV · pp. 3–4

A. OSW Motion Manifold Learning and Geodesic Search

  • The OSW motion manifold encodes successful arm-action snippets in a low-dimensional latent space, learned with a conditional variational model.Its decoder maps a latent code and task condition augmented with the privileged simulator state at OSW start to an action snippet; training combines reconstruction, temporal-smoothing, and KL-divergence losses.Source: A. OSW Motion Manifold Learning and… · p. 4
  • For each rollout, the original OSW snippet is projected onto the learned manifold, with the conditional prior discouraging projection into low-probability regions.The projected center initializes a multi-start local search, and its decoded motion need not exactly reproduce the original RL snippet.Source: A. OSW Motion Manifold Learning and… · p. 4
  • A decoder-induced pullback metric measures latent displacement by its effect on decoded motion, and geodesic endpoints around the projected center form the candidate set.The shooting radius keeps candidates local to the projected original motion and within the bounded latent region; each still requires a complete simulator rollout because local candidates are not guaranteed to succeed.Source: A. OSW Motion Manifold Learning and… · p. 4

B. Event-Aligned OSW Intervention

The OSW is the fixed-length interval immediately preceding the rollout-specific interception event. During candidate evaluation, the teacher retains hand control through the intervention and resumes full control at contact or the OSW’s end.

B. Event-Aligned OSW Intervention

  • For catching, the OSW comprises the HOS control steps immediately before the rollout-specific interception event.If the event occurs too early to provide a complete HOS-step window, search is skipped and the original rollout is retained only if successful.Source: B. Event-Aligned OSW Intervention · p. 4
  • Within the OSW, a candidate arm snippet runs open loop while the frozen teacher maintains closed-loop hand control; full teacher control resumes at contact or the OSW’s end, whichever comes first.Each candidate is evaluated by its complete rollout, not only the modified OSW segment.Source: B. Event-Aligned OSW Intervention · p. 4

C. IL-Student Action-Error Modeling

The paper models action errors of deployable IL students and uses complete perturbed rollouts to select refined or repaired OSW candidates for demonstrations. Eligibility depends on whether the original RL rollout succeeded.

C. IL-Student Action-Error Modeling

  • The phase-dependent student-error model captures delay, channel-wise gain and bias, and temporally correlated residuals across catching phases.Errors are estimated from cross-fitted IL students’ deviations from the frozen RL teacher; phases are determined online from object–palm proximity and contact history.Source: C. IL-Student Action-Error Modeling · pp. 4–5

D. Candidate Validation and Dataset Construction

  • Candidate motions are evaluated with one unperturbed and Q action-error-perturbed complete rollouts, using shared random residuals and innovations for fair comparison.The objective balances impact mitigation against deviation from the original OSW motion.Source: D. Candidate Validation and Dataset… · p. 5
  • For successful originals, eligible candidates must succeed unperturbed, improve the objective, and match or exceed the original perturbed success rate; repairs use an absolute success-rate threshold.The lowest-cost eligible candidate is selected, with ties resolved by higher success rate and then shorter geodesic distance; if none qualifies, a successful original is retained and a failed condition is discarded.Source: D. Candidate Validation and Dataset… · p. 5
  • The final dataset retains successful selected-candidate executions, including successful perturbed rollouts, alongside retained successful RL originals.The student trains on deployable observation histories and previous actions, with labels from actions actually executed in retained rollouts, including modeled perturbations.Source: D. Candidate Validation and Dataset… · p. 5

V. EXPERIMENTS

The experiments evaluate impact mitigation, closed-loop effects of demonstration construction, performance across task conditions, and individual component contributions.

  • The evaluation covers impact mitigation, closed-loop performance, task-condition variation, and component contributions.Source: §V · p. 5

A. Experimental Setup

The study tests catching with a randomized simulation setup and compares an impact-mitigating RL teacher with a catching-only baseline. Despite similar catch completion, the impact-mitigating teacher achieves higher task success and lower peak force.

A. Experimental Setup

  • The simulation uses a 7-DoF xArm7 and 17-DoF ORCA Hand in MuJoCo, with randomized object properties, launch conditions, and initial arm–hand posture.Object speeds are approximately 2.8–5.0 m/s, and evaluation also randomizes observation and actuation effects, timing jitter, and contact dynamics.Source: A. Experimental Setup · pp. 5–6
  • The study evaluates ten shape–size variants and uses three final-student training seeds on the same 3,000 held-out task conditions.Impact metrics are reported only for catch completions.Source: Figure 5 · p. 6, A. Experimental Setup · p. 6
  • Ours-IL uses Outcome-Sensitive Motion Search demonstrations, while Naive-IL and Normal-IL use success-only filtering and unperturbed-only validation, respectively.All final students share an interface, architecture, action-chunk length, and optimization protocol; dataset sizes are controlled across baselines.Source: A. Experimental Setup · p. 6

B. Catching with Impact Mitigation

  • 84.8 percentage points higher task success and 74.5% lower peak force were reported for the impact-mitigating teacher than the catching-only RL baseline.The baselines had similar catch completion; Fig. 7 profiles illustrate a lower impact peak and more gradual deceleration during follow-through.Source: B. Catching with Impact Mitigation · p. 6

C. Demonstration Construction and Policy Performance

Ours-IL improves catching success over the RL teacher while reducing impact force, with gains across object and target-offset conditions. Ablations associate refinement with impact mitigation and repair with most success gains, while full geodesic search and temporal-error modeling produce the strongest reported outcomes.

E. Ablation Studies

  • Combining refinement and repair yields 90.3% task success with 37.3 N peak impact force; refinement primarily improves impact mitigation, while repair provides most of the success gain.Refinement-only reaches 83.2% success and 36.2 N, while repair-only reaches 88.5% and 42.7 N; success-only filtering reaches 83.8% and 41.1 N.Source: E. Ablation Studies · p. 7

D. Performance Across Task Conditions

  • Ours-IL improves all ten shape–size combinations, with the largest gain of 14.5 percentage points on the large box, and has the highest success across lateral target offsets.Figure 9 compares teacher success by object family and size, Ours-IL gains over the teacher, and success versus absolute lateral target offset.Source: Figure 9 · p. 7, D. Performance Across Task Conditions · p. 7

E. Ablation Studies

  • Full geodesic search reaches 44.3/59.8/90.3% repair, refinement, and student-success rates, versus 5.6/8.3/83.4% with direct perturbations.The intervening results are 25.7/33.1/86.8% for an unconditional manifold and 39.6/51.9/88.5% for conditional Euclidean search.Source: E. Ablation Studies · p. 7

C. Demonstration Construction and Policy Performance

  • 90.3% task success exceeds the RL teacher by 5.1 percentage points and Naive-IL by 6.5 points, while peak impact force falls 8.6% relative to the teacher.Unperturbed-only validation has the highest coverage and lowest unperturbed fmax, but Normal-IL trails Naive-IL by 2.1 points; Fig. 7 also shows lower pre-contact relative speed and a lower impact peak for Ours-IL.Source: C. Demonstration Construction and… · p. 7, Figure 7 · p. 7, Table II · p. 8

E. Ablation Studies

  • Phase conditioning improves moment matching, while temporal correlation gives the lowest lag-one error and highest student task success despite a lower repair rate.The compared models use identical candidate sets and matched dataset size; the full phase-wise correlated model is compared with global i.i.d. Gaussian and phase-wise independent models.Source: Table III · p. 8, E. Ablation Studies · p. 8
  • The search cannot repair failures outside the learned OSW manifold, including cases requiring different hand control or post-contact recovery.Candidate selection depends on simulated contact dynamics and an action-error model calibrated from the IL student; robustness to real perception and contact mismatch remains unverified without physical experiments.Source: E. Ablation Studies · p. 8

VI. CONCLUSION

In simulation, the observation-restricted final student surpasses the fixed privileged teacher while reducing impact force. The conclusion presents OSW motion search and complete-rollout validation under modeled student errors as the route to robust demonstrations, with physical deployment and broader throwing conditions left for future work.

VI. CONCLUSION

  • In simulation, the observation-restricted final student surpasses the fixed privileged teacher while reducing impact force.Source: §VI · p. 8
  • Outcome-Sensitive Motion Search constructs demonstrations robust to modeled student execution errors from mixed RL experience through OSW manifold search and complete-rollout validation.Source: §VI · p. 8
  • Physical arm–hand deployment and broader throwing conditions remain future work.Source: §VI · p. 8
Loading 2609.29020v1…