PaperlayerView exact revision on arXiv

Source-linked AI summary

What is the Better Curriculum: Controller-Shaped Grasping Behavior for Contact Force-Sensitive Manipulation

Ziyan Feng, Zizhao Yuan, Yulong Fu, Yuxin He, Zhiyuan Zhang, Zhengjie Zhang, Jinni Zhou, Renjing Xu, Qiang Nie

arXiv:2609.25887v1cs.RO

TL;DR

Fragile direct-contact grasping requires maintaining a narrow force range, but manual collection can be too delayed and coarse to produce repeatable demonstrations. The paper uses a deterministic tactile reflex as a collection-time teacher for tactile-free policies. Reflex-shaped data improves in-distribution grasp quality across ACT and π0.5, while disturbance rejection remains dependent on deployment-time tactile arbitration.

  • Problem

    Manual gripper control can be too delayed and coarse to maintain the narrow force range required for stable grasping of fragile objects.

    Source: §1 · p. 2

  • Method

    A deterministic vision-based tactile reflex regulates the gripper during teleoperated data collection, producing controller-shaped demonstrations for tactile-free policy learning.

    Source: §1 · pp. 2–3

  • Results

    Across ACT and π0.5, reflex-shaped demonstrations improve in-distribution grasp quality, with the clearest evidence on ACT and favorable exploratory evidence on an unseen paper-cup setting.

    Source: §1 · p. 3, §5 · p. 9

  • Takeaways & Limitations

    Tactile feedback can shape demonstrations for tactile-free learning, while rapid disturbance correction at deployment remains a complementary role for a high-rate reflex.

    Source: §1 · p. 3, §5 · p. 9

    Evidence is limited to disposable cups, one unseen paper-cup setting, and one disturbance family; the controlled ablation has one ACT run and collection order was not randomized.

    Source: §6 · p. 9

Abstract

from arXiv · show

How should a robot learn to manipulate objects so fragile that sub-Newton contact forces can cause irreversible damage? Existing visuo-tactile policy learning typically treats tactile sensing as an additional policy input. In direct-contact force-sensitive manipulation, however, the bottleneck can arise earlier, during data collection: manual gripper control is too delayed and coarse-grained to reliably maintain the narrow force range required for stable grasping. We therefore use a deterministic 25 Hz tactile reflex controller as a collection-time teacher, producing demonstrations with controller-shaped grasping behavior for tactile-free policy learning. On Action Chunking with Transformers (ACT), policies trained from reflex-shaped demonstrations recover the teacher's grasping profile and achieve 95% stable grasps on the nominal plastic-cup task, substantially outperforming visually screened manual demonstrations. The same intervention improves in-distribution stability on $π_{0.5}$ and shows a favorable exploratory trend on an unseen paper-cup variant. Under randomized external disturbance, however, the reflex-data $π_{0.5}$ policy still fails in 45% of policy-only trials, whereas a deployment-time reflex arbiter retains all grasps. These results reveal a new role for tactile feedback in force-sensitive manipulation: rather than integrating tactile into the policy, we use it as a collection-time teacher that shapes grasping behavior in demonstrations for policy learning, while disturbance rejection remains controller-dependent, revealing the boundary of tactile-free policy.

1 Introduction

The paper argues that fragile grasping is bottlenecked by collecting repeatable demonstrations, not merely by giving learned policies tactile input. A tactile reflex shapes collection-time demonstrations for tactile-free learning, improving in-distribution grasp quality while leaving disturbance rejection dependent on runtime tactile arbitration.

1 Introduction

  • Manual gripper control can be too delayed and coarse to maintain the narrow force window needed to grasp fragile objects without irreversible deformation.Even with live tactile parameters and slip cues, operators must interpret contact and issue commands through a human observation–response–execution loop.Source: §1 · p. 2
  • The approach runs a deterministic vision-based tactile reflex during teleoperated collection, recording its regulated trajectories as demonstrations for a tactile-free student.The policy interfaces remain unchanged, and the student receives the controller’s tactile regulation only through the demonstrations.Source: §1 · pp. 2–3
  • The paper frames tactile feedback as a way to make successful contact behavior repeatable and learnable, rather than only as an extra policy modality.The work distinguishes upstream demonstration quality from tactile integration into policy learning or execution.Source: §1 · p. 2
  • Across ACT and π0.5, reflex-shaped demonstrations improve in-distribution grasp quality over visually screened manual data, while disturbance rejection still depends on runtime tactile arbitration.The evidence is clearest on ACT, extends to π0.5 in-distribution, and includes a favorable exploratory trend on an unseen paper-cup variant.Source: §1 · p. 3
  • Under randomized external perturbations, reliable grasp retention requires a decoupled high-rate gripper arbiter, marking a boundary for tactile-free policy behavior.The deployment protocol separates nominal regime learning from rapid tactile reactivity.Source: §1 · p. 3

2 Related Work

Prior work commonly treats tactile sensing as a policy modality or deployment-time reactive aid, while this paper uses a deterministic controller to shape demonstrations upstream. Its teacher-student framing is limited to collection-time transfer, not inherited tactile reactivity.

2 Related Work

  • Visuo-tactile policies typically add tactile input for insertion, tool use, or other contact-rich tasks, but do not ask whether tactile can make force-sensitive collection repeatable.DexUMI [21] and TypeTele [22] address embodiment gap, retargeting, or operator interfaces; HapticVLA [23] distills a tactile-aware token for tactile-free inference.Source: §2 · p. 3
  • Here, the teacher is an engineered deterministic controller that writes a stable nominal contact regime into demonstrations rather than adding an observation modality to the learned policy.The paper calls this collection-time transfer “distillation” interpretively; the student does not inherit full tactile reactivity or use tactile input at inference.Source: §2 · p. 3
  • Unlike deployment-time reactive modules that stabilize or constrain learned policies, this controller first changes which contact trajectories are realized and recorded during collection.At deployment, tactile arbitration returns as a decoupled gripper safety layer that supplies high-rate correction the tactile-free student did not inherit.Source: §2 · pp. 3–4

3 Method

The method uses the tactile reflex both to shape demonstrations for behavioral cloning and, optionally, to provide local gripper correction at deployment. It also tests a train-time gripper-trajectory prior and reports episode outcomes alongside tactile-regime diagnostics.

3 Method

  • The method section organizes the controller mechanism, dynamics-loss refinement, and evaluation quantities as distinct parts of the experimental approach.It formalizes collection/deployment decoupling before introducing the refinement and evaluation measures.Source: §3 · p. 4

3.1 Controller-Shaped Collection and Decoupled Deployment

  • The controller shapes demonstrations during collection and can return at deployment as a local, high-rate gripper correction module outside the tactile-free policy loop.The policy predicts arm trajectories and gripper commands in chunks, while the deterministic reflex regulates only the gripper from dual tactile sensors.Source: §3.1 · p. 4, Figure 2 · p. 4
  • In manual collection, the operator controls the gripper with coarse buttons; in reflex collection, TactileReflex [1] regulates it at 25 Hz and logged actions capture realized follower states.Collection modes retain the same object family, action interface, and sensing stack; at deployment the policy controls the arm while an arbiter can reinterpret gripper output as intent.Source: §3.1 · pp. 4–5

3.2 Dynamics-Loss Refinement for Controller-Shaped Regime Entry and Release

  • The dynamics refinement penalizes second-order temporal-difference mismatch in the gripper trajectory, biasing predictions toward the teacher’s contact-to-hold shape and transition sharpness.It is a train-time prior only, with no tactile input at inference and no change to rollout or arbiter logic.Source: §3.2 · p. 5

3.3 Episode Outcomes and Tactile-Regime Evaluation Metrics

  • Evaluation combines whole-episode stable, loose, and drop outcomes with contact-qualified and teacher-near tactile summaries of regime entry and sustained re-entry.The ≥10-frame teacher-near run is the primary regime-entry summary; tactile regime entry need not coincide with visually stable completion, and additional onset, occupancy, and release measures are reported for Experiment 2.Source: §3.3 · p. 5

4 Experiments

The experiments test whether controller-shaped collection improves tactile-free grasp learning across ACT and π0.5, then probe an unseen cup, train-time refinement, and disturbance-time arbitration.

4.1 Setup

  • The evaluation uses matched ACT data-source conditions, π0.5 ID and paper-cup OOD tests, a refinement comparison, and randomized disturbance trials.The primary controlled ACT comparison uses 30 reflex-collected episodes and 30 selected demonstrations from 100 manual trials, with 20 nominal ID trials per policy.Source: §4 · p. 6, §4.2.1 · p. 6, §4.1 · p. 6

4.2.1 Experiment 1a: ACT as the Primary Controlled Test

  • 95% (19/20) stable grasps with zero drops for reflex-data ACT versus 15% (3/20) stable and 30% (6/20) drops for visually screened manual-data ACT.The same manual-data policy reaches 50% stability with the runtime arbiter, while its drop rate remains 30%.Source: Table 1 · p. 6, §4.2.1 · pp. 6–7
  • 30% (6/20) stable outcomes for contact-screened manual-data ACT versus 95% (19/20) for reflex-data ACT, in a single training and evaluation run.Contact screening used tactile logging only for offline ranking; the signals did not actuate the gripper or enter ACT training or inference.Source: §4.2.1 · p. 7
  • The reflex-data ACT policy repeatedly recovers the teacher-defined nominal contact regime, whereas the visually screened manual-data policy rarely does; the arbiter only partly recovers it.The figure shows the reflex-data policy tracking the teacher through contact-to-hold while the manual-data policy deviates after closure.Source: Figure 3 · p. 7, §4.2.1 · p. 7

4.2.2 Experiment 1b: π0.5 Cross-Backbone ID Validation

  • 95% (19/20) stable grasps for reflex-data π0.5 versus 5% (1/20) for visually screened manual-data π0.5; adding the arbiter to the latter reaches 55% (11/20).Additional seeds preserve the same ranking across the three conditions.Source: §4.2.2 · pp. 7–8

4.2.3 Experiment 1c: Unseen Paper-Cup OOD Extension

  • 80% stable versus 30% for reflex-data and manual-data policies on the unseen paper cup, respectively; the difference is exploratory and not statistically significant.The paper cup is mechanically and visually more forgiving than the training cup.Source: Table 3 · p. 8, §4.2.3 · p. 8

4.3 Experiment 2: Controller-shaped behavior refinement with Dynamics-Loss

  • 15/20 to 18/20 teacher-near runs on ACT and 12/20 to 16/20 on π0.5, with higher onset occupancy and earlier release after dynamics-loss refinement.Additional seeds preserve the CQ-onset improvement on both backbones.Source: Table 4 · p. 8, §4.3 · p. 8

4.4 Experiment 3: Exploring the Boundary of Tactile-Free Policy

  • 9/20 policy-only disturbance trials fail, whereas reflex arbitration retains all 20/20 grasps on the same policy.The arm remains policy-controlled while the arbiter has gripper authority; the comparison isolates runtime control rather than planning.Source: §4.4 · pp. 8–9

5 Conclusion

The paper uses tactile reflex control both to shape demonstrations for tactile-free learning and, at deployment, to provide rapid correction under disturbance. Its runtime comparison evaluates policy-only and reflex-arbitrated modes on the same reflex-data π0.5 policy.

5 Conclusion

  • A deterministic tactile reflex acts as a collection-time teacher, shaping demonstrations into a more consistent training distribution for tactile-free policy learning.The resulting policies show their clearest effect on ACT, with the data-source effect also carrying to π0.5 under in-distribution conditions.Source: §5 · p. 9
  • Controller-shaped demonstrations improve in-distribution stability across ACT and π0.5 and show a favorable exploratory trend on an unseen paper-cup setting.The conclusion presents the paper-cup result as exploratory rather than definitive.Source: §5 · p. 9
  • Under external disturbance, high-rate tactile arbitration provides rapid grasp correction complementary to the tactile-free policy; Table 5 compares runtime modes on the same π0.5 policy across 20 disturbed trials per mode.The policy-only student does not inherit high-rate tactile correction under disturbance.Source: §5 · p. 9, Table 5 · p. 9

6 Limitations and Future Work

The evidence is limited to disposable cups, one paper-cup OOD setting, and one disturbance family, and the claims are strongest for fragile containers requiring a narrow force window. Collection and deployment constraints further limit the conclusions.

6 Limitations and Future Work

  • The evaluation covers disposable cups, one unseen paper-cup OOD setting, and one disturbance family, so findings are strongest for fragile containers with a narrow gripper-force window.Broader contact-rich tasks involving arm motion, object geometry, or tool interaction are not tested.Source: §6 · p. 9
  • The contact-screened ACT ablation has only one training and evaluation run, while one expert collected data in a fixed, nonrandomized order.Operator and temporal effects therefore remain possible, and the ablation is corroborative rather than a robustness result.Source: §6 · p. 9
  • Collection-time shaping does not close the deployment-time control gap once contact leaves the nominal regime.The explicit tactile controller preserves gripper-side behavior but cannot coordinate the whole arm in closed loop in real time.Source: §6 · p. 9

A Appendix-Only Notes on Naive Tactile-Image Fusion

Appendix A reports that naive tactile-image fusion did not reliably improve learning in the current setup, with outcomes differing between ACT and π0.5. Appendix B documents a deployment arbiter that changes gripper authority while leaving the arm policy-controlled.

A Appendix-Only Notes on Naive Tactile-Image Fusion

  • These appendix diagnostics concern naive direct fusion under the current data and sensing setup, not richer tactile integration methods.Raw tactile-stream noise is offered only as a plausible, unconfirmed explanation for the results.Source: A Appendix-Only Notes on Naive… · p. 14
  • Naive tactile-image fusion caused severe degradation for ACT, which often lost usable task behavior and could fail to maintain stable pick-and-place performance.This appendix-only variant added left and right tactile image streams as extra ResNet-18 inputs.Source: A Appendix-Only Notes on Naive… · p. 14
  • The π0.5 tactile-image variant retained basic task ability but showed no clear or reliable gain over vision-only in success rate or grasp quality.The appendix treats this as a result for the tested setup, not as a conclusion about tactile information generally.Source: A Appendix-Only Notes on Naive… · p. 14

B Deployment-Time Arbiter Logic

  • In policy-plus-arbiter mode, the arm remains policy-controlled while the arbiter changes gripper authority; a close intent starts the reflex, and sustained open intent can return authority to the policy.Release requires K consecutive open-intent policy ticks; the logic also suppresses immediate re-latching after release.Source: B Deployment-Time Arbiter Logic · p. 15

C Implementation Notes for the Deployed Reflex Controller

The implementation notes describe operational refinements for running the deployed reflex consistently, while emphasizing that the controller’s logic and tactile decision rules remain unchanged.

C Implementation Notes for the Deployed Reflex Controller

  • The deployed controller retains its original three-channel tactile logic and is not presented as a new algorithmic contribution.It uses weaker-side normal-contact onset detection, shear-based slip correction, and center-of-pressure over-compression relief, with normal-force protection highest priority.Source: C Implementation Notes for the… · p. 15
  • Persistent background threads acquire tactile images asynchronously, so the controller uses the latest preprocessed frames instead of blocking for fresh reads at each step.This reduces camera-latency blocking and improves loop regularity.Source: C Implementation Notes for the… · p. 15
  • Left and right tactile metrics are computed in parallel, improving wall-clock throughput without changing the processing pipeline or controller inputs.The pipeline still includes frame differencing, contact-mask construction, normal-contact estimation, optical-flow shear estimation, and center-of-pressure extraction.Source: C Implementation Notes for the… · p. 15
  • Baseline and noise calibration occur during initialization before live streaming begins.Multiple non-contact frames establish a baseline image and per-sensor contact threshold; streaming starts after calibration succeeds.Source: C Implementation Notes for the… · p. 16
  • Runtime logs record timing, loop frequency, controller state, tactile statistics, and corrective actions for debugging and rate verification.Logging is not part of the control law.Source: C Implementation Notes for the… · p. 16
  • The refinements make the fixed controller more practical to run consistently in the collection-time-teacher setting without changing what it computes.They are included as engineering details for operational repeatability.Source: C Implementation Notes for the… · p. 16

D Contact-Qualified Threshold for the Tactile Diagnostic

The appendix clarifies how tactile-regime thresholds differ from material force measurements and documents the experimental controls and implementation details. Its CQ cutoff tests for meaningful contact, while a separate teacher-referenced band tests whether contact is near the controller-shaped stable regime.

D Contact-Qualified Threshold for the Tactile Diagnostic

  • Fn,min is the weaker of two dimensionless normal-contact proxies, chosen because the weaker side better captures loss of meaningful contact.The proxy is not calibrated in Newtons; the 1.5 CQ cutoff reuses the TactileReflex grasp-onset threshold.Source: D Contact-Qualified Threshold for the… · p. 16
  • Fn,min > 1.5 marks a contact-qualified frame, excluding incidental light touch; it is not the teacher-near operating range.The threshold asks whether meaningful grasp contact has begun, while teacher-nearness is assessed separately.Source: D Contact-Qualified Threshold for the… · p. 16
  • Teacher-nearness uses the stable-hold reference band of 2.41 ± 0.7, after the CQ filter; a stricter p10–p90 band is only an appendix alternative.The two stages distinguish meaningful contact from contact near the teacher-shaped stable regime.Source: D Contact-Qualified Threshold for the… · p. 16
  • Visible cup deformation appeared near 0.5 N, and sustained loading below 1 N could cause irreversible deformation, but these loads do not calibrate the tactile proxy.The external-load observations characterize the cup material, not Fn or either tactile threshold.Source: D Contact-Qualified Threshold for the… · p. 16

E Additional Experimental Details

  • The appendix records supporting experimental details on controls, metric roles, failure modes, and statistical notes.It groups these details into setup, comparison design, robustness, and supplementary Experiment 1 notes.Source: E Additional Experimental Details · p. 17

E.1 Platform, Models, and Policy Variants

  • Main tactile-free π0.5 policies use front and wrist RGB streams plus a 7-D robot state; the tactile reflex operates independently at 25 Hz.ACT uses an 18M policy with a 15 Hz full policy-update loop; the main deployed π0.5 uses a 2B+ VLA and executes action steps at 20 Hz under a 10-step full-chunk synchronous rollout.Source: §E.1 · p. 17

E.2.1 Matched-Comparison Controls

  • Manual and reflex comparisons use the same expert, platform, and visual/tactile GUI access, but only reflex collection closes a direct 25 Hz tactile–action loop.The manual baseline was visually screened from 100 trials and matched to 30 demonstrations; its tactile traces were not retained for post-hoc contact screening.Source: §E.2.1 · pp. 17–18
  • Reflex data were collected before manual data without randomized order, leaving temporal effects possible despite practice tending to favor the later manual pool.The main comparison is therefore against a visually screened best-effort manual baseline, not a symmetric contact-screened comparison.Source: §E.2.1 · p. 18

E.2.2 Contact-Screened Controlled Ablation

A contact-screened manual-data ablation tests whether selecting demonstrations by tactile contact quality can match reflex-shaped collection. The appendix also explains how episode outcomes and tactile diagnostics answer different questions.

E.2.2 Contact-Screened Controlled Ablation

  • 95% vs. 30% stable outcomes differed significantly (two-sided Fisher exact p = 3.9 × 10−5) between reflex-data ACT and contact-screened manual-data ACT.The contact-screened set was the top 30 of 100 fresh tactile-logged manual demonstrations ranked by teacher-referenced CQ-near occupancy.Source: §E.2.2 · p. 18, Table 7 · p. 18

E.2.3 Metric Roles and Failure Taxonomy

  • Stable, loose, and drop are video-based whole-episode outcomes, whereas CQ episodes and teacher-near summaries measure tactile-regime entry and occupancy.The failure taxonomy distinguishes no meaningful contact, unstable hold, delayed release/place hesitation, and disturbance-induced drop.Source: §E.2.3 · pp. 18–19
  • The onset-window CQ metric focuses on early teacher-near entry, while full-horizon occupancy can decrease when release becomes more efficient.Time to release measures task start to release onset; release-time reduction is relative to the matched reflex-data baseline.Source: §E.2.3 · p. 18

E.2.4 Statistical Notes

  • Table 8 reports Wilson confidence intervals and two-sided Fisher exact tests for the same single-run binary outcomes summarized in the main paper.The statistical notes do not introduce additional outcome runs.Source: Table 8 · p. 19, §E.2.4 · p. 19

E.3 Additional-Seed Robustness Tables

The appendix gives additional-seed protocols and supplementary scope notes for the π0.5, paper-cup, and ACT evaluations. It treats the paper-cup comparison as exploratory and flags sparse tactile diagnostics and post-grasp stalling as boundaries on interpretation.

E.3 Additional-Seed Robustness Tables

  • Robustness checks retrained each variant with four additional seeds under the same data, recipe, and evaluation protocol, changing only the training seed.Experiments 1a, 1b, and 3 used 20 rollouts per additional seed; the fresh contact-screened ablation had no additional seeds.Source: §E.3 · p. 19, Table 9 · p. 19, Table 10 · p. 20

E.4.1 π0.5 Tactile Corroboration

  • Figure 5 presents π0.5 teacher-referenced tactile corroboration for the ID comparison, including a manual-data policy plus arbiter bridge ablation.The figure caption identifies a policy-only comparison against the teacher reference and a compact summary of the same diagnostic.Source: Figure 5 · p. 21, §E.4.1 · p. 21

E.4.2 Paper-Cup Scope Notes

  • 80.0% stable [49.0, 94.3] vs. 30.0% [10.8, 60.3] on the unseen paper cup was not statistically significant (p = 0.0698), so the result is exploratory OOD evidence.The cup has a larger force margin, higher friction, and clearer visual boundaries than the training plastic cup.Source: §E.4.2 · p. 21, Figure 6 · p. 22

E.4.3 Sparse-CQ and ACT Boundary Notes

  • The 12.5% manual ACT teacher-near value comes from one CQ episode with eight CQ frames, while the 39.4% arbiter value is conditional occupancy, not episode success.The updated arbiter row corresponds to 10/20 CQ episodes and 7/20 sustained teacher-near runs; the reflex-data ACT policy can grasp the unseen cup but often stalls after grasp.Source: §E.4.3 · p. 21, Figure 6 · p. 22

F Dynamics-Loss Refinement Notes

The appendix interprets the dynamics-loss refinement narrowly: it is associated with stronger early regime entry and shorter release time, not uniformly higher tactile summaries or task outcomes.

F Dynamics-Loss Refinement Notes

  • Full-horizon CQ-near occupancy decreases from 68.7% to 54.8% on ACT and from 70.8% to 61.2% on π0.5 with dynamics loss, despite its focus on earlier entry and release.Earlier teacher-near entry and earlier release can improve onset-window measures while reducing later stable dwell time.Source: F Dynamics-Loss Refinement Notes · p. 22
  • The refinement is a scoped train-time result: gains are not uniform across summaries or task outcomes, and manual-data training, additional backbones, and a unified task-outcome protocol remain untested.The authors read it as stronger early regime entry and shorter time to release within the controller-shaped regime.Source: F Dynamics-Loss Refinement Notes · p. 22
Loading 2609.25887v1…