PaperlayerView exact revision on arXiv

Source-linked AI summary

See to Reach, Feel to Grasp: Learning A Blind Grasp Reflex for Anthropomorphic Robotic Hands

Alexander Alexiev, Tzu-Yuan Lin, Sang Min Kim, Ho Jae Lee, Yonghyeon Lee, Sangbae Kim

arXiv:2609.31323v1cs.RO

TL;DR

Dexterous grasping requires adapting localized contacts, while geometry-conditioned control can depend on object shape, pose, and scene configuration. The paper separates arm reaching from a proprioceptive hand policy and uses a learned grasp score to signal readiness for manipulation. It reports high simulated grasp success, score alignment with a physics-based stability measure, and hardware reuse across objects and unseen arm controllers.

  • Problem

    Dexterous grasping depends on localized contact configurations, while geometry-conditioned contact control may depend on object shape, pose, and scene configuration.

    Source: §I · p. 1

  • Method

    An independently controlled arm approaches the object while a hand policy learns grasping and stabilization from proprioceptive history and signals manipulation readiness with a learned grasp score.

    Source: §I · pp. 1–2, A. Learned Hand Module · p. 3, 2) Distillation to a Blind Student · p. 3

  • Results

    96% of YCB and 95% of 3028 randomly sampled GraspXL objects were grasped in simulation; the learned score correlated with Ferrari–Canny ϵ margin at ρ̄ = +0.77.

    Source: §I · p. 2

  • Takeaways & Limitations

    The frozen hand policy was demonstrated on 29 real-world objects and reused with three unseen arm behaviors, supporting composition with independently designed arm controllers.

    Source: §V · p. 8

Abstract

from arXiv · show

In this work we study if a robotic hand using proprioception alone can grasp diverse objects with no visual observation. We present a modular dexterous grasping architecture that separates global arm motion from local contact control. An independently controlled arm guides the hand toward the object, while a reinforcement learning policy grasps and stabilizes it using only hand proprioceptive feedback. We call this \textit{a blind grasp reflex}: grasping without images, object poses, or geometric observations. A learned stable-grasp score determines when the object is securely held, allowing the arm to begin post-grasp manipulation. This separation makes grasping a reusable hand-level skill that can be combined with independently designed arm controllers for various manipulation tasks. Experiments in simulation and on hardware demonstrate robust blind grasping across diverse objects and seamless composition with a range of arm controllers. Moreover, despite never observing contact geometry, the learned grasp score closely aligns with an independent physics-based measure of grasp stability. The resulting approach follows a simple principle: see to reach, feel to grasp. Project page: https://blindgraspreflex.github.io.

I. INTRODUCTION

Dexterous hands must adapt localized contacts, but fixed closing and geometry-conditioned control can be brittle under uncertainty. The paper separates arm-level reaching from proprioceptive hand control and uses a learned grasp score to coordinate manipulation; experiments report strong simulated success and hardware reuse across objects and arm controllers.

I. INTRODUCTION

  • Fixed-configuration closing offers limited contact adjustment after reaching errors or disturbances, while geometry-conditioned local control may depend on object shape, pose, and scene configuration.Dexterous grasp quality depends on localized contact configurations and force distributions that are difficult to estimate without dense tactile sensing.Source: §I · p. 1
  • The architecture assigns global geometric reasoning to the arm and local contact control to a hand policy intended to reuse stabilization across objects, poses, and scenes.The arm approaches a feasible grasp while the hand can respond as contacts emerge and accommodate residual reaching errors.Source: §I · p. 1
  • A learned grasp score signals when the grasp is stable enough for the arm to begin manipulation, while the hand remains active to maintain contact and stability.The score addresses the transition timing problem posed by contacts developing differently across attempts.Source: §I · p. 2
  • 96% of YCB and 95% of 3028 randomly sampled GraspXL objects were grasped in simulation; the learned score correlated with Ferrari–Canny ϵ margin at ρ̄ = +0.77.Hardware demonstrations covered 29 real-world objects and three arm controllers unseen during training.Source: §I · p. 2
  • The work learns a hand-local policy using joint encoders as its only sensory input, pairs it with independently designed arm controllers, and presents qualitative hardware demonstrations on diverse objects without providing geometry to the deployed policy.Source: §I · p. 2

II. RELATED WORKS

Vision-based methods use geometric observations or jointly learned arm–hand control, while blind grasping has also used compliance and tactile sensing. This work instead uses hand-local proprioception and an independently controlled arm, with a learned signal for grasp readiness.

A. Vision-Based Dexterous Grasping

  • Vision-based approaches condition grasp policies on point clouds, object state, or image-derived geometry, and some jointly control arm and hand.Geometry representations can be hand-centric, but still depend on externally perceived geometry; joint control can also make hand behavior less reusable with independent reaching controllers.Source: A. Vision-Based Dexterous Grasping · p. 2
  • This work controls only the hand from local sensory history, without visual or object-geometric observations, while leaving arm motion independent.Visual perception may still guide the arm as the hand adapts to contact through local feedback.Source: A. Vision-Based Dexterous Grasping · p. 2

B. Blind and Proprioceptive Grasping

  • A standard rigid-contact hand grasps without vision using joint encoders alone; joint-position and action-command history lets the policy reason implicitly about contact without tactile, force, or torque measurements.This setup requires no additional contact sensors and sidesteps transferring tactile observations from simulation to hardware.Source: B. Blind and Proprioceptive Grasping · p. 2
  • The policy learns a scalar grasp-readiness signal without direct stability supervision, communicating when manipulation can begin as part of its action space.This combines contact reasoning and readiness signaling in one proprioceptive policy.Source: B. Blind and Proprioceptive Grasping · p. 2

III. METHODS

The framework separates arm motion from local hand control through an engage command and a learned grasp score. The hand policy is distilled from a privileged simulation teacher into a student that operates from joint-position history alone.

III. METHODS

  • The arm generates palm poses and an engage command, while the hand generates finger targets and a grasp score for manipulation timing and continued contact adaptation.A low-level controller converts outputs to joint commands and limits arm and hand velocities.Source: §III · p. 2, Figure 2 · p. 3
  • The frozen hand policy is reused at deployment with alternative arm controllers after training with a fixed two-phase model-based arm controller.Its observations exclude arm state, kinematics, and external perception, although arm motion affects behavior through contact dynamics.Source: §III · p. 3

A. Learned Hand Module

  • Training first uses a privileged teacher with simulated contact and object-state information, then distills a blind student to reproduce teacher actions from joint-position history alone.Both policies share an action space, and the hand module is the only learned controller in the training setup.Source: A. Learned Hand Module · p. 3
  • The hand target blends raw joint outputs toward a low-dimensional PCA manifold derived from retargeted human grasp postures.The blending weight controls prior strength; β = 1 constrains targets to the manifold.Source: A. Learned Hand Module · p. 3

1) Privileged Teacher:

The privileged teacher learns grasping from simulated contact and object-state information, then a blind student is distilled to reproduce its actions from proprioceptive history. The student uses joint measurements and its previous command without tactile, force, torque, or object-state measurements.

1) Privileged Teacher:

  • The privileged teacher learns with RL from simulated hand-link contacts, palm-frame object position, and hand joint angles.An asymmetric critic also observes the object’s world position and velocity.Source: 1) Privileged Teacher · p. 3

2) Distillation to a Blind Student:

  • The blind student predicts the teacher’s joint targets and grasp score from measured joint angles and the previous position command.It is distilled with DAgger on states reached by the student.Source: 2) Distillation to a Blind Student · p. 3
  • Joint-position history and the previous command provide implicit contact cues: contact can slow or stop a finger, creating a persistent target-to-measurement discrepancy.The student learns to interpret these cues over time and adjust its grasp under teacher supervision.Source: 2) Distillation to a Blind Student · p. 3

B. Implicit Learned Grasp Score

The hand outputs a grasp score that connects local grasp control to the arm’s decision to lift or begin later manipulation. A compact arm interface separates the hand’s grasping from arm motion and permits compatible controller substitutions without retraining the hand policy.

B. Implicit Learned Grasp Score

  • During training, score thresholds select reaching or lifting, and a low score can return the arm to reaching.The arm retains its previous phase between the low and high thresholds.Source: B. Implicit Learned Grasp Score · p. 4
  • The teacher learns the grasp score through task-reward consequences, without explicit stability labels or an auxiliary classification loss.Lifting too early can reduce height and hold rewards, while delaying a viable lift postpones them; the student then learns to approximate the teacher’s signal from proprioceptive history.Source: B. Implicit Learned Grasp Score · p. 4

C. Arm Module

  • The arm module receives the grasp score and outputs a desired palm pose and engage signal; the hand does not observe the arm’s internal state.The engage signal selects a fixed splayed pre-grasp posture or activates the learned hand command.Source: C. Arm Module · p. 4
  • The interface supports pairing the frozen hand policy with compatible scripted, planned, reactive, teleoperated, and vision–language arm controllers without retraining.Compatibility requires the specified signal exchange and arm motions that let the hand establish and maintain contact.Source: C. Arm Module · p. 4

D. Low-Level Velocity Controller

The low-level controller converts desired arm and hand targets into actuator commands while exposing tracking gains and velocity limits for hardware tuning. The hand command update uses the previous command, allowing tracking error to persist when contact stops a finger.

D. Low-Level Velocity Controller

  • The low-level controller tracks the arm’s desired palm pose and the hand’s joint targets, exposing gains and velocity limits for hardware tuning while keeping the hand policy fixed.Both arm and hand commands use PD feedback with gravity compensation.Source: D. Low-Level Velocity Controller · p. 4
  • On hardware, a quadratic program tracks the desired arm twist subject to joint-position, velocity, and linearized collision-avoidance constraints.During training, damped least-squares inverse kinematics maps the twist to joint velocities.Source: D. Low-Level Velocity Controller · p. 4
  • The hand’s velocity command is constructed from the policy’s joint target with a speed limit and slows near the target.The command is updated from the previous command rather than measured joint position, so contact-stopped fingers can retain tracking error and produce servo effort.Source: D. Low-Level Velocity Controller · p. 4

E. Policy Training

Policy training combines coverage, height, and hold rewards, and trains a privileged teacher with PPO before distilling a blind student with DAgger. The hold reward encourages sustained grasps near an elevated goal.

E. Policy Training

  • The task reward weights contact coverage, object height, and hold terms, with rewards scaled by the control period.Coverage rewards establishing and maintaining inner-hand contact; height provides a dense incentive to raise the object, and hold rewards time near an elevated goal.Source: E. Policy Training · pp. 4–5
  • The teacher is trained with PPO using a recurrent actor and asymmetric critic to integrate contact and motion information over time.Both use a single-layer GRU followed by an MLP.Source: E. Policy Training · p. 5
  • The frozen teacher is distilled with DAgger while the student acts, training it on states reached by its own actions and teacher-provided targets.The student minimizes mean-squared error over the full action vector, including the grasp score.Source: E. Policy Training · p. 5

IV. EXPERIMENTS

Experiments evaluate a proprioceptive hand policy on a torque-controlled arm, using randomized simulation training and a grasp protocol that tests both changing object positions and unseen locations.

A. Experimental Setup

  • The hardware setup pairs a 20-DoF anthropomorphic hand with a 7-DoF torque-controlled arm, while the hand policy runs at 11.9 Hz.The low-level controller runs at 1 kHz and the arm module at 200 Hz.Source: A. Experimental Setup · p. 5
  • Teacher and student policies train in 4096 parallel MuJoCo Warp worlds, with 4 ms physics steps and 84 ms policy control periods.Each episode lasts 10 seconds.Source: A. Experimental Setup · p. 5
  • Training randomizes YCB object identity, scale, mass, pose, palm offset, external velocity disturbances, and hand velocity limits and servo gains.The full YCB object set is used at several uniform scale factors.Source: A. Experimental Setup · p. 5
  • Each method gets 30 seconds to grasp, with success defined as holding the object within 10 cm of its target above ground for 2 seconds.Evaluation also changes object location during execution and tests generalization to locations outside the training grasp area.Source: A. Experimental Setup · p. 5, Figure 3 · p. 5

B. Baselines

The experiments compare blind grasping against scripted closure, vision-based end-to-end control, and a fixed-timer variant. The modular method is comparable on static grasps and retains performance across changing locations and a qualitative test of real-object diversity.

B. Baselines

  • The three alternatives are manually tuned scripted closure, joint arm-hand vision-based RL, and the same hand policy with a fixed 5-second lift timer.The end-to-end policy uses object point clouds, updated after object motion in dynamic evaluation.Source: B. Baselines · p. 5

C. Grasping Performance

  • The end-to-end baseline leads on static grasping, but degrades on dynamic objects and outside the training workspace; our method stays within a few static-success points while retaining both.The table caption identifies these baseline comparisons, and the results describe broader workspace coverage for the factorized method.Source: Table II · p. 6, C. Grasping Performance · p. 6
  • The frozen hand policy successfully grasped all 29 tested real-world objects without giving the hand policy object geometry.This was a qualitative object-diversity test, not a repeated-trial success-rate benchmark.Source: C. Grasping Performance · p. 6

D. Grasp Score Analysis

The learned grasp score tracks a contact-derived measure of grasp quality despite having no contact-based supervision. The same frozen hand policy also works with unseen arm controllers, mounting configurations, platforms, and a collision-avoiding controller.

D. Grasp Score Analysis

  • On successful grasps, the learned score and Ferrari–Canny ϵ remain high after lift; on failures, both decrease as the grasp deteriorates.The score is neither supervised by nor given the contacts used to compute ϵ.Source: D. Grasp Score Analysis · p. 7, Figure 6 · p. 7
  • Across episodes, grasp score and Ferrari–Canny ϵ have Spearman correlation ¯ρ = +0.77 overall, +0.78 on successes, and +0.61 on failures.The score is not calibrated to ϵ's scale, so rank correlation measures their monotonic association.Source: D. Grasp Score Analysis · p. 7

E. Composing with Different Arm Modules

  • The frozen hand policy works with three unseen arm behaviors, including language-guided pick-and-place, reactive tracking, and a lightbulb routine with sustained wrist rotations.The arm engages or releases the grasp, while the grasp score signals readiness for subsequent manipulation.Source: E. Composing with Different Arm Modules · p. 7
  • The same policy grasps with unseen mounting configurations and different robot arms, despite changes to upstream kinematics, mounting geometry, and control behavior.These tests do not retrain the hand policy.Source: E. Composing with Different Arm Modules · p. 7, Figure 7 · p. 7
  • At deployment, a QP controller performs online collision avoidance while satisfying task-space velocity commands, and the arm grasps successfully with three obstacles.The QP considers link-to-obstacle distances and joint limits, replacing the training-time pseudo-inverse velocity controller.Source: E. Composing with Different Arm Modules · p. 7, Figure 8 · p. 7

F. Ablations

Ablations examine projection strength, student observations, and hand-control parameters. The conclusion presents grasping as a reusable hand-level skill that can work with independently designed arm controllers.

F. Ablations

  • The PCA projection coefficient β = 0.5 has the highest success rate among tested values, while allowing joint targets to refine the projected pose.This can compensate for poor retargeting between the human and robot hands.Source: F. Ablations · p. 7, Table III · p. 8
  • The q − qcmd observation performs best, and the policy retains high success when hand stiffness and maximum velocity vary after training.Slower maximum velocity is substantially out of distribution and takes much longer to grasp.Source: F. Ablations · pp. 7–8

V. CONCLUSION

  • The paper concludes that the frozen hand policy can be reused with unseen arm behaviors, mounting configurations, and robot platforms.The reported hardware demonstrations include 29 real-world objects and three previously unseen arm behaviors.Source: §V · p. 8

Related papers

Loading 2609.31323v1…