PaperlayerView exact revision on arXiv

Source-linked AI summary

Blind Dexterity: Whole-Body Humanoid Manipulation via Pure Proprioception

Aditya Bhatt, Oleg Kaidanov, Puze Liu, Jan Peters

arXiv:2608.29487v1cs.RO

TL;DR

The paper asks whether whole-body humanoid manipulation can work with only proprioception and no dedicated exteroceptive or tactile sensing. It trains contact-rich policies and separate estimators, finding broad blind manipulation capabilities and increasingly decodable object state after informative contact.

  • Problem

    The paper investigates whether manipulation with spatial uncertainty can use only onboard proprioception and compliance, and how much object state becomes decodable after contact.

  • Method

    Policies use joint encoders, compliant actuation, and short proprioceptive histories, while separate task-specific estimators decode object features from those histories.

  • Results

    Across qualitatively different tasks, proprioception-only policies perform blind walking, football trapping, skateboard mounting, and suitcase-handle lifting, with object-state estimation improving after informative contact.

  • Takeaways & Limitations

    Joint encoder-based proprioception with compliant actuation is a practical substrate for blind whole-body loco-manipulation and interactive perception.

  • Takeaways & Limitations

    Contact sensing has extremely low spatial resolution, and hardware transfer depends on simulation and real-world actuator, friction, and compliance behavior matching closely.

Abstract

from arXiv · show

We present blind, whole-body manipulation skills on a Unitree G1 humanoid using only onboard proprioception, without cameras, markers, force-torque, or tactile sensors. Despite this minimal sensing, the trained policies exhibit surprising capability across qualitatively different tasks: push-resilient bipedal walking without IMU feedback, active soccer ball trapping with a foot, seeking and lifting a suitcase by its handle, and mounting a randomly positioned skateboard. We argue that these capabilities arise from a key underappreciated signal: the way the joint encoder readouts evolve under purposeful compliant contact, effectively forming a whole-body tactile channel. By generating contact-rich motions, the trained policies actively probe the environment; as a result, task-relevant object state (e.g., pose) becomes increasingly decodable from short proprioceptive histories. We expose this information using compact task-specific state estimators trained alongside, but fully separately from, the policies; their prediction errors decrease rapidly after informative contact. Our results indicate that joint encoder-based proprioception, combined with compliant actuation (now widely available on commercial robots and low-cost motors) is already a strong, practical substrate for whole-body dexterous manipulation and interactive perception, and therefore a natural foundation on which richer sensing can be layered.

I. INTRODUCTION

The paper asks whether humanoids can perform manipulation using only proprioception and shows that compliant encoder signals support blind, contact-rich skills and interactive perception.

  • The study tests whether humanoids can manipulate and perceive objects using internal proprioception instead of cameras or dedicated tactile and force sensors.
  • On a Unitree G1, proprioception-only policies perform push-resilient walking, football trapping, skateboard mounting, and suitcase-handle lifting.
  • Auxiliary estimators show that task-relevant object features become decodable from short proprioceptive histories after contact.
  • Previous joint targets let compliant tracking deviations expose torque and contact information through ordinary encoder signals.
  • The work positions encoder-based contact signals as a complement to vision during occlusion and poor lighting, rather than as a replacement for richer sensing.
  • Unlike specialized tactile sensors, the approach reinterprets compliant joint-space deflections as an informative sensing channel for interactive perception.

III. SYSTEM AND PROBLEM FORMULATION

The system studies partially observable contact-rich tasks in which a Unitree G1 policy uses onboard proprioception and previous actions while object state remains latent.

  • III. SYSTEM AND PROBLEM FORMULATION: The Unitree G1 supplies IMU readings and encoder-reported joint positions and velocities, while cameras, depth, force/torque, and tactile sensors are excluded.
  • III. SYSTEM AND PROBLEM FORMULATION: Pure proprioception denotes onboard internal sensing, whereas encoder-only variants additionally remove inertial measurements.
  • A. Problem Statement: Each task is formulated as a partially observable Markov decision process with latent object pose and physical parameters.
  • A. Problem Statement: The actor observes onboard proprioception and previous actions, then maps their history to actions that maximize expected return on contact-rich tasks.
  • A. Problem Statement: The study asks whether substantial spatial uncertainty can be handled without exteroceptive or dedicated tactile sensing.
  • A. Problem Statement: It also asks how much task-relevant object state becomes decodable after contact and whether explicit estimator feedback improves control.

A. Reinforcement Learning Formulation

Policies are trained with reinforcement learning to map short proprioceptive histories to desired joint positions, using encoder residuals during compliant contact as an implicit signal.

  • A. Reinforcement Learning Formulation: Each policy maps a short history of recent proprioceptive observations and actions to actions through a feedforward MLP trained with deep reinforcement learning.
  • A. Reinforcement Learning Formulation: The reward combines task completion, balance, smooth motion, safety shaping, and penalties for excessive joint velocities and joint-limit violations.
  • A. Reinforcement Learning Formulation: The actor can recover joint-tracking residuals from desired and measured positions, with velocity-dependent terms in the residual dynamics.
  • A. Reinforcement Learning Formulation: External contact changes these residual dynamics, providing a noisy and spatially coarse proxy for contact-induced joint loading without direct force sensing.
  • A. Reinforcement Learning Formulation: Domain randomization varies masses, friction, and motor strengths to improve robustness to modeling errors and unmodeled contacts during hardware transfer.
  • A. Reinforcement Learning Formulation: Policies operate at 50 Hz, while object-interaction histories use K = 5 and locomotion histories use K = 1.
  • A. Reinforcement Learning Formulation: Asymmetric actor–critic training gives the critic privileged simulator state that is withheld from the deployed actor, using PPO optimization.

B. Variable Stiffness Control

For suitcase tasks, variable stiffness control changes per-joint gains to support softer contact search and stiffer insertion and lifting.

  • B. Variable Stiffness Control: Suitcase +VS policies output per-joint gain multipliers α_t through a sigmoid parameterization.
  • B. Variable Stiffness Control: Lower gains amplify contact-induced tracking deviations, while the gain schedule supports softer search and stiffer insertion and lifting.

C. Proprioceptive State Estimation

The study trains separate, task-specific estimators to decode latent object properties from short proprioceptive histories, while keeping estimator supervision independent from policy learning. These estimators use simulator-recorded state targets and are diagnostic rather than category-level perception systems.

  • Estimator design: Auxiliary estimators map short proprioceptive histories to task-specific latent object features, such as object pose relative to the robot base.Ground-truth state features are available during training as regression targets.
  • Estimator design: Each estimator is trained as a separate supervised network alongside the policy, using L2 regression on simulator trajectories.Its optimizer is synchronized with the adaptive PPO learning rate, but estimator loss remains outside the PPO objective.
  • Training procedure: Inputs and regression targets are standardized per dimension to zero mean and unit variance to keep optimization well-conditioned.The targets are simulator-recorded state feature vectors.
  • Scope and interpretation: The estimators assume the object family used during training and do not claim category-level perception or generalization to unseen geometries.This defines the main scope boundary of the reported state-estimation approach.
  • Scope and interpretation: In -SE variants, the estimator is diagnostic only; in +SE variants, its previous prediction is appended to policy observations as an ablation.No estimator-loss gradients reach the policy.

V. EXPERIMENTS

The experiments evaluate blind proprioceptive control across progressively demanding tasks, combining quantitative simulation studies with qualitative real-robot validation. Locomotion ablations show that proprioception can support push recovery and inertial estimation, while previous commanded actions are important for tracking and estimation.

  • Experimental scope: The empirical study spans four tasks designed to expose different facets of the encoder-as-tactile hypothesis, with simulation evaluation and real-robot qualitative validation.The tasks progressively increase demands on blind proprioceptive control.
  • Experimental scope: Manipulation experiments use standard locomotion-policy observations including projected gravity, IMU angular velocity, joint states, and previously commanded joint targets.Real-world photos and videos document representative Unitree G1 episodes.
  • IMU-free walking and push recovery: All locomotion policies learn in simulation and transfer to the real robot; the IMU-free policy remains less robust than the IMU-equipped policy but recovers from pushes.The study presents this as a real-world demonstration using only joint-encoder measurements.
  • IMU-free walking and push recovery: Under a large velocity impulse at 5s, the IMU-equipped policy deviates least, the IMU-free policy recovers with larger deviation, and the policy without previous actions fails.The proprioceptive estimator reconstructs inertial measurements with reasonable accuracy for the IMU-free policy.
  • Observation ablations: Removing previous actions greatly reduces both velocity-tracking performance and inertial state-estimation ability, despite IMU-free policies retaining push resistance.Table I compares push recovery, velocity tracking, and inertial state-estimation mean-squared error across observation variants.
  • Observation ablations: Commanded and measured joint positions and velocities encode body-force information, enabling sufficient orientation and inertial-state inference for IMU-free walking.The relevant signal includes contact-direction information from the feet.

B. Active Football Localization and Trapping

The football policy learns to locate and trap randomly placed balls through active foot-based exploration using proprioception alone. Informative contact rapidly improves separately estimated ball position, supporting successful trapping.

  • Task setup: The robot localizes a football sampled within a 0.3 m×0.4 m rectangle and traps it under one foot using touch at the foot.Episodes last 6s and reward placing the right foot above the ball while maintaining foot–ball contact.
  • Interactive perception: At first contact around 0.73s, the estimated ball position improves rapidly as repeated taps stop the ball from rolling away.The representative episode reaches localization error below 2cm while the ball becomes firmly trapped.
  • Active seeking: The blind policies differ from the distilled student by learning to sweep and wiggle one foot, then turn toward and trap the ball after contact.The distilled student instead plants its foot near the spawn-distribution center and succeeds mainly when the ball appears nearby.
  • Interactive perception: A 0.1 s proprioceptive history can suffice for accurate ball localization because purposeful contacts encode object position in joint-encoder readings.The estimator is used to expose information available to the policy, while the policy itself learns contact-rich behavior.
  • Policy comparison: State-estimate feedback is seed-sensitive: the +SE policy performed slightly worse on average than -SE, although its best seed used estimate feedback.Poor early estimates could distract or bias +SE exploration, whereas -SE is not affected by that bias.

C. Blind Skateboard Mounting

The skateboard task requires blind mounting under randomized position and yaw, and the blind policies solve it by actively probing the board. Seeking, nose/tail contacts, and dragging progressively reduce pose-estimation error.

  • Task setup: The skateboard center is sampled from a 0.2 m × 0.5 m rectangle with yaw over a ±22.5◦ range, and success requires both feet to remain on the deck for at least 2 s.The board may roll and slide, while the estimator predicts its 3D position and continuous 6D pose representation.
  • Results: Blind policies learn active contact strategies that coincide with progressive reductions in separately estimated skateboard pose error.These strategies include seeking, nose/tail contact, and dragging.
  • Comparison: Unlike prior skateboard approaches described here, this method does not require a foot attached to the deck or an initially known skateboard pose.The approach instead interactively perceives the board before mounting.

D. Gentle Blind Manipulation (Suitcase)

The suitcase task tests gentle blind exploration of a recessed handle under randomized object and table conditions. Blind policies usually find the handle, while variable stiffness reduces tipping during exploration.

  • Task challenge: The robot must locate a recessed suitcase handle and lift the suitcase without tipping it, making gentle exploration important.A variable-stiffness variant adapts arm stiffness during exploration.
  • Interactive perception: Table contact first localizes the suitcase vertically, side contacts improve its complete pose estimate, and repeated hand searches locate the handle opening.The figure describes these stages across episodes with different suitcase sizes, positions, and table heights.
  • Task setup: The suitcase varies across a 0.2 m × 0.2 m planar region, ±45◦ yaw, table height, height, and mass.Success requires aligning with the handle, establishing contact, and lifting the suitcase clear of the table.
  • Results: The privileged policy performs best and the distilled policy worst, while the other blind policies find the handle in the majority of cases.The distilled student frequently knocks the suitcase over while imitating direct handle-reaching behavior.
  • Ablations: Variable stiffness consistently reduces tipping by ∼4–5 percentage points, whereas state-estimate feedback provides no consistent benefit.The +VS+SE variant achieves the best single-seed success of 90.3% and the least estimation error.

VI. DISCUSSION AND LIMITATIONS

Across object-interaction tasks, blind-from-scratch policies outperform distilled students, while explicit estimator feedback is inconsistent and informative contacts improve decodability. The sensing channel remains low-resolution and hardware transfer is only qualitatively evaluated.

  • Cross-task findings: Blind-from-scratch policies reliably outperform distilled students across object-interaction tasks, which struggle during pre-contact exploration.The authors link this pattern to the absence of object-influenced proprioception before contact.
  • Cross-task findings: Explicit state-estimator feedback provides no consistent mean improvement because its effects are small or seed-sensitive across football, skateboard, and suitcase tasks.Short raw proprioceptive histories often suffice for control, while inaccurate early estimates can hinder optimization.
  • Cross-task findings: Estimation errors drop systematically after informative contacts, showing that task-relevant object state is decodable from encoder-based histories without rewarding estimator accuracy.This pattern holds within the task-specific object families considered.
  • Limitations: Encoder-based contact sensing has extremely low spatial resolution, and different contact configurations can produce similar deflection patterns.Precise contact localization is therefore challenging compared with dense tactile skins.
  • Limitations: Hardware trials establish transfer feasibility rather than reliability because demonstrations were not collected under a fixed protocol.PD dynamics, friction, and compliance mismatches can alter the relationship between commands, encoder residuals, and contact.
  • Limitations: Fixed-window MLPs cannot integrate evidence across distant contacts, while MSE-trained estimators collapse ambiguous beliefs to point estimates.The authors identify longer-memory architectures and probabilistic estimators as possible alternatives.

VII. CONCLUSION

The study shows that joint encoder proprioception combined with compliant actuation can support blind whole-body loco-manipulation and interactive perception on a commercial humanoid robot.

  • Joint encoder-based proprioception with compliant actuation provides a practical substrate for blind whole-body loco-manipulation.
  • Blind policies actively probe the environment, making task-relevant object state increasingly decodable from short proprioceptive histories.
  • Across object-interaction tasks, policies trained from scratch consistently outperformed distilled students that exploit initial object-position distributions instead of learning active search.
  • Encoder feedback is presented as a strong baseline for platforms with compliant actuation and a foundation for layering vision or dedicated tactile hardware.
Loading 2608.29487v1…